Abstract
Purpose: To address the shortage of physicians practicing in rural areas of Michigan, the Wayne State University School of Medicine developed an integrated rural core curriculum to interest students in rural practice careers. Here we focus on the evaluation strategy used to determine the extent to which students in the new rural medicine interest group who self-identified as selecting a rural clerkship or externship did secure a clinical training experience in a rural setting.
Methods: Three measures of rurality were compared to determine whether students were placed in rural training settings: (1) the percentage of the county living in rural areas; (2) a county-level dichotomous measure of rural/nonrural; and (3) a dichotomous measure based on urban area boundaries within the county. Practice address and geographic data were integrated into geographic information systems software, which we used to map out rural characteristics of Michigan counties through a process called thematic mapping; this shows characteristic variation by color-shading geographic features. In addition, reference maps were created showing the boundaries of urban areas and metropolitan/micropolitan areas. Once these processes were completed, we overlaid the practice location on the contextual-level geographic features to produce a visual representation of the relationship between student placement and rural areas throughout the state.
Results: The outcome of student placement in rural practices varied by the definition of rural. We concluded that, although students were not placed in the most rural areas of Michigan, they received clerkship or externship training near rural areas or in semirural areas.
Conclusion: This process evaluation had a direct impact on program management by highlighting gaps in preceptor recruitment. A greater effort is being made to recruit physicians for more rural areas of the state rather than urban and semirural areas. Geographic information systems mapping also defined levels of ruralism for students to help them make informed selections of training sties. This is especially important for students who are not sure about a rural experience and might be discouraged by placement in a remote rural area.
The shortage of physicians in rural America is one of our most pressing health care issues. Rural communities are vastly underserved when compared with their urban counterparts.1 Factors contributing to this disparity include the decline of the number of general practitioners,2 physicians choosing to participate in nonrural areas,3 and the low retention of physicians practicing in rural areas.4 The state of Michigan is no exception with its long-standing maldistribution of physicians between metropolitan and nonmetropolitan areas. Currently, all of Michigan's 13 county-wide primary care health professional shortage areas are rural.5 The family medicine traditions of training in ambulatory and hospital care settings, caring for adults of either sex and for children, providing maternity and newborn care, and providing comprehensive continuous care have made family physicians the preferred rural health care professionals. In fact, without family physicians, 58 of Michigan's 83 counties would be considered health professional shortage areas.5
Although it is well recognized that growing up in a rural area is the strongest predictor of eventual practice in a rural community, the second factor is freshman-year plans to specialize in family medicine.6 Thus, to address the shortage of physicians practicing in rural areas, medical schools must make efforts to interest their students in rural practice careers. However, peaking the interest of incoming medical students is a challenge, which is why Whitcomb7 recommended that schools emphasize strategies to support efforts in the recruitment and retention of rural physicians. In response to this call for action, the Wayne State University (WSU) School of Medicine developed an integrated rural core curriculum (RCC) that includes recruiting and matriculating students from rural settings; providing avenues for students to interact with faculty and preceptors with rural backgrounds and/or rural practice experience; offering seminars and group activities to expand awareness of special issues in rural medicine; and promoting rural practice training experiences via externships, clerkships, and elective rotations. This robust 4-year pipeline strategy,8 which emphasizes the advantages of primary care and rural practice and supports student interest, is important to counter the discouragement that students report hearing from specialist faculty attending physicians while they train in required hospital-based, specialty-oriented rotations.
Program Evaluation Challenges
Whereas the long-term goal of the RCC is to increase the number of WSU medical school graduates choosing to practice in small town/rural Michigan communities, its ultimate success cannot yet be determined because the program was only launched in 2007. However, process evaluation is important to determine whether the RCC objectives are being met and to identify programmatic changes needed to maintain fidelity to these objectives. Here we focus on the evaluation strategy used to determine the extent to which students in the new rural medicine interest group who self-identified as selecting a rural externship, clerkship, or elective rotation did indeed secure a clinical training experience in a rural setting.
Two challenges to evaluating this objective are addressed. The first challenge follows from the initial uncertainty of student interest in rural practice at the time of the RCC program launch in 2007. The WSU School of Medicine is oriented as an urban and tertiary/quaternary specialty training program. Students live for 4 or more years in a bustling urban environment rich with diversity and cultural experiences not available in more remote locations. By contrast there are other advantages and opportunities available to rural family physicians that may be unavailable to family physicians in urban/suburban practice communities; however, before the inception of our RCC program, these were not discussed in a systematic way with medical students who may have had an interest in both family medicine and future practice in a rural community. Thus, not knowing if any students at this urban university would engage with the RCC and seek a rural clinical placement, we did not, at the outset, devote resources to systemically evaluate the practice location of currently available preceptors. Rather, we relied on the Michigan Department of Community Health's designation of rural counties to identify rural preceptors.9 Happily, the RCC attracted substantially more students than anticipated and the student demand for rural clinical placements highlighted the need to identify the proportion of students participating in the RCC activities who were successful in securing clinical placements in rural settings.
Even though practice locations were verified as rural using a county level definition, Lin and colleagues10 cautions against using simplistic rural measures. This lead to our second evaluation challenge: to select the most relevant definition of “rural” from among several in use to evaluate our student's clinical placements. Because of the lack of consensus regarding the definition of rural, we choose to use several popular definitions to provide a robust evaluation.
Methods
Our analytical objective was to determine the degree to which the 28 students who participated in the Rural Medicine Interest Group in 2007 and 2008 were placed in rural areas for externships, clerkships, and elective rotations. The methodological challenge was to determine the most appropriate definition of “rural.” We decided to use 3 measures and examine the information provided by each.
Definitions of Rural
The practice location addresses where the students received their externship, clerkship, and elective rotation training were used to define the rural nature of each practice site according to 3 commonly used definitions of “rural.” The strengths and weaknesses of each definition are provided below, along with the components of the evaluation system and the processes used to create it.
Definition #1
The first measure of rural was the percentage of the county population living in rural areas. The advantage of using this definition was that many counties in Michigan contain both rural and urban areas, and this measure provided us with an indicator of this variation within each county. This was important because our evaluation was based on the assumption that counties can be used as proxies for practice catchment areas, ie, the geographic area from which practices draw patients. It is highly probable that practices outsides of heavily urbanized areas (eg, Detroit) draw patients from both urban and rural areas, and thus the county-level percentage of the population that lives in rural areas provides a crude indication of the percentage of the practice population that is considered rural. However, assumptions about practice catchment areas are also the weakness of the measure because there is little to no systematic evidence to support this assumption.
Definition #2
Whereas the rural population of the county provides an interval level continuous measure (see definition #1), we also opted to use an ordinal-level continuous measure: location within a metropolitan area. This provided for an easy county-level dichotomous measure of rural. Although this measure is easy to understand (ie, rural or not rural) and easy to implement, it cannot capture the diversity within and between counties. A rural county in Michigan's upper peninsula is much different from periphery rural counties outside metropolitan areas like Flint or Detroit.
Definition #3
Our final measure of ruralality was dichotomous and based on urban-area boundaries as defined by the US Census Bureau.11 This type of measure provided us with an exact indication of whether a practice was located within an urban or rural area. However, just because a practice is located with an urban area does not mean that it does not also attract a rural population, which is why we chose to use the 2 county-level measures above to roughly capture patient catchment areas and the likelihood that patients come from a rural area. Overall, our measures of rurality are not without controversy (see Rickett et al12), but we think that they provided a valid and reliable way of measuring rurality for program evaluation purposes.
US Census Data
The US Census Bureau's 2000 decennial census data were used to define the first rural measures: percentage of the population that was rural. We used the US Census Bureau's 2000 Summary File 3 county level summary file (SUMLEV = 050) for the State of Michigan, specifically total population (P1) and rural population (P5). In addition, state and county boundaries along with other geographies of interest (ie, urban areas, metropolitan areas, and zip code tabulation areas) were used to define the second and third measures of rural: location within a nonmetropolitan county and location within an urban area. The necessary files were obtained from the US Census Bureau's topologically integrated geographic encoding and referencing system,13 which is free and downloadable through the bureau's website.
Practice address and geographic data were integrated into the Pitney Bowes MapInfo Professional 6.5 (Stamford, CT) geographic information systems (GIS) software system through which we mapped out rural characteristics of Michigan counties through a process called thematic mapping, which shows characteristic variation by color shading geographic features. In addition, reference maps were created to show the boundaries of urban areas and metropolitan/micropolitan areas. Once these processes were completed, we overlaid the practice location on the contextual-level geographic features to produce a visual representation of the relationship between student placement and rural areas throughout the state.
Results
As expected, we found that the outcome of student placement in rural practices depended on the definition of rural. Our first method of analysis using rural definition #1 was to examine the percentage of the population that lives in rural areas (see Figure 1). We found that a slight majority of students (15 of 28) were placed with preceptors that were located in counties where the rural population comprised 40% of the population or greater. In a state where only 25% of the total population lives in rural areas, this seemed an appropriate threshold. On further analysis, only one student was located in a county were the rural population comprised 80% or more of the population, whereas 7 students were placed in counties where less than 20% of the population was rural. This does not mean that students were not located in small towns or rural areas of the county, but the map demonstrates that when the county is used as the unit of analysis very few students were placed in the most rural areas of the state.
Student placement by county-level rural population (rural definition #1).
Using rural definition #2, where the metropolitan, micropolitan, and nonmetropolitan/nonmicropolitan county designations were proxies for nonrural, semirural, and rural areas, respectively, most students (n = 14) were placed in nonrural counties, 8 were placed in semirural counties, and only 6 students were placed in rural counties (ie, not micro- or metropolitan; see Figure 2).
Student placement by metropolitan, micropolitan, and rural counties (rural definition #2).
Evaluating the program based on the student's county of placement suggested that the county unit of analysis might be too large. Many counties in the state contain both rural and urban areas and it was difficult to determine from these maps whether the students were placed in rural/small town areas. Using rural definition #3 (the more strict definition of rural), we overlaid the student placement with urban boundaries (see Figure 3) and found that 21 of 28 students were placed within urban areas. Even students that were considered to be in rural area in previous maps were located in urban areas as defined by the US Census Bureau.11 However, further scrutiny revealed that many of these locations were located in very small urban areas (<5 square miles) or on the periphery of larger urban areas.
Student placement by urban and rural areas (rural definition #3).
Discussion
We concluded that although students were not being placed in the most rural areas of Michigan, they were being placed near rural areas or in semirural areas. Using the metropolitan, micropolitan, rural designation, we found that only a few students were placed in rural counties. Conversely, we found that using a continuous measure of rurality at the county level showed that a larger percentage of students were being placed in what we considered to be rural counties. Finally using a noncounty-based measure, a majority of students were placed in urban (nonrural) areas. However, the placements of students tended to be in very small urban areas (<5 square miles).
Our analysis led to several programmatic developments. First, the results had a direct impact on program management by highlighting gaps in preceptor recruitment. As the second phase of the RCC is being designed and implemented, a greater effort is being made to recruit new physician preceptors from the more rural areas of the state rather than the urban and semirural areas.
Secondly, we are now able to create pools of preceptors based on the range or level of ruralism where they are practicing. This allows us to provide more exact information to students and better place them into settings that support their interests. Students wanting a clinical experience in an extremely rural area will have preceptors to choose from, as will those who seek a more semirural experience.
Third, the results of this process evaluation has also led to developments in other programmatic areas, including the development of a community-based participatory research study addressing the shortage of family physicians in Michigan's rural thumb area.
Finally, our evaluation has led to the development of educational materials for RCC participants. These include the creation of a presentation titled “Rural Michigan by the Numbers,” in which the results of the evaluation and additional data are used in conjunction with GIS to graphically display and describe Michigan's rural areas and the characteristics of the population that reside in these areas. These materials incorporate a variety of the data sets that help us examine and visualize demographic, workforce, and resource characteristics of rural areas in Michigan. These are also used to introduce students to the RCC.
In addition to drawing conclusion related to the RCC program, we also found that GIS was a valuable evaluation tool despite the fact that GIS as an evaluation tool has received little attention in scientific literature.14 We successfully used GIS for process evaluation purposes, but it can also be used for program development through resource allocation, accessibility planning, risk assessments, or intervention targeting.15 Once a program has been designed, GIS also can be used in its implementation and monitoring. However, the usefulness of GIS extends beyond academic medicine into the clinical setting. Its uses can include determining practice catchment areas, resource allocation (ie, opening or closing a practice), producing demographic portraits of communities in which practices operate, and evaluating programs and processes. GIS can be used to create a supportive environment for decision making16 and its products can be used to inform stakeholders and policymakers alike through geovisualization, data reduction, and data integration,17 at both the individual and population levels.18
Notes
This article was externally peer reviewed.
Funding: Blue Cross Blue Shield of Michigan Foundation, grant numbers 1124.RFP.PD and 1125.II.
Conflict of interest: none declared.
- Received for publication July 8, 2009.
- Revision received September 2, 2009.
- Accepted for publication September 8, 2009.
