Research ArticleOriginal Research

Community Size and Organization of Practice Predict Family Physician Recertification Success

Bradley M. Schulte, David M. Mannino, Kenneth D. Royal, Sabrina L. Brown, Lars E. Peterson and James C. Puffer
The Journal of the American Board of Family Medicine May 2014, 27 (3) 383-390; DOI: https://doi.org/10.3122/jabfm.2014.03.130016

Abstract

Objective: Health disparities exist between rural and urban areas. Rural physicians may lack sufficient medical knowledge, which may lead to poor quality of care. Therefore, we sought to determine whether medical knowledge differed between family physicians (FPs) practicing in rural areas compared with those practicing in metropolitan areas.

Methods: We studied 8361 FPs who took the American Board of Family Medicine maintenance of certification (MOC) examination in 2009. Data sources were examination results and data from a demographic survey of practice structure and activities, completed as part of the examination application process. FPs' location of practice was categorized as either rural or metropolitan using a moderate and conservative definition based on reported community size. Univariate statistics assessed differences in FP characteristics between rural and metropolitan areas. Logistic regression analyses determined the adjusted relationship between rural status and the odds of passing the MOC examination.

Results: Metropolitan FPs were less likely than their rural counterparts to pass the MOC examination using both the moderate (odds ratio, 0.67; 95% confidence interval, 0.54–0.83) and conservative (odds ratio, 0.56; 95% confidence interval, 0.42–0.74) definitions. Physicians in solo practice were less likely to pass the examination than physicians in group practice.

Conclusion: Rural physicians were more likely to pass the MOC examination, suggesting that rural health disparities do not result from a lack of provider knowledge.

Health disparities in rural populations are substantial when compared with those living in nonrural areas.1,2 Higher rates of diabetes, coronary artery disease, obesity, hypertension, and other chronic diseases are seen in rural populations.3 One explanation for these disparities revolves around access to physicians in rural areas. Access is hindered both by the scarcity of physicians practicing in these areas as well as insufficient health insurance coverage for rural patients.4⇓–6 Another contributor to health disparities may be the quality of health care provided in rural areas. There are multiple efforts to assess and improve the quality of health care in rural areas,7 but if the providers lack the requisite medical knowledge to provide care, quality may suffer.

Assessment of the quality of care delivered by an individual physician is challenging. However, scores attained by physicians on high-stakes standardized exams such as licensure and certification exams have been suggested as reasonable surrogates for quality of care.8⇓–10 Knowledge measured by certification exams has been shown to be the foundation on which all the other skills necessary for quality care delivery are built, and better examination scores have been associated with the delivery of higher quality care.11

Member boards of the American Board of Medical Specialties administer examinations as part of their Maintenance of Certification (MOC) process. MOC was adopted by the American Board of Medical Specialties in 2000 with the goal of improving the quality of care delivered by board-certified physicians. As one component of MOC, the examination serves as an objective measure of cognitive expertise.12,13

We previously demonstrated that family physicians who are international medical graduates (IMGs), are in solo practice, or who practice in poor areas of the country are less likely to meet their MOC requirements, which includes passing the MOC examination.14 Family physicians practice in rural areas in a higher proportion than any other specialty,3 and a large number of rural physicians are IMGs or are in solo practice15; thus we sought to determine whether rural family physicians were more or less likely to pass the American Board of Family Medicine (ABFM) MOC examination.

Our specific objective was to determine whether the likelihood of passing the ABFM MOC examination was predicted by the size of the community in which the physician practiced or the organizational structure of the practice. We hypothesized that the organization of the practice and the size of the community in which the physician practices would, when controlling for other demographic variables, have a limited effect on the scores attained on the ABFM MOC examination.

Methods

Study Sample

Our sample included family physicians seeking to maintain certification by the ABFM in July 2009. Only MOC candidates were included because these physicians completed the demographic survey described below as part of the examination registration process.

MOC Examination

The MOC examination is taken on a computer and has a core set of 260 questions administered to all examinees and 2 modules with 45 questions each; the modules are selected by each examinee from a group of 8 choices. The ABFM MOC exams are psychometrically sound with high reliability.12 A national panel of family physicians determines the pass/fail threshold for this examination every 3 years through participation in a study in a standard setting.

Demographic Survey

As part of the examination registration process, applicants must complete a demographic survey that includes questions on practice characteristics. Data collected include information regarding the size of the physician's community of practice, the organization of their practice, possession of hospital admitting privileges, and number of years in practice.

For our analyses we coded the variables to reflect known associations with MOC participation or examination performance. We coded practice organization as solo, group, or other to capture important associations between solo practice and MOC participation. Categories collapsed into the “other” category included partnership, government, health maintenance organization, industrial, administration, full-time educator in family medicine, educator not in family medicine, locum tenens, and other. Responses regarding the “size of community in which you practice” were <5,000, 5,000–9,999, 10,000–24,999, 25,000–49,999, 50,000–99,999, 100,000–249,999, 250,000–499,99, 500,000–999,999, and ≤1,000,000. We collapsed these categories for analysis as described in the “Rural and Metropolitan Definitions” section. We determined medical school faculty status from a question asking whether respondents were full-time faculty, part-time faculty, or not faculty. Respondents were asked whether they were certified by another medical specialty board, and we created a variable indicating so if they were.

Other demographic data were obtained from ABFM administrative data. These data included sex, age, medical training location, and type of medical degree held. Location of medical training was coded as United States/Canada or international. Because of the small number of Canadian-trained physicians in the sample, we combined physicians trained in the United States and Canadian because they perform similarly on the MOC examination.16

Rural and Metropolitan Definitions

Two definitions of rural were used for this study: a moderate definition and a conservative definition. The moderate definition characterizes populations having ≤25,000 residents as rural and populations having ≥500,000 residents as metropolitan. The literature shows that the majority of researchers use a population threshold of 50,000 to differentiate between metropolitan and nonmetropolitan areas.17 This is supported by the widely used definitions of metropolitan statistical areas established by the Office of Management and Budget, which define rural counties by exclusion.18 A metropolitan statistical area is defined as an area “…with cities or urbanized areas with population exceeding 50,000.”18 The Census Bureau's definition of an urbanized area reads, “An area consisting of a central place(s) and adjacent territory with a general population density of at least 1000 people per square mile of land area that together have a minimum residential population of at least 50000 people.”19 By using a definition with more stringent standards (metropolitan area, >500,000), we were more likely to properly classify physicians by their location of practice.

The conservative definition of rural identifies populations of ≤10,000 residents as rural and any population ≥1000,000 residents as metropolitan. The moderate and conservative definitions were chosen because of the variability in the definitions of metropolitan used by United States government organizations. These 2 definitions allowed us to ensure that our results more accurately encompassed the rural and metropolitan characteristics present.

Statistical Analysis

We used descriptive statistics to characterize the physician sample and student t tests and χ2 tests to determine significant differences by rural/metropolitan status. Multivariate logistic regression analyses determined the associations between community size and organization of practice with likelihood of passing the MOC examination. We ran 2 identical regression models, each using one of the rural/metropolitan definitions. Examination result, recorded as pass or fail, was the dependent variable. We controlled for practice organization, age, sex, location of medical training (United States/Canada or international), degree held (MD or DO), academic appointment, and presence of other certification. Group practice, male sex, training in the United States, an MD degree, no academic appointment, and no other certifications served as the referent groups for the regression analyses. We used receiver operating characteristic plots and assessed area under curve (AUC) functions for each regression to ensure the stability of our models. Given the literature suggesting that rural physicians are more likely to be in solo practice,15 we included an interaction between organization of practice, age, and rural status. All analyses were performed using SPSS software version 18 (SPSS, Inc., Chicago, IL).

Human Subjects

The University of Kentucky Institutional Review Board certified this study as exempt.

Results

Our sample consisted of 8361 family physicians with a mean age of 52.2 years (range, 35–90 years); 65.4% were male, 92.1% held an MD degree, and 85.7% trained in the United States or Canada (Table 1). Of the physicians in the sample, 44.2% practiced in a group setting, whereas 18.7% were solo practitioners. While the majority of physicians reported no academic appointment (72.8%), 409 (4.9%) were full time educators and 1864 (22.3%) were part time educators. A passing score on the MOC examination was achieved by 85.5%, with a mean score of 512 (standard deviation, 113). The passing score was 390.

View this table:
Table 1. Demographics of Family Physicians taking the American Board of Family Medicine Recertification Examination in July 2009 (n = 8361)

Comparing physician characteristics by our rural/metropolitan definitions, rural physicians were slightly older than their metropolitan counterparts (P < .05) (Table 2). Rural physicians also were more likely to be in solo practice (P < .05). Examination pass rates across the conservative and moderate definitions of rural were similar to the overall pass rates (Figure 1). However, an increase in the percentage passing using the conservative definition was evident, whereas a decrease in the pass rates of metropolitan physicians became apparent when the definition was more restrictive.

View this table:
Table 2. Demographics of Family Physicians taking the American Board of Family Medicine Recertification Examination in July 2009, Stratified by Rural Definition
Figure 1.
Figure 1.

Examination results of family physicians taking the American Board of Family Medicine recertification examination in July 2009 stratified by rural definition. *Moderate rural definition: rural ≤ 25,000 in community, metropolitan ≥ 500,000 in community. †Conservative rural definition: rural ≤ 10,000 in community, metropolitan ≥ 1,000,000 in community.

The regression models for both the moderate and conservative definitions of rural showed acceptable model fit based on the results of the Hosmer-Lemeshow goodness of fit test, with P values of 0.316 and 0.938, respectively. The receiver operating characteristic plots indicated a statistically significant ability to predict examination performance, with the AUC increasing from the moderate to the conservative rural definition (AUC, 0.740 and 0.748, respectively). Tests for interactions between practice organization, age, and rurality were not significant and were not included in the final model.

Results for the moderate rural definition regression model are seen in Table 3 and Figure 2. Using the moderate definition, metropolitan physicians had a 33.1% lower odds of passing the MOC examination (odds ratio [OR], 0.67; 95% confidence interval [CI], 0.54–0.83). Physicians in solo practice (OR, 0.45; 95% CI, 0.35–0.58) also were less likely to pass the examination compared with physicians in group practice. Using the moderate definition, age was a predictor of failing the examination (OR, 0.95; 95% CI, 0.93–0.96). IMGs had a 78.7% decrease in the odds of passing the examination compared with US- and Canadian-trained physicians (OR, 0.21; 95% CI, 0.17–0.27). Osteopathic physicians exhibited decreased odds of passing the examination compared with allopathic physicians (OR, 0.28; 95% CI, 0.20–0.39). Neither academic appointment nor non-ABFM certification was associated with passing the examination.

View this table:
Table 3. Adjusted Associations between Physician Characteristics and Odds of Passing the July 2009 American Board of Family Medicine Recertification Examination by Two Rural Definitions
Figure 2.
Figure 2.

Plot of the odds ratios for regression analyses for prediction of likelihood of passing American Board of Family Medicine Maintenance of Certification for Family Physicians examination using moderate rural definition (rural ≤ 25,000 in community, metropolitan ≥ 500,000 in community).

Using the conservative definition of rural and metropolitan reduced the sample size to 2108 physicians (Table 3 and Figure 3). Adjusted associations between physician characteristics and odds of passing the MOC examination were largely in the same direction and magnitude using the conservative definition as when using the moderate definition.

Figure 3.
Figure 3.

Plot of the odds ratios for regression analyses for prediction of likelihood of passing American Board of Family Medicine Maintenance of Certification for Family Physicians examination using conservative rural definition (rural ≤ 10,000 in community, metropolitan ≥ 1,000,000 in community).

Discussion

Using a large cohort of family physicians seeking to maintain certification and successively restrictive definitions of rurality, we found that rural family physicians have a higher pass rate than their metropolitan counterparts on the ABFM MOC examination. Our results suggest that board-certified family physicians who practice rurally and choose to maintain their certification exhibit a similar, if not greater, fund of medical knowledge than their metropolitan counterparts. Given this information, we cautiously infer that these physicians may be delivering at least similar or higher-quality care in those domains that are directly affected by physician knowledge.11

Certain physician characteristics were found to have important associations with odds of passing the recertification examination. Family physicians in solo practice had reduced odds of passing the examination compared with physicians in group practice. This decrease is of particular concern because many rural physicians are in solo practices.15 As solo practitioners, these physicians lack infrastructure supporting quality improvement and education that physicians in other settings may enjoy. These solo physicians also may be too busy attending to the business of running their practice and seeing patients than keeping up with advances in medical knowledge. Particular concern that this may be in greater danger of happening among rural solo family physicians might seem justified, but we found no statistical interaction between rurality and solo practice in our analyses. Confirming prior research,20 we found that osteopathic family physicians exhibited significant decreases in the likelihood of passing the MOC examination in every regression model. However, caution must be taken when interpreting these results; osteopathic physicians may certify with the American Osteopathic Board of Family Medicine, and results from the ABFM examination should not be used as a blanket statement that all osteopathic family physicians have a lower fund of medical knowledge. Similarly, we found that IMGs had lower odds of passing the examination, but it is known that differences in passing rates exist among selected groups of IMGs. In particular, previous research has shown that US-born IMGs tend to perform less well on certification examinations than foreign born IMGs.21 Unfortunately, we were unable to control for this in our study.

Limitations

Our study has several limitations. First, the sample of family physicians surveyed was not randomly selected; rather, it represented a census of those family physicians who opted to maintain their certification by applying for the MOC examination. ABFM diplomate participation in MOC is >85%,14 but this estimate includes only physicians who are currently certified. Family physicians may opt to not certify with the ABFM or might fail to attain certification yet can still practice medicine. The knowledge of these physicians was not tested in our study because we had no way of including them in our sample. We also were unable to find an estimate in the literature on the percentage of rural family physicians who were board certified. Second, the demographic variables from the application survey are self-reported and thus may not be entirely accurate. However, other work compared reported rates of use of electronic health records between this data set and those from a nationally representative database and found comparable responses, suggesting the data are valid.22 Third, as mentioned previously, osteopathic-trained physicians who seek certification elsewhere and allopathic physicians not seeking certification are not included in this dataset, and thus our results do not apply to them. Fourth, our definitions of rurality and metropolitan were based on the size of the practice community as reported by the physician. This definition is an imperfect measure of rurality because some physicians in small suburbs of large areas may identify themselves as practicing in a community of 10,000 to 24,999, which is very different from a physician practicing in a remote town of 10,000 to 24,999. Using 2 progressively more restrictive definitions of rural, our results remained stable, which argues for the validity of this approach. Future work could use county or practice ZIP code for a more accurate classification.

Conclusions

ABFM-certified rural family physicians attempting to maintain their board certification exhibited increased medical knowledge, as measured by the MOC examination, when compared with their metropolitan counterparts. Our findings therefore suggest that rural family physicians certified by the ABFM have the foundational medical knowledge necessary to provide high-quality health care. Many other variables may play a role in determining the quality of care delivered by physicians. A rural physician's ability to deliver quality of care comparable to that provided by their urban counterparts may be influenced by the population characteristics or attributable to a lack of adequate health care resources. Future research must explore the interplay between physician participation in MOC, the availability of resources within rural and metropolitan areas, and the ability to deliver high-quality health care. This research is essential to ensure that rural populations have access to high-quality, equitable care.

Notes

  • This article was externally peer reviewed.

  • Funding: none.

  • Conflict of interest: none declared.

  • Received for publication January 10, 2013.
  • Revision received October 9, 2013.
  • Accepted for publication October 10, 2013.

References

  1. 1.↵
    1. Shi L,
    2. Macinko J,
    3. Starfield B,
    4. Politzer R,
    5. Wulu J,
    6. Xu J
    . Primary care, social inequalities and all-cause, heart disease and cancer mortality in US counties: a comparison between urban and non-urban areas. Public Health 2005;119:699–710.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Bennett KJ,
    2. Olatosi B,
    3. Probst JC
    . Health disparities: a rural–urban chartbook. Columbia: South Carolina Rural Health Research Center; 2008. Available from: http://rhr.sph.sc.edu/report/(7–3)%20Health%20Disparities%20A%20Rural%20Urban%20Chartbook%20-%20Distribution%20Copy.pdf. Accessed October 6, 2013.
  3. 3.↵
    1. Simmons LA,
    2. Huddleston-Casas CA,
    3. Morgan KA,
    4. Feldman D
    . Mixed methods study of management of health conditions in rural low-income families: implications for health care policy in the USA. Rural Remote Health 2012;12:1879.
    OpenUrlPubMed
  4. 4.↵
    1. Brooks RG,
    2. Walsh M,
    3. Mardon RE,
    4. Lewis M,
    5. Clawson A
    . The roles of nature and nurture in the recruitment and retention of primary care physicians in rural areas: a review of the literature. Acad Med 2002;77:790–8.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Reschovsky JD,
    2. Staiti AB
    . Access and quality: does rural America lag behind? Health Aff (Millwood) 2005;24:1128–39.
    OpenUrlAbstract/FREE Full Text
  6. 6.↵
    1. Ricketts TC
    . The changing nature of rural health care. Annu Rev Public Health 2000;21:639–57.
    OpenUrlCrossRefPubMed
  7. 7.↵
    Institute of Medicine. Quality through collaboration: the future of rural health care. Washington, DC: National Academies Press; 2004.
  8. 8.↵
    1. Holmboe ES,
    2. Lipner R,
    3. Greiner A
    . Assessing quality of care: knowledge matters. JAMA 2008;299:338–40.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Sharp LK,
    2. Bashook PG,
    3. Lipsky MS,
    4. Horowitz SD,
    5. Miller SH
    . Specialty board certification and clinical outcomes: the missing link. Acad Med 2002;77:534–42.
    OpenUrlPubMed
  10. 10.↵
    1. Puffer J
    . The American Board of Family Medicine certification examination: a proxy for quality. Fam Med 2011;43:433–4.
    OpenUrlPubMed
  11. 11.↵
    1. Hess BJ,
    2. Weng W,
    3. Holmboe ES,
    4. Lipner RS
    . The association between physicians' cognitive skills and quality of diabetes care. Acad Med 2012;87:157–63.
    OpenUrlPubMed
  12. 12.↵
    1. Royal KD,
    2. Puffer JC
    . The reliability of ABFM examinations: implications for test-takers. Ann Fam Med 2011;9:463–4.
    OpenUrlFREE Full Text
  13. 13.↵
    1. Hess BJ,
    2. Weng W,
    3. Lynn LA,
    4. Holmboe ES,
    5. Lipner RS
    . Setting a fair performance standard for physicians' quality of patient care. J Gen Intern Med 2011;26:467–73.
    OpenUrlPubMed
  14. 14.↵
    1. Xierali IM,
    2. Rinaldo JC,
    3. Green LA,
    4. et al
    . Family physician participation in maintenance of certification. Ann Fam Med 2011;9:203–10.
    OpenUrlAbstract/FREE Full Text
  15. 15.↵
    1. Baldwin LM,
    2. Hart LG,
    3. West PA,
    4. Norris TE,
    5. Gore E,
    6. Schneeweiss R
    . Two decades of experience in the University of Washington Family Medicine Residency Network: practice differences between graduates in rural and urban locations. J Rural Health 1995;11:60–72.
    OpenUrlPubMed
  16. 16.↵
    American Board of Family Medicine. Examination pass rates. Available from: https://www.theabfm.org/moc/passrate.aspx. Accessed October 6, 2013.
  17. 17.↵
    1. Cromartie J,
    2. Bucholtz S
    . Defining the “rural” in rural America. Amber Waves 2008;6:28–34.
    OpenUrl
  18. 18.↵
    National Rural Health Resource Center. An alternative approach to defining rural for the purpose of providing emergency medical services (EMS). Available from: http://www.ruralcenter.org/tasc/resources/alternative-approach-defining-rural-purpose-providing-emergency-medical-services-ems. Accessed October 7, 2013.
  19. 19.↵
    United States Census Bureau. Census 2000 urban and rural classification. Available from: https://www.census.gov/geo/reference/ua/urban-rural-2000.html. Accessed March 12, 2014.
  20. 20.↵
    1. O'Neill TR,
    2. Royal KD,
    3. Schulte B,
    4. Leigh TM
    . Comparing the performance of allopathically and osteopathically trained physicians on the American Board of Family Medicine's Certification examination. 2009. Available from: http://www.eric.ed.gov/ERICWebPortal/search/simpleSearch.jsp?newSearch=true&eric_sortField=&searchtype=basic&pageSize=10&ERICExtSearch_SearchValue_0=ED506669&eric_displayStartCount=1&_pageLabel=ERICSearchResult&ERICExtSearch_SearchType_0=no. Accessed October 6, 2013.
  21. 21.↵
    1. Boulet JR,
    2. Swanson DB,
    3. Cooper RA,
    4. Norcini JJ,
    5. McKinley DW
    . A comparison of the characteristics and examination performances of U.S. and non-U.S. citizen international medical graduates who sought Educational Commission for Foreign Medical Graduates certification: 1995–2004. Acad Med 2006;81:S116–9.
    OpenUrlCrossRefPubMed
  22. 22.↵
    1. Xierali IM,
    2. Hsiao CJ,
    3. Puffer JC,
    4. et al
    . The rise of electronic health record adoption among family physicians. Ann Fam Med 2013;11:14–9.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools

Jump to section

Keywords