Abstract
Background: Graduates of US allopathic and international medical schools comprise the majority of physicians who began family medicine residency in July 2011. Different procedural skills may be taught in allopathic and international medical schools leading to variation in the procedures that graduates can perform independently at the beginning of residency training. A mismatch between assigned resident tasks and procedural skills mastered during medical school may jeopardize patient safety.
Methods: A survey was distributed nationwide to 3287 family medicine residents in July 2011 to determine the proportion of graduates of allopathic and international medical schools who self-reported the ability to perform each of 41 procedures independently.
Results: Surveys were completed by 681 residents (response rate = 21%). The proportion of allopathic and international graduates self-reporting the ability to perform 7 ambulatory, 4 inpatient and 4 maternity care procedures was statistically significantly different.
Conclusions: All graduates self-reported the ability to perform few procedural skills independently upon entry to residency. More allopathic graduates self-reported the ability to perform ambulatory procedures, whereas more international graduates self-reported the ability to perform inpatient and maternity care procedures. Evaluation of individual resident competencies is key to tailor patient care responsibilities and supervision appropriately to resident abilities.
Resident physicians provide much of the direct medical care to patients within teaching hospitals in the United States.1 Autonomy to complete tasks independent of oversight by an attending physician is earned as a resident physician progresses through training. Yet what skills, if any, should physicians be expected to be competent to perform at the start of residency training?
Resident physicians may be on call overnight, assuming responsibility for admitting and caring for patients in the hospital, from the first day of residency. A recent survey of 3003 residents from residency programs representing 10 specialties found that new residents often performed procedures without direct faculty involvement during the first few months of residency training.2 Although oversight by a senior resident would be expected, having new residents perform tasks at which they are not yet proficient is at least part of the basis of the so-called July phenomenon. The July phenomenon refers to a decrease in the quality of patient care that is attributable to the inexperience of new trainees who assume responsibility for caring for patients in July of their first postgraduate year. A 2011 systematic review of all studies published from 1989 through 2010 found that mortality increases and efficiency decreases in hospitals when new residents begin clinical service.3 However, the evidence is less clear regarding which tasks are most problematic for new residents, the impact of specific models of training on mitigating risk, or the degree of risk that an individual patient might face if hospitalized in July compared with another time of year.3
Prior studies suggest that a mismatch exists between competencies attained in medical school and competencies residency programs expect new residents to perform independently.4⇓⇓–7 With respect to procedural skills, the disconnect between competencies attained in medical school and those expected by residency programs may be in part due to variation among medical school experiences and competency assessment required for graduation.8⇓⇓–11
Such differences in experience and expectation may be apparent when comparing graduates of US allopathic medical schools with graduates of international medical schools because the training models, medical culture, and competencies expected may differ greatly. In family medicine residency programs, detecting differences in procedural competency may be important because both allopathic and international medical graduates comprise a significant proportion of resident trainees. In July 2011, 3443 medical school graduates started residency in family medicine residency programs accredited by the Accreditation Council of Graduate Medical Education (ACGME).12 Of these 3443 new residents, 1582 (46%) were graduates of allopathic medical schools, whereas 1228 (36%) were graduates of international medical schools.12
Limited studies have compared the procedural skills acquired by graduates of allopathic and international medical schools. A study of allopathic and international graduates found that surgery program directors found no difference in clinical skills between the 2 groups.13 However, an analysis of performance on the internal medicine in-training examination from 1988 to 2000 revealed that international medical graduates outperformed US medical graduates from 1995 through 2000.14 In addition, a 2006 analysis of 6 internal medicine residency programs in Maryland revealed that international graduates in those programs were more likely to have prior experience practicing medicine, less fatigue, and higher self-esteem than their allopathic counterparts.15 Despite these studies, a knowledge gap remains. Do allopathic and international medical school graduates entering ACGME-accredited family medicine residency programs differ in their self-reported ability to perform common procedures independently at the beginning of residency training?
A new resident may be assigned tasks and responsibilities to perform independently because assumptions are made about the competence they attained during medical school. If a mismatch exists between assigned tasks and skills mastered in medical school, patient safety may be jeopardized. Allopathic and international medical schools may offer different experiences or expect different procedural competencies for graduation. As a result, graduates of medical school may begin residency with different skill sets and may require different degrees of instruction and supervision to perform tasks expected of resident physicians. Ensuring that adequate instruction and supervision is provided to new residents is critical to ensure patient safety. This study seeks to define the proportion of allopathic and international medical school graduates who self-report the ability to perform common procedures independently and to determine whether differences exist between allopathic and international medical school graduates.
Methods
This study was completed July 1 through September 30, 2011, and was approved by the University of Kansas School of Medicine–Wichita Institutional Review Board. This study is a secondary data analysis and part of a larger study.
In July 2011, each ACGME-accredited family medicine residency program listed on either the American Academy of Family Physicians website or the American Medical Association's Fellowship and Residency Electronic Interactive Database Access System with positions for postgraduate year (PGY) 1 residents was sent a packet of surveys via US mail. Each packet contained a survey for the residency program director. Data related to the program directors' responses are not included in this analysis. However, each packet also included sufficient surveys for the estimated number of PGY 1 residents (class of 2014) in the program.
Each resident survey was accompanied by a self-addressed, stamped envelope for confidential return of the survey. Returning a completed survey implied consent to participate in the study. A single mailing of 3287 resident surveys was performed.
The survey instrument asked respondents whether they were able to perform each of 41 procedures independently at the start of the internship. The survey instrument defined “perform independently” as the ability to recognize the need for a procedure; provide informed consent to the patient through a discussion of the risks, benefits, and alternatives; and perform the procedure without prompting or coaching from a faculty member or other supervisor. When responding, the residents were asked to ignore billing, liability, and institutional policies that determined who must be present when a procedure was performed. Common ambulatory, inpatient, and maternity care procedures were included after review of the requirements of the ACGME Review Committee for Family Medicine Programs and the Liaison Committee for Medical Education for medical schools.16⇓–18 Demographic characteristics about the respondents were collected, including sex, age, and name of medical school from which they graduated.
The principal outcome of interest for this study was the proportion of graduates of allopathic and international medical schools who self-reported the ability to perform each of the procedures independently. A χ2 analysis was performed to measure the difference between the proportions of graduates of each type of medical school who self-reported ability to perform each procedure independently (N = 41). Given the large number of comparisons included in our analysis, a Bonferroni correction was performed. Comparisons were considered statistically significantly different if the P value was ≤.01.
Results
Of the 3287 family medicine PGY 1 residents, 681 (21%) returned completed surveys. Of the 681 surveys returned, 336 (52%) were from graduates of US allopathic medical schools and 192 (30%) were from graduates of international medical schools. The remaining 113 respondents (18%) were graduates of US osteopathic medical schools.
Table 1 summarizes the proportion of allopathic and international medical school graduates who self-reported the ability to perform each of 41 procedures independently at the time of entry into residency. Of the 41 procedures, a statistically significant difference exists among the proportion of allopathic and international graduates self-reporting the ability to perform 15 procedures. Seven ambulatory, 4 inpatient, and 4 maternity care procedures were noted to have statistically significant differences between the proportion of allopathic and international medical school graduates who self-reported the ability to perform the procedure independently. Table 2 provides a summary of the procedures that each type of medical graduate was statistically significantly more likely to self-report the ability to perform.
Proportion of Allopathic and International Medical School Graduates Who Self-Report Ability to Perform Procedures
Comparison of Allopathic and International Medical School Graduate Self-Reported Competence
Discussion
Several interesting trends emerged from comparison of the self-reported ability of allopathic and international medical school graduates to perform specific procedures. Fewer than 80% of allopathic medical school graduates reported the ability to perform 34 of the 41 procedures independently. Similarly, less than 80% of international medical school graduates reported the ability to perform 36 of the 41 procedures independently. Procedures included in the survey included both those that are fairly basic to perform, such as a throat culture, as well as those that are more advanced, such as a caesarean section or colonoscopy. Our findings suggest that medical school graduates enter residency with few procedural skills they feel able to perform independently. Of particular concern may be that less than 80% of graduates self-reported that they were able to read an echocardiograph, interpret electronic fetal monitoring, or perform endotracheal intubation independently. A resident on call may be expected to perform these tasks or may be the only physician available to do so. Our results suggest that few residents self-report the ability to perform the procedures independently at the outset of residency, creating a potential challenge to patient safety.
A closer examination of the 15 procedures that demonstrated a statistically significant difference between the proportions of allopathic and international medical graduates self-reporting competency reveals interesting trends. A statistically significantly higher proportion of allopathic graduates self-reported the ability to perform 6 common ambulatory procedures, including laceration repair, Papanicolaou smear, wet mount/potassium hydroxide studies, throat culture, fluorescein eye examination, and excisional skin biopsy. A statistically significantly higher proportion of international graduates self-reported the ability to perform 4 inpatient procedures (place a nasogastric tube, place a central line, perform an arterial blood gas measurement, and place anterior packing for epistaxis) and 4 maternity care procedures (repair a second-degree perineal laceration, assess fetal station during a cervical examination, determine fetal position using a Leopold maneuver, and assess cervical effacement). The influence of hospitalists and laborists, differences in emphasis in curriculum, or differences in the amounts of time students spend in ambulatory or inpatient environments may explain these differences. In addition, with work hour restrictions, allopathic medical students may be less likely to be actively involved in procedures because residents may have fewer experiences and be less willing to teach rather than actively perform a task. Furthermore, differences in prior practice experience may also account for the differences in self-reported ability to perform procedures independently. Future studies are needed to clarify the differences in educational experiences that result in these trends.
It is important to note several limitations to our study. Results reflect aggregate, self-reported data, which may be significantly different from any individual experience from any specific school. We considered international medical school graduates as a single group and did not stratify by location of school or whether a graduate was US or foreign born. The lack of stratification may be problematic because some international medical graduates undergo clinical training in the United States in settings similar to those in which US allopathic medical graduates train. The self-report of residents may over- or underestimate true competence to perform any given procedure. Our findings were not confirmed by an analysis of objective measures of competence or a report of the specific number of procedures performed by each individual. Our sample included all individuals who started residency in family medicine in July 2011, including those who may have had previous training in another specialty or prior practice experience. Likewise, we did not survey residents in any other specialty who may, as medical students, have elected to take difference courses or had difference experiences that led to different self-reported abilities at the outset of residency training.
In addition, the initial mailing underestimated the number of PGY-1 residents starting family medicine residency by 171 when compared with the final analysis of the 2011 National Resident Matching Program.12 Residents who were excluded unintentionally may have opinions that differ from those reflected in our results. Our response rate was 21%. The residents who chose not to respond to the initial and only request to participate may differ in their self-reported abilities when compared with those who did respond.
Our results suggest that there are many common procedures that medical school graduates do not report that they are able to perform independently when beginning residency, such as performing an assessment of cervical dilation, effacement, or fetal station on a pregnant patient or urine microscopy. In addition, the types of procedures that allopathic and international medical school graduates are likely to become competent to perform during medical school may differ: allopathic students perform more ambulatory procedures, whereas international students perform more inpatient and maternity care procedures. These findings highlight the need for assessment and educational interventions at both the medical school and residency levels. Medical schools should specify the procedural skills for which students are expected to be competent to perform at the time of graduation. Furthermore, residency program directors should individually assess each resident's competence at the outset and during residency before allowing independent performance of procedures. Such assessments might use high and low fidelity simulators, standardized patients, video observation, or other formal standardized testing procedures to determine competence.
Moreover, such assessments ought to be used to tailor education, supervision, and responsibilities of residents to ensure that patient safety is protected. A lack of familiarity with possible risks as well as the pressure to impress colleagues and faculty, especially early in residency, may lead new residents to be reluctant to ask for help when performing a procedure that they are expected to already be able to complete independently. Without direct supervision by a physician who has been documented to be competent to perform the procedure, patient safety may be at risk.
Notes
-
This article was externally peer reviewed.
-
Funding: This study was supported by a research grant from the US Department of Health and Human Services, Health Resources and Services Administration (D58HP20802, (RDK-PI), Residency Training in Primary Care).
-
Conflict of interest: none declared.
- Received for publication May 15, 2012.
- Revision received October 12, 2012.
- Accepted for publication October 18, 2012.
