The American Board of Family Medicine's 8 Years of Experience with Differential Item Functioning

Article Figures & Data

Download figure
Open in new tab
Figure 1.
Percentage of questions reviewed by DIF review process stage.

Table 1.

Family Medicine Certification Examination Core Content Questions Flagged for DIF by Racial Group 2013-2020

Focal Group	Category	2013	2014	2015	2016	2017	2018	2019	2020	Count Mean (SD)	Percentage Mean (SD)
Black	Advantage	11 (2.6%)	10 (2.4%)	10 (2.3%)	11 (2.6%)	11 (2.6%)	10 (2.3%)	10 (2.3%)	18 (3.7%)	11.4 (2.9)	2.6% (0.5)
	Disadvantage	19 (4.4%)	9 (2.1%)	8 (1.9%)	7 (1.6%)	13 (3.0%)	11 (2.6%)	10 (2.3%)	11 (2.3%)	10.7 (3.9)	2.5% (0.9)
	Total	30 (7.0%)	19 (4.5%)	18 (4.2%)	18 (4.2%)	24 (5.6%)	21 (4.9%)	20 (4.6%)	29 (7.6%)	22.1 (5.1)	5.3% (1.3)
	Net advantage	−8 (−1.9%)	1 (0.2%)	2 (0.5%)	4 (0.9%)	−2 (−0.5%)	−1 (−0.2%)	0 (0%)	7 (1.4%)	0.7 (4.7)	0.1% (1.0)
Hispanic	Advantage	6 (3.3%)	5 (3.6%)	6 (5.4%)	3 (4.0%)	7 (5.8%)	2 (4.9%)	5 (3.7%)	7 (1.4%)	4.9 (1.8)	4.0% (1.4)
	Disadvantage	4 (0.9%)	4 (1.0%)	2 (0.5%)	7 (1.6%)	2 (0.5%)	5 (1.2%)	3 (0.7%)	7 (1.4%)	4.6 (1.9)	1.0% (0.4)
	Total	10 (4.2%)	9 (4.6%)	8 (5.9%)	10 (5.6%)	9 (6.3%)	7 (6.1%)	8 (4.4%)	14 (2.8%)	9.4 (2.3)	5.0% (1.2)
	Net advantage	2 (0.5%)	1 (0.2%)	4 (0.9%)	−4 (−0.9%)	5 (1.2%)	−3 (−0.7%)	2 (0.5%)	0 (0%)	0.3 (2.9)	0.2% (0.7)
Asian	Advantage	14 (3.3%)	15 (3.6%)	23 (5.4%)	17 (4.0%)	25 (5.8%)	21 (4.9%)	16 (3.7%)	19 (3.9%)	17.9 (3.3)	4.3% (0.9)
	Disadvantage	19 (4.4%)	10 (2.4%)	14 (3.3%)	7 (1.6%)	10 (2.3%)	10 (2.3%)	8 (1.9%)	8 (1.7%)	10.9 (4.3)	2.5% (0.9)
	Total	33 (7.7%)	25 (6.0%)	37 (8.7%)	24 (5.6%)	35 (8.1%)	31 (7.2%)	24 (4.6%)	27 (5.6%)	28.7 (5.1)	6.7% (1.4)
	Net advantage	−5 (−1.2%)	5 (1.2%)	9 (2.1%)	10 (2.3%)	15 (3.5%)	11 (2.6%)	8 (1.9%)	11 (2.3%)	7.0 (5.7)	1.8% (1.4)
Total number of flags		73	53	63	52	68	59	52	70	61.3 (8.5)
Number of core questions		430	430	426	426	429	430	432	484	435.9 (19.6)

Abbreviation: SD, standard deviation.
Note: The thresholds used to flag questions using the Rasch differential item functioning procedure were an absolute value of the logit contrast ≥ 0.70 and P < .05. The reference group was White. Advantage means that the question was easier for the focal group. Disadvantage means that the question was more difficult for the focal group. Net advantage = Advantage – Disadvantage. Flagging questions was not attempted if there were fewer than 200 responses overall or if the smaller of the 2 groups had fewer than 50 responses. Some questions received multiple flags.