Reliabilities of Abstract Reviews by Family Medicine Meeting, Year, and Submission Category
| Submission by Year, Meeting, Category | N, Submissions | Number of Reviews, Mean | Intracluster Correlation Coefficients (95% CI) | Reliability of Within-Abstract Mean Score (95% CI) | |
|---|---|---|---|---|---|
| Abstract-Level | Reviewer-Level | ||||
| North American Primary Care Research Group | |||||
| 2016 oral presentation on completed research | 392 | 2.0 | 0.18 (0.09-0.26) | 0.22 (0.15-0.30) | 0.30 (0.17-0.41) |
| 2017 oral presentation on completed research | 400 | 2.0 | 0.15 (0.08-0.24) | 0.25 (0.17-0.33) | 0.26 (0.15-0.39) |
| 2018 oral presentation on completed research | 388 | 2.0 | 0.17 (0.08-0.26) | 0.15 (0.08-0.26) | 0.29 (0.15-0.41) |
| 2019 oral presentation on completed research | 374 | 3.0 | 0.10 (0.03-0.16) | —* | 0.24 (0.08-0.36) |
| 2019 poster on completed research | 139 | 2.0 | 0.03 (0.00-0.21) | —* | 0.07 (0.00-0.34) |
| 2019 poster on research in progress | 238 | 2.0 | 0.11 (0.00-0.24) | —* | 0.18 (0.00-0.38) |
| 2019 workshop | 41 | 2.9 | 0.00 (0.00-0.18) | —* | 0.00 (0.00-0.39) |
| Society of Teachers of Family Medicine, 2018 Spring Meeting | |||||
| Completed research poster | 30 | 3.0 | 0.33 (0.11-0.55) | 0.11 (0.00-0.27) | 0.60 (0.27-0.79) |
| Completed research project | 75 | 3.0 | 0.50 (0.37-0.63) | 0.08 (0.00-0.16) | 0.75 (0.64-0.84) |
| Completed scholarly project | 37 | 2.9 | 0.21 (0.02-0.41) | 0.34 (0.00-0.72) | 0.44 (0.06-0.67) |
| Completed scholarly project poster | 49 | 2.5 | 0.18 (0.00-0.40) | 0.15 (0.00-0.35) | 0.35 (0.00-0.63) |
| Developing scholarly project poster | 112 | 2.3 | 0.13 (0.02-0.24) | 0.37 (0.16-0.58) | 0.25 (0.04-0.42) |
| Trainee work-in-progress poster | 244 | 2.0 | 0.27 (0.16-0.38) | 0.22 (0.04-0.40) | 0.42 (0.28-0.55) |
| Lecture discussion | 238 | 2.9 | 0.17 (0.10-0.25) | 0.22 (0.06-0.38) | 0.37 (0.24-0.49) |
| Panel discussion | 20 | 3.0 | 0.20 (0.00-0.44) | 0.19 (0.00-0.48) | 0.42 (0.00-0.70) |
| Pre-conference workshop | 14 | 7.0 | 0.34 (0.12-0.56) | 0.16 (0.00-0.35) | 0.78 (0.49-0.90) |
| Scholarly topic roundtable discussion | 89 | 2.9 | 0.12 (0.01-0.24) | 0.16 (0.01-0.31) | 0.28 (0.03-0.48) |
| Seminar | 180 | 3.0 | 0.16 (0.07-0.24) | 0.15 (0.00-0.29) | 0.36 (0.18-0.49) |
| Workshop | 57 | 3.0 | 0.33 (0.17-0.49) | 0.10 (0.00-0.26) | 0.60 (0.38-0.74) |
↵* Reviewer identifiers were not available for the 2019 North American Primary Care Research Group meeting, preventing estimation of reviewer-level ICCs.
CI, confidence interval; ICCs, intracluster correlation coefficients.