Abstract
Background: Comprehensiveness in primary care is defined as the breadth of services provided by a health care clinician team and is an important metric related to patient outcomes and care delivery. We describe a novel measure of comprehensiveness based on ICD-10 codes.
Methods: We compare the distribution of ICD-10 codes from the care of a large population at a regional academic health system to the distribution of codes from the National Ambulatory Medical Care Survey (NAMCS) using linear regression and the mathematical inner product.
Results: The linear regression between the pattern of ICD-10 codes for the selected population and the NAMCS has a slope 1.00, 95% CI 0.57:1.43, P = .0002, R2 0.62. When considering specific specialty areas of practice, primary care is distinct from specialty care based on the inner product between the distribution of care for a given specialty independent of whether a regional or national reference population is used.
Conclusion: The distribution of care based on ICD-10 codes provides a stable and possibly generalizable reference for comprehensive care. The inner product of an ICD-10 care distribution and a reference provides a quantitative estimate of comprehensiveness that distinguishes primary care from specialty care.
- Administration
- Benchmarking
- Comprehensive Health Care
- Family Medicine
- Health Care Surveys
- Linear Regression
- National Ambulatory Medical Care Survey
- Primary Health Care
Introduction
Comprehensiveness in primary care is defined as the breadth of services provided by a health care clinician team. The literature suggests that a higher level of comprehensiveness is associated with improved health outcomes and more cost-effective patient care.1–3 Empirical evidence supports the implementation of comprehensiveness as a critical metric within primary care settings. Moreover, there is potential for developing targeted measures of comprehensiveness that are not only efficient to calculate but also practically valuable in enhancing health care delivery.3
In this work, we describe a novel measure of comprehensiveness. We propose a measure based on compiling ICD-10 codes documented for ambulatory encounters and comparing the distribution to a reference data set that represents all the care for a large population by all types of clinicians. Since medical diagnosis codes such as ICD-10 are a standardized system used worldwide, they can be leveraged for data analysis within and between health systems.4 Compiling the entirety of diagnosis codes across all fields of care in a large population at the level of ICD-10 chapter serves as a benchmark or “reference pattern” for defining comprehensive care. Physicians who address a broader range of conditions, as indicated by ICD-10 codes from multiple chapters, are presumed to provide more comprehensive care.
The National Ambulatory Medical Care Survey (NAMCS) provides nationwide statistics on visits to ambulatory care physicians. Because the NAMCS randomly samples data across clinicians nationwide,5 rather than limiting by patient age, demographic, or diagnosis, it provides a proxy for simulating a true reference pattern for comprehensive care at a national level.
We describe an approach to comparing the pattern of care delivered by a clinician, group, specialty or system to this national benchmark as a simple metric for comprehensiveness of care. We give examples of how the measure differs among clinicians across a range of specialties and academic versus nonacademic care settings.
The objective of this study is to distill comprehensiveness to a calculable metric that can be used to evaluate care patterns by leveraging national health care data. In this pilot study we analyze a dataset from a Regional Academic Health System (RAHS) in comparison to NAMCS benchmarks for the purpose of demonstrating feasibility of this metric. The study will be limited to data from these sources (NAMCS and RAHS) from ambulatory encounters over 12-month periods.
Methods
We compared the distribution of ICD-10 codes from Table 13 in the 2019 NAMCS5 with the ambulatory care delivered in a RAHS using linear regression. We excluded the 4 ICD-10 chapters not covered by NAMCS: perinatal care, congenital malformations, external causes of morbidity and mortality, and factors influencing health status. The NAMCS data includes prevalence of diagnoses in ICD-10 chapters estimated through rigorous process. The RAHS data are actual data from clinical practice in a large regional health system over a single year when 2,170,000 patients were seen in 8,510,000 visits, resulting in a total of 26.1 million ICD-10 codes.
Because this study does not involve human subject research the university IRB waived review of the project.
We estimated a comprehensiveness metric (CM) of care of selected medical specialties at the RAHS using the mathematical inner product of the 2 distributions. This approach assumes that the elements of each distribution (the amount of care in each chapter) are independent. The inner product of 2 vectors is a way to quantify their similarity. The inner product is calculated by multiplying each component of one vector by the corresponding component of the other vector, and then summing up these products. The component values are the weights of the care distribution in any one ICD-10 chapter normalized by the root mean square of the weights. The inner product between any 2 normalized samples ranges from 0 to 1.
Results
The pattern of ambulatory care described by ICD-10 codes for a large population served by a regional academic health system correlates with the national sample distribution of care. When either of these is used as a reference for fully comprehensive care, the inner product between these distributions and the care delivered by primary care specialties is closer to the reference for comprehensive care than care delivered by other specialties.
Figure 1 shows the correlation between the proportion of diagnoses in each chapter of the ICD-10 code distribution from NAMCS 2019 (x-axis) and the distribution observed for all ambulatory care over a one-year period at the RAHS (y-axis).
Scatterplot of the normalized proportion of diagnoses in each ICD-10 chapter (point labels) from the National Ambulatory Medical Care Survey (NAMCS) and the Regional Academic Health System (RAHS). Dotted line is a linear regression (slope 1.00, 95% CI 0.57:1.43, P = .0002, R2 0.62).
Table 1 shows the CM for a selection of clinician groups including a range of primary care clinicians and the top 6 specialty clinicians at the RAHS based on of number of visits. For each clinician group, the inner product between the NAMCS reference pattern (CM NAMCS) and the RAHS (CM RAHS) reference pattern from all ambulatory care visits is shown. The Mann Whitney U test was used to compare the values of CM because CM is a continuous variable and it is not safe to assume the data are in a normal distribution. When using either the NAMCS reference pattern or local pattern based on all ambulatory care, the values for primary care were statistically different from the specialties using a test (P = .04 for NAMCS, P = .02 for local). When comparing primary care between local and benchmark datasets, there was not a statistically significant difference (P = .3). Similarly, when comparing the specialties across local and benchmark datasets there is no statistically significant difference. (P = .3).
The Inner Product of the Normalized Distributions of ICD-10 Codes for Clinical Disciplines Using NAMCS Distribution (CM NAMCS) or All Ambulatory Care at the Regional Academic Health System as a Reference (CM RAHS)
Discussion
The correlation between the distribution of care based on ICD-10 codes from a national sample and from a large regional health system from different years points to the potential generalizability of this approach to measuring the comprehensiveness of care. Using the inner product to compare any distribution of care to the distribution of care from all specialties in a large diverse population serves as a measure of comprehensiveness.
The significant correlation (slope 1.00, 95% CI 0.57:1.43, P = .0002, R2 0.62) of the pattern of care delivery between the RAHS and NAMCS points to the possible generalizability of this metric. This distribution will be a better estimate of comprehensive care to the degree that it includes a large population and encompasses all the specialties that contribute to the care of a population. We anticipate variations in the proportion of care in the different chapters due to regional variation of disease burden and practice pattern which may point to the role of a local reference.
The table shows the variation of the metric when using the national and the RAHS all ambulatory care as the reference for full comprehensiveness. In general, the metric is a higher number when using the RAHS data as the comparison. This is reasonable because practice patterns in a regional system are likely to be more homogeneous compared with national practice patterns. Independent of which reference is used, the primary care disciplines are distinct from the specialty disciplines, have higher comprehensiveness metrics and have the same order of ranking. With respect to the metric for the specialty areas, the key observation is that the specialized areas have a significantly lower comprehensiveness. This is what is expected and points to a potential use of this metric as an indication of how far the practices of a particular clinicians or practice deviate from primary care. A “primary care” clinician that has a relatively narrow care pattern will have a comprehensiveness metric that looks more like a specialist. The relative order of the specialty disciplines is different when using the 2 different reference data sets. This likely points to how the practice patterns at the RAHS deviate from the patterns represented by the national NAMCS data.
In the setting of a subset of primary care disciplines (Family Medicine, Internal Medicine and Pediatrics) this metric shows some level of variation between clinicians in different specialties. The relative ranking is reasonable given that internal medicine clinicians do not provide pediatric or obstetric care and pediatric clinicians do not provide obstetric care or care for adults. Similarly, while OB/GYN clinicians are also primary care, their care is limited to women’s health conditions and thus most of the care they provide falls into a small set of ICD codes overall. This metric is significant because it is a demonstration of the comprehensiveness of care in Family Medicine when compared with other primary care disciplines.
The statistical comparisons of primary care to specialty care show that, independent of which reference is used, the metric distinguishes primary care from specialty care. We also observe that the CM values for primary care are statistically the same independent of which reference is used. The same is true for the CM for the specialties. These observations indicate that the CM metric to distinguish primary care from specialty practice patterns is not sensitive to which reference pattern for comprehensive care is used.
This is a pilot study and the first report of this approach to estimating comprehensiveness using a simple to calculate metric. Future studies can explore how sensitive this metric is to variables including clinician group size (this can be calculated at the individual clinician level or any number of clinicians), evolution of practice through training and after, patient outcomes, quality and cost of care, and relationship between this metric and other measures including clinician wellness and job satisfaction. This estimate may have particular value in the setting of training as a means of estimating comprehensiveness and scope of care where those are critical goals of training.6–8
Limitations
Exclusion of Specific ICD-10 Chapters: The decision to exclude four ICD-10 chapters from analysis may omit significant care, especially in perinatal care and congenital malformations. The other excluded chapters are external causes of morbidity and mortality and factors influencing health status. These codes were eliminated to enable direct comparison to the NAMCS data which does not include them.
Potential for Institutional Bias: The comparison between national patterns of care and data from a single institution (RAHS) might introduce bias related to specific institutional practices, policies, or patient populations. Similarly, the analysis may not generalize to geographical variations in healthcare delivery that could affect the availability and type of services provided.
NAMCS Limitations: NAMCS has limitations due to sample size and selection bias. It is also limited in the number of diagnostic chapters that are reported and does not break out patterns of care by specialty discipline. However, it remains a valuable proxy for national patterns in ambulatory care.
Conclusion
This study presents a feasible approach to evaluating comprehensiveness by leveraging national survey data as a benchmark. The insights garnered from this methodology and findings provide a strategic direction for health care systems aiming to align more closely with national care delivery standards, fostering a more universally comprehensive health care model.
Notes
This article was externally peer reviewed.
Funding: This work was supported by internal funds at the University of Pittsburgh.
Conflict of interest: None.
To see this article online, please go to: http://jabfm.org/content/38/3/561.full.
- Received for publication July 1, 2024.
- Revision received January 23, 2025.
- Accepted for publication January 29, 2025.
