Abstract
Background: Significant investments and effort are being devoted to health care reform, yet little is known about the costs of improvements. Practical tools are needed to allow for systematic assessment of practice expenses. We report here a field trial of a standardized expenditure data collection instrument.
Methods: Combining economic and primary care practice consultation, an expenditure data collection instrument was created. The instrument underwent observed feasibility testing and was fielded by 10 practice-based research networks in 30 practices conducting 10 different health behavior change interventions.
Results: Start-up and operating expenses were successfully collected for 87% and 97% of the practices, respectively. Data collection time and effort were considerable but acceptable. Three elements were necessary to collect expenditure data: (1) an intervention-specific data collection instrument, (2) a field guide, and (3) economic oversight and assistance. Fully 90% of networks reported that they planned to collect expenditure data in the future and study participation increased the likelihood of their participation in a future expenditure study.
Conclusions: It is feasible to systematically collect intervention-specific expenses in primary care using formal expenditure methods. However, most practices and researchers lack the knowledge, expertise, and resources to collect such data independently. Further assistance and education is necessary to obtain reliable information about the expenses to transform and improve primary care.
Quality improvement, evidence translation, and practice redesign are necessary to improve the delivery of primary care in America. Examples range from payers incentivizing and mandating e-prescribing or quality performance reporting to calls for transforming existing practices into patient-centered medical homes.1–5 These changes have great potential for benefit, yet the costs and resources required for implementation remain unknown. Without information about additional expenses associated with redesign, practices cannot stipulate the financial requirements they bear to implement systems and services, payers can claim sufficient payment has already been made, and policymakers can defer action because judgment about value is not possible.
As new models of care and revamped reimbursement systems emerge,4,6,7 primary care practices need to use established financial and economic methods to understand the expenses associated with the provision of care. These same methods will also need to be used to understand comparative effectiveness. Collecting this information will require primary care researchers and clinicians to become more knowledgeable and skillful in collecting expenditure information. Currently there are few tools designed to collect and analyze expenditure data in primary care settings.
Prescription for Health (P4H), a 6-year, 2-cycle (2003 to 2004 and 2005 to 2007) initiative of the Robert Wood Johnson Foundation, funded 22 primary care practice-based research networks (PBRNs) to develop and test 27 practical health behavior change interventions.8–10 The focus of the initiative was to develop and test whether these interventions improved patients’ health behaviors.11–14 However, the funders recognized that, for these interventions to be translated outside of the research setting and into the real world, an understanding of expenses incurred by practices to implement and field the interventions was necessary. As a result, credible practice expenditure estimates were systematically collected using a standardized instrument by all 10 PBRNs participating in the second cycle of the initiative. These expenditure results, which have been previously published, clearly demonstrate that implementing and fielding health behavior change interventions is not free for primary care practices. In fact, practices bore a mean monthly expense of $58 (range, $1-$354) per patient who participated in the intervention.15 This article reports on the development, feasibility, and our experience using a standardized expenditure data collection instrument as a model for systematically gathering expenditure data for implementing practice innovations.
Methods
We conducted a 4-step practical field trial to create and field an expenditure data collection instrument combining economic and primary care practice expertise. The steps included creating a preliminary version of the instrument, pilot testing and modifying the instrument, fielding the final instrument in 30 practices from 10 PBRNs that were conducting different health behavior change interventions, and surveying the PBRNs after intervention to evaluate their experiences with the collection of expenditure data.
Instrument Development
An expenditure steering committee, with expertise in economics, primary care, and the P4H initiative, was assembled to develop a common and generalizable method to assess expenditures that would allow for aggregation and assessment of information across the 10 funded projects. The steering committee consisted of an economist, the primary care researchers conducting the P4H studies (PBRN members), and members of the P4H National Program Office. Limited resources were available to fund the collection and analysis of expenditure data ($5,000 per network) so a simple, generalizable, and actionable process was required.
The overarching research question guiding the steering committee was, “What are the incremental in-practice expenses attributable to the P4H interventions?” Guided by this central question and after consideration of alternative standard economic and financial methods (cost minimization analysis, cost-effectiveness analysis, cost utility analysis, cost benefit analysis), the committee determined that the most appropriate methodology was an expenditure analysis.16 The committee did not opt to use a more simple costing method; they chose instead a formal expenditure analysis approach to generate more valid, reproducible, and generalizable findings. Similarly, the committee decided not to use modeling, forecasting, or estimations; they chose instead to conduct a prospective observational evaluation of actual practice expenses. The perspective adopted (ie, who bore the expense) included only practice expenses and not researcher, community, or patient expenses. Intervention development expenses were excluded, making the assumption that once these interventions were developed practices would only need to incur expenses to tailor and implement the existing intervention to their location. We also elected to not assess opportunity costs.
Next the committee combined formal expenditure methods with the language and needs of primary care to create a standardized expenditure data collection instrument.16–19 The instrument consists of one figure (a patient flow diagram), 3 Excel (Microsoft, Redmond, WA) tables (“Start-Up Expenses,” “Staff Salaries,” and “Operating Expenses”), and a field guide. The instrument includes an example of a patient flow diagram with instructions in the field guide about how to create a customized diagram for an individual intervention and use it to organize data collection in the 3 tables. The patient flow diagram is used to record the number of patients that complete each intervention step. From the diagram, the number of patient participants is transferred to the operating expense table to calculate recurrent expenses (ie, patient care activities); additionally, information about nonrecurrent expenditures on capital assets and overhead expenditures are recorded. Patient flow, operating expenses, nonrecurrent expenses, and overhead expenses are collected at baseline (before starting the intervention) and at 2 steady-state time periods after intervention. By subtracting each steady state expense from baseline expenses and averaging the differences, the average incremental intervention expense may be calculated. The field guide defines economic terms, clarifies the steps needed to collect the expenditure data, and describes how to categorize expenses for appropriate incremental expenditure calculations. An example of a completed intervention-specific patient flow diagram and operating expense table are provided in Figures 1 and 2. An example of the instrument and a field guide, describing how to use the instrument in detail, are available online.20
Completed patient flow diagram. This is a completed intervention-specific patient flow diagram for one of the participating Prescription for Health practices. The expenditure steering committee and the practice-based research networks created the flow diagram in Microsoft Excel. Each practice-based research network then asked 3 representative participating practices to complete the diagram. Practices entered the number of patients completing each intervention step at baseline and during 2, 1-month steady-state periods of the intervention. Data in Figure 1 is for one 1-month steady-state period. The main purpose of this diagram was to organize and frame the data collection for baseline and operating expense figures (Figure 2). BMI, body mass index.
Completed operating expense table. This is a completed intervention-specific operating expense table. Similarly, the structure of this table was created by the expenditure steering committee and the practice-based research networks. Practices then used the flow diagram (Figure 1) to complete the number of patients eligible for and completing each intervention activity in section A of this table. Practices used additional data sources to complete the remaining cells in sections A, B, and C. Data shown in here is for same 1-month steady-state period used in Figure 1. As with Figure 1, practices would complete this table for a baseline period and for 2, 1-month steady-state periods. By subtracting expenses for each 1-month steady-state period (1 month shown in this table) from the baseline expenses (not shown) and averaging the values, the economist could calculate the incremental expenses for each practice's intervention. BMI, body mass index; FTE, full time equivalent; P4H, Prescription for Health.
Pilot Testing
The preliminary instrument was piloted in 6 practices that had previously implemented the same health behavior change intervention during the first cycle of P4H (a website with patient resources).21,22 Initially, one office manager was asked to complete the instrument while being observed and timed (phase 1). After refining the instrument based on phase 1 findings, all 6 of the practice's office managers were asked to complete the instrument (phase 2). Their responses were compared with the practices’ financial records for accuracy and they were surveyed about the process; this information was used to finalize the instrument for fielding during the second cycle of P4H.
Instrument Fielding and Assessment
Ten PBRNs were funded during the second round of P4H. Each PBRN designed and evaluated a different health behavior change intervention in a total of 75 practices across the nation.23 The practices included private practices and community health centers, large (>10 physicians) and small practices in rural, urban, and suburban locations. The interventions included health information technology interventions, modified or new staff roles, and partnerships with existing community resources.8,11–14 The 10 funded PBRNs were asked to select 3 representative practices and prospectively assist the practices in collecting expenditure data using the instrument while concurrently implementing and evaluating the intervention's effect on patient health behaviors. With the assistance of the PBRN, each practice determined the specific data sources necessary to complete the expenditure instrument and collected the data. Either the practice or the PBRN could enter the data into the expenditure instrument. Throughout data collection, the expenditure steering committee was available to clarify the instrument and the economic methods for the PBRNs and practices. In addition, the committee reviewed all completed instruments with the PBRNs to ensure accuracy and assist in fixing any errors. Data were collected for 1 month at baseline before fielding the intervention, during the entire start-up period as practices prepared to field the intervention, and during 2 separate 1-month intervals during steady states of intervention delivery. After expenditure data collection were complete, all PBRNs completed a brief survey about the process of collecting expenditure data using the standardized instrument.
Results
Pilot Testing
During the first phase of pilot testing, one office manager was provided a generic flow diagram and a generic set of tables that included the entire range of potential expenses. The nonspecific flow diagram and extraneous cells in the comprehensive tables created substantial confusion and resulted in the office manager only being able to complete the instrument with detailed guidance. For the second phase of pilot testing, the instrument was modified to include only intervention-specific steps and expenses in the flow diagram and tables. Office managers were able to complete the instrument without direction from the steering committee and reported a mean of 93 minutes needed to assemble the expenditure data and complete the instrument. The expenditure data recorded by the office managers using the instrument were consistent with practice financial records and the 6 office managers independently recorded similar types and values for expenses, with 3 exceptions: each office separately reported costs shared across the offices, resulting in duplicate reporting; one office did not report staff known to be involved in delivering the intervention; and one office reported a 2- to 5-fold greater length of time for a patient to go through specific intervention delivery steps than the other practices. In response, the steering committee decided that the economist would review all data, validate or correct data outliers with each PBRN, and consolidate duplicate entries of shared expenses.
Instrument Fielding
The expenditure steering committee distributed a customized expenditure data collection instrument and field guide to each of the 10 PBRNs. All PBRNs were asked to review these documents and report any discrepancies and further customization that was necessary. The steering committee assisted PBRNS that required changes to their instruments or field guides, at times developing several versions of these documents until the PBRNs deemed them accurately representative of their interventions. An example of a completed patient flow diagram and steady state expenditure table with a description of how the instrument was used is provided in Figures 1 and 2. Although PBRN input was essential to the creation of the customized instruments, significant assistance was required from the steering committee to clarify economic terms, create a flow diagram and expenditure list that captured all potential expenses and intervention steps (all networks initially neglected some potential expenses and intervention steps), and categorize expenses for appropriate economic analysis (networks had difficulty differentiating recurrent expenses, overhead, and capital assets).
Once the instrument was customized, the PBRNs asked the practices to collect the specific data. At this stage, all practices required their PBRN's assistance with data collection. PBRNs needed to clarify the economic terms and prompt practices to identify all potential expenses, similar to the prompting that the PBRNs needed from the expenditure committee. Specific practice expenditure data sources tended to vary based on the intervention and included office managers, clinicians, financial staff, financial records, electronic medical records, personnel activity logs, and direct observation (see Table 1). Although all practices were able to provide expenditure information, most PBRNs provided assistance to record the data in the expenditure data collection instrument and the economist was needed to calculate the overall expenditures (see Table 1).
Reported Processes Used by Practice-Based Research Networks to Collect Expenditure Data
Ultimately, all 10 PBRNs were able to field the expenditure instrument and collect expenditure data. One PBRN was able to collect data from only 2 of the requested 3 practices and one was unable to collect start-up expenses, both because of work pressures unrelated to the study that these participating practices were experiencing at the time of data collection (eg, one practice was adopting a new electronic medical record). Overall, start-up and operating expenses were successfully collected for 26 (87%) and 29 (97%) of the 30 practices, respectively.
PBRN Survey Responses
The majority of PBRNs reported that the time and effort required for data collection was acceptable (with 1 being “very acceptable” and 5 being “acceptable”; 4 was a “neutral” response; see Table 2). Several PBRNs experienced moderate difficulties customizing their expenditure tables and figures to their intervention (6 had a “neutral” response and 4 responded it was “difficult”). The PBRNs considered that the expertise provided by the expenditure steering committee was a necessary resource (8 reported it was “very important” and 2 reported it was “important”). Overall, 90% of the PBRNs reported that they planned to collect expenditure data in the future and that their participation in the P4H expenditure study increased the likelihood that they would participate in a future expenditure study.
Practice-Based Research Network Evaluation of Expenditure Data Collection Instrument
Discussion
More research is necessary to estimate the cost of primary care and its value. The P4H expenditure study confirmed that it is possible to systematically measure primary care expenses, and the average reported P4H intervention practice expense—$58 per participant—was substantially higher than current reimbursements.15 Most PBRNs and practices were able to collect credible expenditure data while creating, fielding, and assessing novel health behavior change interventions. The time and effort were generally considered acceptable and the experience appreciably increased their likelihood of collecting expenditure data in the future.
However, both PBRNs and practices had difficulties with the formal expenditure methodology, frequently reverting to simple cost analyses, collecting absolute intervention costs rather than incremental expenses, and underestimating or neglecting real expenses. Practices frequently knew their general costs and revenue, but had trouble attributing specific resources to specific interventions. The nature of primary care makes such attributions difficult. Intervention expenses were often intermingled with other nonintervention-related office and clinical activities. Commonly, staff time for intervention activities was brief, lasting only several minutes, but was often repeated by many staff members many times per day. These expenses were not only difficult to measure, but were often overlooked. However, collectively these expenses represented significant resource requirements for practices.
Practices and PBRNs both had difficulties assigning the correct financial category to expenditures. Differentiating whether an expense was a development versus start-up expense or an intervention delivery versus research study expense was a challenge. The PBRNs had extensive research and data collection experience but had minimal experience with collecting expenditure data. The practices knew about their clinical activities and their expenses, but not about systematic data collection and expenditure methods. However, all these challenges were overcome by creating an expenditure guide to define economic terms and describe incremental data collection methods, providing a data collection instrument to standardize data collection, using the field experience of the PBRNs to coordinate and oversee data collection, and establishing a steering committee to provide external expertise and support.
The observed challenges with collecting expenses in primary care are not unexpected. Systematic collection of intervention expenses is not part of existing primary care research or practice culture. In the current relative-value scale reimbursement model, even if a practice understands their expenses they have little power to modify their fees for more appropriate reimbursement.24 Likewise, although lack of reimbursement is a frequent barrier to the ultimate dissemination of interventions created by primary care researchers, researchers are more focused on evaluating whether interventions are effective rather than generating the expenditure data necessary for dissemination.25
The practice expenditures resulting from this study demonstrate a clear mismatch between actual primary care expenses and current reimbursements, demonstrating why it is necessary to assess what practice interventions really cost, particularly for health behavior change interventions.15 Medicare currently only reimburses for smoking cessation counseling and dietary counseling (but only for diabetics) whereas many commercial insurers do not even provide this insufficient coverage.26 In 2006, Medicare reimbursed $13 to $25 for smoking cessation counseling.27 However, the average P4H intervention expense was $58 per patient participant.15 Modeling techniques are commonly used to determine practice costs and determine reimbursement rates, but models are frequently inconsistent with real practice expenses. For example, the Future of Family Medicine's financial task force reported that group visits, team-based care, and clinical practice guideline software could increase a 5-physician practice's annual revenue by $5,664 to $15,411.6 Conversely, as we reported previously, similar P4H interventions cost practices $2 to $63 per patient participant.15
In this article, we described experiences of PBRNs and practices with collecting expenditure data using expenditure methods. These experiences lead us to several suggestions to facilitate the collection of primary care expenditures:
Funders should ask for and support expenditure data collection as part of the outcomes assessment of interventions, similar to what the Robert Wood Johnson Foundation did with P4H.
PBRN researchers, who have both primary care practice and data collection expertise, can serve as a resource for collecting expenditures, both in educating practices and catalyzing change through ongoing work.
A standardized data collection instrument customized to specific interventions—such as the one we used in P4H—can systematize data collection, allowing for the collection of expenditures that are consistent, valid, and generalizable.
External economic expertise and oversight is necessary to assist with the collection and analysis of expenditure data.
Payers need to incentivize collection of expenditure data and use this information to ensure appropriate practice reimbursement.
This article reports on a process and the experiences of collecting expenditure data in primary care practices for health behavior change interventions. Fully, 30 practices from 10 PBRNs from across the nation participated in the study, providing robust and generalizable findings about the experience. An expenditure data collection instrument was used to standardize data collection, which can serve as the groundwork for the development of future expenditure data collection instruments. In addition, the instrument could be used to collect additional expenses that we chose not to collect (eg, intervention development, opportunity costs, etc) or expense perspectives that we did not consider (eg, patient, community programs, payer, etc).
Although the reported findings have relevance to intervention dissemination and practice redesign, our study does have several important limitations. We conducted a prospective observational study with no comparison group. The PBRNs selected representative practices for participation and probably selected practices that they presumed would be successful with expenditure data collection. Despite the participation of such practices and the creation of the expenditure data collection instrument, we found that both practices and PBRNs had difficulties with using expenditure methods. Although this seems like a limitation, meaning that the average primary care practice could not take our instrument and field it in practice without assistance, this is a key finding that highlights the need for enhanced knowledge and expertise in primary care to generate expenditure data in the future.
Conclusion
Economic and financial research can add value in primary care and will be a necessary component of future work involving comparative effectiveness research, translation of evidence, and creation of medical homes. Although more work is necessary to develop and refine data collection tools and primary care practices and researchers need training and experience with systematically collecting expenses, the P4H experience suggests that collecting expenditures is feasible.
Acknowledgments
The authors would like to thank the individuals who assisted with the development and assessment of our expenditure instrument (Doug Fernald, MA; Sue Flocke, PhD; Jodi Holtrop, PhD; Abhi Kole; Kurt Stange, MD, PhD; Tom Summerfelt, PhD; and Alfred Tallia, MD, MPH) as well as the innovating PBRNs and their participating practices, without whom this study would not have possible.
Notes
This article was externally peer reviewed.
Funding: This study was funded by The Robert Wood Johnson Foundation under the Prescription for Health Initiative.
Conflict of interest: none declared.
- Received for publication April 21, 2009.
- Revision received September 24, 2009.
- Accepted for publication September 30, 2009.
