Abstract
Objectives: The discipline of family medicine seeks to build its research enterprise. To assess the state of family medicine research in the United States, this study identifies and describes research articles published by family medicine researchers from the United States in 2003 and assesses the growth in articles, authors, and publishing journals since 2000.
Methods: We searched for all research articles published in 2003 in scholarly, English-language journals authored by individuals in US family medicine organizations and by family physicians from the United States. Search approaches included a hard copy review of 22 journals and Medline searches of articles by family medicine authors and organizations. Similar search approaches, previously reported, were used to identify articles published in 2000.
Results: For 2003 we found 790 research articles dispersed across 285 journals from 801 family medicine researcher-authors. Twenty-nine journals published 6 or more family medicine research articles; 159 journals published just 1 research article from the discipline. Family medicine journals published 18% of the discipline's research articles. People in academic departments authored the vast majority (89%) of the discipline's research. Between 2000 and 2003 family medicine's research articles increased by an estimated 58%, its authors increased by 41%, and journals used increased by 82%.
Conclusions: Family medicine's research enterprise in the United States is larger and more productive than generally recognized, and it is growing. Nevertheless, family medicine likely publishes fewer research articles than some other clinical disciplines.
Building a healthy research enterprise is central to family medicine's ability to provide the best possible care to patients and is important to the future of the discipline.1,2 Although family medicine benefits from the research of other clinical disciplines and from research in the basic and social sciences, its practitioners also need answers to clinical questions from studies involving family medicine's own patient populations and practice settings.3–6 Similarly, innovations in the organization of family physicians’ offices and in the training of future family physicians depend on data from family medicine's health services and education researchers. Some observers within and outside the discipline have felt that family medicine does not carry out enough research7–10 and that increasing financial and other pressures within its academic departments are making research more difficult.11–12 Recognizing these concerns and the discipline's need for research, family medicine's national leaders have lobbied to support the discipline's researchers with more funding, training programs, and publishing venues.1,13–16
Whether family medicine's research enterprise is healthy and growing or succumbing to external pressures should be understood and monitored. The health of a discipline's research enterprise—having a sizable and productive researcher workforce, committed and capable research institutions, adequate research funding, and vibrant scientific methods—is likely best reflected in what it produces, ie, by what is seen in the corpus of the discipline's published papers. A large number of research articles published by a broad body of researchers and institutions in journals reaching a wide and appropriate audience reflects a flourishing research enterprise. Many biomedical disciplines have assessed the state of their research efforts by using a variety of bibliographic search techniques to identify their fields’ published articles and then analyzing them.17–19 Prior evaluations in family medicine have assessed the discipline's published “scholarship” without distinguishing research reports from other forms of writing, like clinical reviews and editorials,7,11,20,21 or have assessed family medicine's research published only within the disciplines’ handful of journals.10 Only one previous effort has attempted to characterize family medicine's complete research output, a study we conducted in collaboration with the North American Primary Care Research Group Committee on Building Research Capacity, which identified all research articles published by family medicine organizations in the United States in the years 1999 and 2000. This earlier study found that, when viewed in its entirety, family medicine's research output is greater than generally assumed.22,23 The current study updates this earlier investigation by identifying and describing research articles published by the discipline of family medicine in the United States in 2003, and estimating the growth in papers and publishing authors, institutions, and journals since 2000. We define the “discipline of family medicine” as all family physicians and all people working in family medicine's organizations, including family medicine academic departments and family medicine residencies.
Methods
Eligibility Criteria
Eligible Journals
Research articles were eligible when published in the United States or in international scholarly English-language journals in 2003 in print or electronic formats (Table 1). Scholarly journals are periodicals whose purpose is to add to the knowledge base of specific fields, that publish articles submitted by authors not on the staff of the journals themselves, that are published by a scholarly or professional organization or university, and whose readership is the researchers and/or practitioners of their fields.24,25 This excluded articles appearing in popular magazines, news, and general interest periodicals; non-periodicals (eg, books); and those produced without a publisher (eg, committee reports and working papers).
Eligibility Criteria for US Family Medicine's Research Articles in 2000 and 2003
Articles Eligible as Research
We classified articles as research if they demonstrated a systematic approach to gathering and analyzing data, consistent with federal definitions.26,27 Articles eligible as research included traditional hypothesis-driven and exploratory studies, evaluations of educational and service programs that report systematically gathered outcome data, and meta-analyses and other types of formal literature synthesis that systematically assessed (as data) methods and/or findings across studies. Ineligible articles included traditional (nonsystematic) clinical reviews, editorials, synopses of articles published in other journals including “patient-oriented evidence that matters,”28 book reviews, and virtually all letters to the editor. Although case reports of 1 or 2 subjects were ineligible, case series that provided descriptive statistics of pooled subjects were eligible, as were case studies that presented detailed descriptions of educational, community, or clinical interventions with outcome data. Some Family Physicians Inquiries Network reports29 were eligible, but most loosely synthesized the scant data available on their topics and, therefore, were ineligible as research. Articles presenting research were eligible regardless of the nature of their research questions, their relevance to practicing family physicians, their study methods, or the quality of the research.
Eligible Authors
Eligible authors were those, regardless of their academic degrees or disciplines, working in US family medicine organizations. This included academic family medicine departments (allopathic and osteopathic), family medicine divisions of joint departments (eg, departments of “family and community medicine” and “family and preventive medicine”), family medicine residencies set within nonacademic medical centers, family medicine clinical departments of hospitals, and other family medicine organizations (eg, family medicine-run practice-based research networks and the American Academy of Family Physicians). In addition, all family physicians from the United States were eligible regardless of where they worked. Those who were not family physicians but had secondary academic appointments in family medicine departments were eligible; those with adjunct appointments were not.
Steps in Identifying Published Research
To identify the discipline's research published in 2003, we used a variety of complementary and overlapping search approaches that, when followed sequentially, added fewer and fewer new papers, suggesting that there were fewer articles remaining unfound after each approach. The search approaches were generally the same as those we used when reviewing publications in 1999 and 2000,22 but with some changes made principally to improve search efficiency.
We previously found that, in 1999 and 2000, 18 journals published 51% of the discipline's research articles.22 We began our search for 2003 articles with an issue-by-issue hard copy search of these 18 journals, along with the 4 family medicine journals newly publishing research since 2000 (Fig 1). We next conducted a series of searches of the National Library of Medicine's Medline bibliographic database using the search limits of 2003 publication year and English language. We first searched the affiliation field (ie, organization) for all authors, whether listed as first author, second author, etc, under the terms <“family” and “practice”> and <“family” and “medicine”>. We then electronically searched for 2003 articles under the last names and first initials of all 385 authors of 2 or more research articles in 1999 and 2000. We tried searching under the names of a sample of 75 of the 409 authors of only one research article over the 2 earlier years, but this yielded few articles for 2003 so we did not search the rest of the names. We next searched under the names of 2003 to 2004 members of the North American Primary Care Research Group from the United States who had not been searched up to this point (n = 417), and then searched under the names of the approximately 200 new family medicine co-authors of eligible articles identified thus far for 2003. We concluded the process by searching under the names of new authors suggested to us by the chairs of 33 academic joint departments (see below) and by adding new articles found in compendia provided to the study by a sample of 15 US family medicine departments (see below).
Sequence of steps taken to identify US family medicine's 2003 research articles and the number of new articles found at each step.
Steps in Assessing Article and Author Eligibility
We obtained a complete hard copy of every potentially eligible article and 2 reviewers independently assessed its eligibility as “research.” A third investigator independently reviewed and concurred in all instances with the eligibility designations of a sample of 120 articles deemed to be research and the ineligibility determinations of 107 articles deemed not to be research.
The eligibility of authors was determined principally by their institutional affiliations noted on their articles. In ambiguous cases, authors’ affiliations were clarified through their organizations’ web sites and/or we verified that authors were family physicians and, therefore, automatically eligible, using on-line physician databases.
It was sometimes unclear whether individuals working in a joint department, such as a “Department of Family and Community Medicine,” were in its family medicine division. We surveyed the chairs of 41 joint departments and asked them to confirm whether each author we identified from their department was in its family medicine division, was a family physician, considered him/herself to be a “family medicine researcher,” and whether the chair felt we should count this person as a family medicine researcher. They also suggested names of other 2003 authors in their department. Thirty-three usable surveys (80%) were returned.
Differences in Search Approaches Used in 1999/2000 and 2003
Compared with the present study, the earlier study that assessed family medicine's research published in 1999 and 2000 used identical author and article eligibility criteria and the same general search strategies, but differed at 3 search steps22: (1) for 2003 we started with a hand search of only 22 journals, whereas for 1999 and 2000 we began with a hand search of 80 seemingly relevant journals; (2) for 2003 we included electronic searches of the names of authors of 2 or more eligible articles in 1999 and 2000; and (3) for 2003 we electronically searched the names of North American Primary Care Research Group members who had not already been searched.
Analysis
Analyses were primarily descriptive, presenting total and subgroup counts of identified research articles, family medicine authors and their organizations, and journals for 2003. Tallies were made of the articles published in “family medicine journals,” defined as journals with “family medicine,” “family practice” or “family physician” in their titles, and in the 5 weekly general medicine journals, specifically, The Journal of the American Medical Association, New England Journal of Medicine, BMJ, Lancet, and Annals of Internal Medicine. Comparison figures for numbers of articles, authors, and journals for the specific publication year of 2000 were generated from the databases of articles and authors previously reported jointly for 1999 and 2000.22,23
We assessed how completely the group of eligible articles identified through the above steps reflected all research articles published by US family medicine organizations and family physicians in 2003 and in 1999/2000. Many departments maintain lists of their publications that they perceive to be reasonably complete; these lists have been used as a source of “all” publications in previous research.21 We obtained compendia of publications from a purposive sample of academic departments for which we had found few, some, or many eligible articles and that were located in all regions of the United States. We assessed the research eligibility of each article on departments’ lists and then assessed the proportion of these research articles that had been found through our online and hard-copy search processes at each time period.
This study using principally publicly available data were submitted to and exempted from human subjects review by the Office of Human Research Ethics of the University of North Carolina School of Medicine.
Results
Eligible Research Articles
We found 790 research articles by US family medicine researcher-authors published in 2003 (Figure 1). Seventy percent were found through the first 2 steps of hard copy searches of 22 journals and electronic searches for “family” plus “medicine” or “practice” in the author affiliation field. Four hundred seventy-seven of the 790 articles (60%) listed a family medicine researcher as the lead author.
Authors
In 2003, 801 US family physicians and others working in family medicine organizations authored research articles (Table 2). Three hundred forty-four (43%) of the 801 authors were lead authors on at least one research article. Almost two-thirds (64%) of family medicine researcher-authors were listed on just one paper and, on average, family medicine researcher-authors were listed on 1.88 research articles. The most prolific family medicine researcher authored 16 research articles; 57 people authored 5 or more research articles.
Numbers of Family Medicine Researcher-Authors and the Number of Articles they Authored in 2003
Most family medicine researcher-authors had MD degrees (56%); of these, one-third also had master's or doctoral degrees (Table 2). Adding in those with DO, MB, and MBBS degrees, physicians as a whole comprised 58.3% of all family medicine researcher-authors. Nonphysicians with doctorates constituted the second largest group of authors (23%).
Journals
Research articles from US family medicine authors were published in 285 different journals in 2003 (Table 3). Most of these journals (56%) published only one such article that year. These one-article journals included those with both prominent and limited circulation that were focused on specific diseases, organs, and disciplines (eg, Hypertension, Contraception, Thyroid, Circulation, and Urology); journals from state medical societies, and international journals (eg, European Journal of Public Health and Scandinavian Journal of Primary Health Care).
Journals Publishing Family Medicine Research Articles in 2003
Twenty-nine journals published 6 or more family medicine research articles in 2003. These 29 journals together published more than one-third of the discipline's 790 research articles. “Family medicine journals” published 147 research articles in 2003, which was 18.0% of all research articles. Twenty-two articles (2.8%) were published by the 5 weekly general medicine journals.
Organizations
The vast majority (n = 699; 89%) of the discipline's research articles in 2003 listed one or more authors from a family medicine academic department (Table 4). Forty-nine articles (6%) listed authors affiliated with non-university hospital-based residencies and 90 articles (11%) listed family medicine authors working in all remaining settings combined.
Number and Percentages * of Family Medicine Research Articles from the Various Types of Organizations in 2003
Authors in relatively few of family medicine's allopathic academic departments produced a disproportionate amount of the discipline's published research (Table 5). Thirteen of the 98 allopathic family medicine departments that published research articles in 2003 together authored 294 articles, or 37.2% of the discipline's 790 total research articles. In contrast, 52 (54%) of publishing allopathic departments published 5 or fewer articles, together accounting for 134 articles, or 17% of all articles.
Research Article Output from Allopathic Academic Departments of Family Medicine in 2003
Authors in 4 osteopathic departments of family medicine were listed on 9 research articles in 2003. People in 32 nonacademic medical center-based or administered family medicine residencies (some with university affiliations) were listed as authors.
Comparisons to Data from 2000
The 790 eligible research articles found for 2003 were 58% more than the 499 articles found for 2000 (Table 6). From 2000 to 2003 there was also a 41% increase in the number of family medicine researcher-authors (from 569 to 801), an 82% increase in the number of journals that published family medicine research articles (from 157 to 285), an 81% increase in the number of journals that published 6 or more articles (from 16 to 29), and a 57% increase in the number of articles published in top-tier journals (from 14 to 22). Conversely, the number of research articles published in family medicine journals decreased 19% (from 176 to 142), which, when coupled with the increase in total number of articles, yielded a 49% decrease in the proportion of all family medicine research articles published in family medicine journals (from 35.3% to 18.0%). The proportion of articles published by academic departments remained unchanged at 89%.
Comparisons of Articles, Authors, Journals, and Departments for 2000* and 2003†
Evaluating the Completeness of the Group of Articles Found
We counted 122 eligible research articles from 2003 on the compendia of publications we obtained from 15 academic departments. One hundred (82%) of them had been identified through this study's search approaches. Our search processes for the 2 years 1999 and 2000 identified 146 of 184 (79%) of the eligible articles contained on the compendia of 12 academic departments. The capture rates of eligible articles of the 2 periods, therefore, seemed comparable. With approximately 20% of eligible articles missed, we estimate that the full number of research articles published by US family medicine authors in 2003 was 964.
Discussion
Family medicine is a broad discipline with no well-circumscribed field of scientific inquiry to target with key words in online searches, no compendium with the names of all its researchers whose work can be searched, and no finite searchable list of journals that its researchers use. This study, therefore, used a variety of search strategies to find research articles from 2003 from family physicians and others in family medicine organizations in the United States. We found far more articles (n = 790) from more family medicine researcher-authors (n = 801) published in more journals (n = 285) than had been found in 2000 and more than in any previous study of family medicine's publications, research or otherwise.
That research from US family medicine researcher-authors appeared in 285 domestic and international journals in a single year in large part explains why the scope of their work is not generally recognized.8,11 Evaluations that assess the content of only family medicine-specific journals9,30,31 will now miss approximately 80% of the discipline's published research; such studies are useful in assessing the contributions of family medicine journals but they will not reflect the discipline's overall research.32 Although broad disciplines like family medicine may be especially prone to publishing outside of their discipline's journals, all disciplines are likely doing so; half of the research published from 1961 to 2005 by the seemingly focused field of nephrology appeared in non-renal journals.33
With family medicine's researchers using many journals, “family medicine journals” delivered only 18% of the discipline's research to readers in 2003. There have been concerns that family medicine's researchers are sometimes unable to publish their work because of a shortage of available journal space. Our data demonstrate that there is an ample number of journals available for the discipline's work. A more appropriate question is whether there are enough journals and journal space for family medicine's researchers to reach practicing family physicians with relevant and important study findings. The answer to this question, however, depends more on clinicians’ journal reading preferences and their other approaches to acquiring information—clinicians prefer review articles to primary research34—than on numbers of journals willing to publish work from the discipline. Other important influences include how often family medicine researchers now publishing quality work in non-family medicine journals would have preferred to publish in family medicine's journals and in which type of journals their work reaches the largest and most appropriate audiences and has greatest impact.
Are nearly 1000 research articles per year the right volume for the discipline? There are no set benchmarks. With over 207 million office visits each year to over 80,000 family physicians in the United States,35,36 we might assume that one paper for approximately every 200,000 office visits and 80 practitioners is too few. Comparisons with publishing volumes for other clinical disciplines provides some perspective. One study18, using search methods reasonably close to ours, counted 4287 cardiology research and review articles in 77 Medline-indexed cardiology journals in 2002 with first authors from the United States, and presumably (although not verified) most of these authors were cardiologists or researchers in cardiology organizations. A similarly structured study17 found 2031 research and review articles in 2003 in 30 respiratory journals that were by first authors from the United States, of whom most probably worked in pulmonary divisions and departments. Although not fully comparable to our study in their search methods, these studies from other disciplines likely reflect a volume of research articles that is several times greater than the 477 articles we found for 2003 from US family medicine first authors.
Despite the interest of some within the discipline in seeing all family physicians integrally involved in the generation of new knowledge1 and requirements that all family medicine residents receive training in research,37 these data show that, as in other disciplines, the vast majority of family medicine's research comes from people in its academic departments. Only 19 articles in 2003 were from practitioners in purely clinical settings and only 49 were from people working in residencies outside academic centers. Although all family physicians should know how to critically read the literature and use data to assess outcomes in their practices, these data indicate that nonacademic family physicians are not the ones creating and disseminating knowledge. This suggests that the discipline should strategically focus its research and development efforts on its academic departments.
There was great variation across academic departments in the number of research articles published in 2003, with the 13 most productive departments authoring over one-third of the discipline's total published research. It is not known whether family medicine's research is concentrated in fewer high-producing departments than research in other clinical disciplines and if family medicine's success in research is therefore at greater risk from any downturn in circumstances at these relatively few centers. It remains to be seen whether some departments that now publish just a few research articles each year can, with the right supports, produce more.38
Our data do not tell us how often family medicine's published research is read by and influences other researchers, clinicians, educators, and policy makers, or how much it ultimately contributes to people's health. It is also not clear whether family medicine's growing research output is serving the information needs of the discipline's practitioners or if it is answering the discipline's call for new knowledge. Given the nearly 300 journals where it is published, family medicine's research is clearly reaching many outside the discipline, and perhaps this is where it is having greatest impact.
There have been concerns that the mounting financial stresses on academic medical centers and time pressures on faculty are causing family medicine's research enterprise to flounder.11,12 Data from this study indicate that this is not occurring: from 2000 to 2003 the discipline's published research output grew by more than 50% in number of articles, 41% in number of family medicine researcher-authors, and 82% in number of journals used. Research growth in the face of financial adversity may be because of a growing discipline-wide culture that values research and creating a stronger evidence base for what family physicians do,1 which could be motivating people and departments to prioritize research despite its challenges.
The discipline's recent growth in research is likely also because of, in part, family medicine's 20-year investment in building its research enterprise. This investment has included the American Academy of Family Physicians and its Foundation's support for research centers of excellence, the Joint Grant Awards Program,39 the Robert Graham Center,40 practice-based research networks,41,42 and the Grant Generating Project, the latter in partnership with the North American Primary Care Research Group and the Society of Teachers of Family Medicine.16 Successful advocacy has also helped create Federal and foundation fellowship programs to expand training opportunities for primary care researchers.43–45
Limitations
This study, begun in 2004, assessed research published in 2003. It is not known how the discipline's research has fared since 2003. This study did not assess family medicine's production of other types of written scholarship like book chapters, editorials, and clinical reviews and did not assess the numerous research articles from family medicine investigators in other countries.7
Two new search steps used for 2003 could have inflated growth estimates from 2000: (1) electronically searching under the names of authors of 2 or more research articles in 1999 and 2000 and (2) searching under the names of North American Primary Care Research Group members who had not yet been searched for 2003. These 2 steps together yielded 102 articles and, therefore, could account for at most one-third of the growth measured between 2000 and 2003. We further note that although the first step of hand searching issues of journals involved 80 journals for 2000 and only 22 journals for 2003, and the second step of searching for articles listed with family medicine affiliations was the same in both years, these 2 steps together yielded 11% more articles in 2003 (n = 553) than found through all search steps in 2000 (n = 499). Again, growth is clearly not entirely a methods artifact. Comparisons of the articles we found at each time period against compendia from departments suggest that a similar 20% was missed at both reviews. Despite somewhat differing search approaches, we believe that the measured growth amount is reasonably accurate. If estimates were off by the maximum possible one-third, it would not meaningfully change the overall conclusion of substantial growth over the 3 years.
Some authors listed as their affiliations on their articles their universities, schools, hospitals, and practices and not their departments or other units that carry the “family medicine” or “family practice” label. This might occur more often for authors in settings such as community hospitals and practices, leading us to miss more of their articles in the affiliation field search step and then underestimate the number of articles from their settings.
Conclusions
We conclude that research in family medicine in the United States is a larger and more productive enterprise than is generally appreciated and it is growing. Should we believe or feel that there is now enough research happening in family medicine? Against crude relative bench marks, the volume of research in family medicine is likely still less than that of other disciplines. For a discipline that overlaps with the content of all other clinical disciplines and addresses the physical, psychological, and community aspects of people's lives, how could the discipline ever feel there is enough research? It is likely that an “appropriate” volume of research will be reached not only at some threshold number of published papers, but when research in the field attains real visibility and an appearance of adequacy to those within and outside the discipline, and when it engenders a sense that progress is being made in answering the discipline's most important questions. Calls for more family medicine research suggest that it is not yet visible enough and that too many of the discipline's questions remain unanswered. The dispersion of the discipline's research across hundreds of non-family medicine journals will continue to make it hard to recognize the field's full cumulative contributions.
With a substantial and growing number of research articles, the discipline should move beyond pessimistic discussions about the survivability of family medicine research to a more informed, data-based discourse on how best to accelerate its growth, refine its direction, bolster its visibility, and maximize its impact on patients and the discipline.
Acknowledgments
The authors thank the Academic Family Medicine Organizations' leadership for encouraging the collection of these data in the belief that it is important for the discipline to document its progress in research development. We also thank the North American Primary Care Research Group's prior Committee on Building Research Capacity/Academic Family Medicine Organization Research Subcommittee for initiating this effort. Further, we are grateful to Jennifer Groves and Larry Logan, who assembled this report's database of articles for 2003, and thank the host of research assistants who conducted this study's library work and many hundreds of literature searches. Drs. Robert Gwyther and Larry Green provided helpful comments on an early draft of this paper.
Notes
This article was externally peer reviewed.
Funding: none.
Prior Presentation: Portions of this study and its findings were presented at the Annual Meeting of the North American Primary Care Research Group, Quebec City, Quebec, Canada, October 18, 2005; and at the Associations of Departments of Family Medicine Winter Meeting, Albuquerque, New Mexico, February 26, 2006.
Conflict of interest: none declared.
- Received for publication June 7, 2007.
- Revision received August 9, 2007.
- Accepted for publication August 13, 2007.
References
In this issue
Jump to section
Related Articles
Cited By...
- In which journals do family and community physicians in Brazil publish? The Trajetorias MFC project
- BUILDING RESEARCH & SCHOLARSHIP CAPACITY IN DEPARTMENTS OF FAMILY MEDICINE: A NEW JOINT ADFM-NAPCRG INITIATIVE
- CALLING ALL SCHOLARS TO THE COUNCIL OF ACADEMIC FAMILY MEDICINE EDUCATIONAL RESEARCH ALLIANCE (CERA)
- Optimism: A Good Theme for Family Medicine
- Thirty Years of Family Medicine Publications in Israel (1975-2004): What, Where, and How Much?
- North American Primary Care Research Group President's Award
- Research Activity in Family Medicine: The "Best of Times" or "Can I Have More, Please"?