Family Physicians’ Opinions and Attitudes to Three Clinical Practice Guidelines
=================================================================================

* Robert M. Wolfe
* Lisa K. Sharp
* Ralph M. Wang

## Abstract

*Background:* Clinical practice guidelines (CPGs) have been promoted as a strategy to measure and improve the quality of patient care. However, more effort has been expended on creating guidelines than implementing them. We surveyed family physicians about their knowledge of and attitudes toward 3 well-publicized CPGs.

*Methods:* A survey questionnaire was sent to a national sample of 600 family physicians selected at random from a file from the American Academy of Family Physicians (AAFP).

*Results:* After 2 mailings, the response rate was 71%. For each of the 3 guidelines, roughly 60% of respondents were familiar or somewhat familiar with the CPG. More than half of family physicians studied said they have changed their medical practices based on CPGs, and only 3% said they do not believe in guidelines and would not use them. Use of CPGs was lowest among physicians in solo practice and among those working in rural areas. However, no significant differences in CPG use or familiarity were noted based on number of years in practice. Many respondents indicated an interest in keeping CPGs current via the internet.

*Conclusions:* Most American family physicians find CPGs to be helpful, and familiarity with them is fairly uniform across most subgroups studied.

The Institute of Medicine defines clinical guidelines as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”1 Because of the increasing complexity of medical care and the desire to improve quality and reduce costs, as many as 1500 guidelines have been issued in the United States alone.2 Part of the impetus comes from government agencies and various insurance companies, including health maintenance organizations (HMOs) desiring to ensure certain standards for patient care. Specialty organizations have also promoted CPGs both for the above reasons and also as a way of stating policy decisions that support the beliefs, values, or needs of that specialty.3

A number of studies have looked at practitioners’ compliance with published guidelines and found significant deviation from CPG recommendations.4–6 Reasons for low guideline compliance included guideline implementation problems and lack of belief in guidelines by physicians.5 The latter attitude was also noted in a study of Australian general practitioners in which 85% agreed that guidelines were “developed by experts who don’t understand general practice.”7 Several studies of American family physicians’ attitudes toward CPGs found that only about one third or less of family physicians changed their practices as a result of using CPGs.8,9

The purpose of this study was to explore American family physicians’ knowledge of, and attitudes toward several recent guidelines that address aspects of medical care frequently encountered in family medicine and to compare the results with those of other studies of this type. The 3 guidelines chosen for study were the “Managing Otitis Media with Effusion in Young Children” guideline issued by the Agency for Health Care Policy and Research (AHCPR) in 1994 (the AHCPR has since been renamed the Agency for Health care Research and Quality, or AHRQ), the “Summary of Policy Recommendations for Periodic Health Examination” issued by the American Academy of Family Physicians (released November 1996, revised July 1997), and the “Antithrombotic Therapy in Atrial Fibrillation” guideline issued by the American College of Chest Physicians. We chose these guidelines for several reasons. First, each of these guidelines represents a common problem seen in the average family physician’s office. Second, each of the guidelines used evidence-based medicine (EBM) for its development. Finally, each guideline was developed differently: the Periodic Health Examination guideline was developed by the American Academy of Family Physicians (AAFP); the Atrial Fibrillatio” guideline was developed by a specialty board not affiliated with the AAFP, and the Otitis Media with Effusion guideline was developed by a consortium of the AAFP, the American Academy of Pediatrics (AAP), and the American Academy of Otolaryngology-Head and Neck Surgery under contract with the Agency for Health Care Policy and Research (AHCPR). This presented an opportunity to examine the degree of involvement of the AAFP in developing a guideline with the acceptance of that guideline by family physicians.

Finally, we modeled this study on a prior study of pediatricians’ attitudes toward CPGs,2 enabling us to compare the attitudes of pediatricians and family physicians. We sought to answer the following questions: 

1.  How aware are family physicians of recent guidelines that impact on the daily practice of family medicine?

2.  How familiar are family physicians with these guidelines?

3.  Do family physicians use or refer to these guidelines on a regular basis?

4.  Do family physicians have access to the information presented in the guidelines if they wish to refer to them?

5.  Do family physicians find these guidelines helpful? Do they find guidelines helpful in general?

6.  How could the process of disseminating guideline information be improved? What would the preferences for the format for such information be?

7.  What factors, if any, correlate with different attitudes toward CPGs?

## Methods

The survey questionnaire was based on a previously validated instrument used to study pediatricians’ attitudes to CPGs.2 Changes were made to make the instrument appropriate for family medicine. (The author of that study, Dmitri Christakis, provided permission to adapt the questionnaire for this study.) The revised survey was piloted among 5 family physicians in the Department of Family Medicine of Northwestern University’s Feinberg School of Medicine, and their feedback as incorporated into the final version of the survey.

We recruited our sample from 600 members randomly selected from the active membership of the AAFP. Participants were sent a 4-page questionnaire along with a prepaid return envelope. As an incentive to participate, a $1 gold coin was attached to each survey in the first mailing. Physicians were assured that their participation was voluntary and responses would be confidential. Those who had not responded within 4 weeks were sent the survey again.

The study was approved by the Northwestern University institutional review board.

### Survey Instrument

Participants were asked if they were aware of the 3 guidelines under study (answer options were “yes,” “somewhat,” or “no”). We included “somewhat” in our study to refine our understanding of the range of awareness of these guidelines. Those who knew of a particular guideline were asked whether a copy was available in their office for review. They were also asked how helpful they found the guideline and were asked about what they perceived as its limitations. Helpfulness was reported on a scale from 1 to 5, with 1 being “not at all helpful” and 5 being “extremely helpful.” Several options for limitations were given for each guideline, including “too cumbersome,” “too time-consuming to apply,” “too confusing,” “too cookbook,” “too difficult to access or find,” “not applicable to my patients,” and “don’t believe in guidelines.” The option “other” was provided to allow participants to express their own feelings about the guidelines. Finally, the participants were asked whether the guidelines had changed their management of that particular condition.

At the end of the questionnaire, participants were asked demographic data including practice type (family practice group, solo practice, staff-model HMO, university, multispecialty group, or other practice); practice location (rural, suburban, or urban); and year of graduation. We also asked if participants were board-certified in family practice. The final question was, “If a collection of Clinical Practice Guidelines relevant to family medicine could be made available to members of the American Academy of Family Physicians, which of the following would express your preferences for such a collection?” Choices were “should be summarized in an outline form,” “should be summarized in an algorithm/flowsheet form,” “should include evidence from the literature,” “should include flowsheets or reminder forms for use in patient charts,” “family physicians in active practice should help decide what guidelines to include,” and “I do not believe in guidelines or find them helpful, and I would not use them.” An option “other” was given to allow participants to express their own ideas. Space was left at the end of the questionnaire to allow participants to express any other comments they had.

### Statistical Analysis

χ2 Analysis was used for comparing categorical variables. Student’s *t* test was used to compare means on continuous variables. Multivariate logistic regression was used to model dichotomous dependent variables. SPSS 10.5 for the PC (SPSS, Chicago, IL) was used for all analyses.

To allow comparison with “helpfulness” data from Christakis and Rivara,2 who used a 1-to-10 scale, Christakis’ scale was adjusted using the formula HADJ = [4/9 × (H − 1)] + 1 where H is the helpfulness score reported in the pediatric guideline study and HADJ is H adjusted to the scale of the present study. The formula is derived by adjusting both scales down to zero baseline and adjusting for the proportional difference, then returning the baseline to 1.

## Results

### Sample

Of 600 surveys sent to physicians in the initial mailing, 6 were returned blank and 3 were returned by family practice residents, who were included in the data pool. Of the 594 eligible participants, 392 responded to the first mailing, and 36 responded to the second mailing for a total of 420 surveys returned (response rate, 71%). There were no significant differences in demographics between respondents to the first and second mailings. Demographic information on survey respondents is summarized in Table 1. The demographic and practice characteristics of respondents in our sample correspond closely to the active membership of the AAFP.10 On average, the respondents spent 86% of their time on clinical duties (range, 0 to 100%), with 83% of respondents spending 80% or more of their time on clinical duties. Thirty-eight percent of respondents reported that some of their practice sites made use of Clinical Practice Guidelines.

View this table:
[Table 1.](http://www.jabfm.org/content/17/2/150/T1)

Table 1. 
Characteristics of Survey Recipients

### Awareness of Guidelines

The level of familiarity was similar for all 3 guidelines (Table 2). Thirty-four percent of respondents were familiar with the AHCPR otitis media guideline, 35% were familiar with the AAFP periodic health examination guideline, and 31% were familiar with the American College of Chest Physicians guideline on antithrombotic therapy. When including respondents who answered either “yes” or “somewhat” regarding familiarity, the percentages increased to 62%, 61%, and 60%, respectively. Thirteen percent of respondents reported being very familiar with all 3 guidelines, and 14% were not familiar with any of them.

View this table:
[Table 2.](http://www.jabfm.org/content/17/2/150/T2)

Table 2. 
Guideline Familiarity

### Helpfulness of Guidelines

The mean helpfulness scores ranged from 3.05 to 3.63 ( Figure 1). The Antithrombotic Therapy guideline received the highest helpfulness score (3.63; S.D. 0.93), followed by the Periodic Health Examination guideline (3.31; S.D. 1.01). The Otitis Media guideline was found to be least helpful (3.05; S.D. 1.01). Helpfulness scores for each guideline did not vary significantly by practice type or location. There were no significant statistical differences in the helpfulness scores for any of the 3 guidelines when the physician scores were subcategorized and compared by year of graduation, practice type, or environment (rural/urban/suburban) (see Figure 2).

![Figure 1.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/17/2/150/F1.medium.gif)

[Figure 1.](http://www.jabfm.org/content/17/2/150/F1)

Figure 1. 
Helpfulness of CPGs. ♦, Otitis Media; ▪, Periodic Health Exam; ▴, Antithrombotic Therapy.

![Figure 2.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/17/2/150/F2.medium.gif)

[Figure 2.](http://www.jabfm.org/content/17/2/150/F2)

Figure 2. 
Graduation year versus helpfulness scores. ♦, Help 1; ▪, Help 2; ▴, Help 3.

### Change in Practice Behavior

Of all respondents who had any familiarity with a particular guideline, 44% reported changing their patient management as a result of the otitis guideline, 51% as a result of the health examination guideline, and 64% as a result of the antithrombotic therapy guideline. When the denominator is all survey respondents regardless of CPG familiarity, 28% of family physicians reported changing their patient management because of the otitis guideline, 31% as a result of the health examination guideline, and 34% as a result of the antithrombotic therapy guideline. Degree of familiarity affected the percentage of respondents who reported change in their practice from a guideline. Of those indicating full familiarity with the guidelines, 59% reported changing their patient management because of the otitis and preventive medicine guidelines, and 70% reported changing their management because of the antithrombotic therapy guideline. By comparison, for those indicating that they were only “somewhat” aware, the percentages reporting change dropped to 27%, 39%, and 48% for these 3 guidelines, respectively.

### Perceived Problems with Guidelines

Of the various perceived problems with the guidelines, the most common limitation, reported by 29% to 31% of respondents, was difficulty of access (Table 3). Perceived problems were fairly consistent across all 3 guidelines. The only significant preference difference we noted was that academic family physicians had a higher preference for CPGs based on EBM than nonacademic physicians (73% vs 46%). This preference also appeared when looking at year of graduation above and below the median year of 1986: 59% of those who graduated after 1986 preferred guidelines based on EBM, whereas only 39% of those who graduated before 1986 expressed this preference.

View this table:
[Table 3.](http://www.jabfm.org/content/17/2/150/T3)

Table 3. 
Limitations of Specific Clinical Practice Guidelines

Between 13% and 18% of respondents checked “other” and listed personal ideas. Some comments included physicians stating that they already practiced in accordance with the guideline (and presumably did not need it) and that patients were not amenable to following guidelines (particularly regarding the otitis guideline, where several respondents said that parents wanted antibiotics). Some wrote that their organization required them to follow a different guideline than the one referred to. For example, several physicians commented that their organization followed the recommendations of the United States Preventive Services Task Force (USPSTF) rather than the “Summary of Policy Recommendations for Periodic Health Examination” issued by the American Academy of Family Physicians. Several commented that there were too many guidelines and that it was hard to choose between them. An additional 12% of respondents wrote some general comments at the end of the questionnaire.

### Current Use of Guidelines and Preferences for a Guideline Collection

Thirty-eight percent of respondents reported currently using CPGs in their practices, with no significant difference based on date of graduation. When sorted by practice type, usage was highest in staff model HMOs (100%) and lowest in solo practice (23%). Sorted by practice location, use of these 3 CPGs was highest among suburban physicians (43%), followed by urban physicians (39%) and lowest among rural physicians (28%).

Preferences for the format of a collection of guidelines is shown in Table 4. A roughly equal percentage of respondents chose “outline,” “algorithm,” or “flowsheet/reminder form” as formats they would like guidelines to be presented in (range, 53% to 58%). However, there were significant differences by year of graduation. For those physicians who graduated before the median year of 1986, 63% chose “outline,” and 49% chose “algorithm,” whereas for those graduating from 1986 onward, 53% chose “outline,” and 63% chose “algorithm.” About half of respondents preferred that guidelines be evidence-based; this preference was highest among university-affiliated physicians (73%) and lowest among solo practitioners (30%). Only 3% expressed that they did not believe in using guidelines in medical practice.

View this table:
[Table 4.](http://www.jabfm.org/content/17/2/150/T4)

Table 4. 
Preferences for Format of CPGs

Several themes were raised in the “other comments” section where physicians could express themselves spontaneously. The most common suggestion was that guidelines should be accessible via the Internet for ease of access and to keep the guidelines up to date. Another suggestion was that guidelines should be integrated into an electronic medical record so that they could be accessed at the point of care. One physician suggested incorporating a “tickler” system into such integration so that the physician would be reminded of needed prevention services for specific patients. Several physicians suggested a guideline collection on a handheld personal digital assistant (PDA) device. Another concern was the difficultly of choosing between conflicting guidelines issued by different groups. Finally, one physician expressed concern that the use of guidelines might entail some legal risk and that “… [guidelines] may be used against us legally.”

### Comparison with Study of Pediatricians’ Attitudes toward CPGs

The otitis media guideline used in this study was also used in Christakis and Rivara’s study of pediatricians’ attitudes toward CPGs.2 The pediatricians gave the otitis media guideline a helpfulness score of 5, which adjusts to 2.8 on this scale of the present study, less than the family physicians’ score of 3.3 for the same guideline. Only 28% of pediatricians familiar with this guidelines reported changing their behavior because of this guideline, compared with 44% of family physicians. When all survey respondents are included, 14% of pediatricians reported change in management because of this guideline, compared with 28% of family physicians.

## Discussion

We undertook this study to evaluate to what degree family physicians were using CPGs, and to see what aspects of CPGs they found useful to help plan the development of future guidelines. We also polled our sample of physicians to assess what, if any, modifications they would like to see made in the CPGs they use. We selected 3 CPGs: one a guideline jointly endorsed by pediatricians and family physicians, a second guideline created and endorsed by family physicians, and finally one created by internists but widely used by family physicians.

What our data showed is that roughly one third of family physicians were familiar with each of the guidelines, one third were somewhat familiar with them, and one third were unfamiliar with them. The helpfulness scores varied among the 3 guidelines. For example, of the 3 guidelines, family physicians were least familiar with the anticoagulation guideline, but those who were familiar with it gave it the highest helpfulness score of any of the CPGs we studied. This is understandable, because this guideline sets the standards for managing patients on warfarin, a common occurrence for physicians caring for elderly patients. This CPG also had the least complaints of being “too time consuming” to apply—only 4% vs about 15% each for the other 2 guidelines. The highest awareness score was expressed for the otitis media guideline. Again, this is understandable, because the AHCPR widely distributed this guideline to family physicians and pediatricians in the United States. However, the algorithm was complex, reflected in the section of the questionnaire concerning CPG problems, where family physicians gave this the highest rating of “too complicated” of the 3 CPGs.

The awareness scores and helpfulness scores were surprisingly consistent across all the subcategories we used to segregate family physicians, including location (rural, urban, suburban), practice type (solo, FP group, etc), and year of graduation. This suggests that most family physicians are exposed to similar information, either through the *American Family Physician* journal and other journals or via accessing similar internet sites for information updates, essentially putting them “on the same page.” In this regard, our findings differed from the study of pediatricians done by Christakis et al2 and the study of family physicians done by Wolff et al,9 which showed significant differences among respondents regarding guideline awareness and usefulness, based on year of graduation. This might be a result of the passage of 5 or 6 years between those studies and ours, during which time the acceptance of CPGs as a clinical tool may have become more widely accepted, along with the retirement of a small group of the oldest physicians who were likely to be most resistant to changes in practice style.

Looking at the one guideline that our study had in common with the pediatric study, the otitis media guideline, family physicians showed more familiarity with it and gave it a higher helpfulness score than did pediatricians.2 This may reflect the extra 5 years between the studies during which this guideline may have become more widely used, as well as several articles and monographs referring to this guideline that have appeared in the *American Family Physician* journal and AAFP monographs since 1998, all of which were distributed to active AAFP members.11–13

Although other studies of American family physicians asked general questions about guidelines, rather than queries about specific CPGs, the actual usage of guidelines and effect on practice reported by respondents in our study are similar to findings from these other studies. Wolff et al3 found that 34% of family physicians stated that CPGs had influenced their clinical practice, and James et al8 reported that 28% of family physicians changed their practice of medicine by adopting at least one CPG in the previous year. This compares closely with our findings concerning the percentage of respondents who had changed their practice of medicine as a result of a CPG, which ranged from 28% to 33% for the 3 guidelines we studied.

Preferences for the style or format of CPGs showed only small differences among family physician subsets. The only significant difference, the higher interest in having EBM be considered in guideline development among younger physicians, probably reflects the more recent emphasis on EBM in medical school curricula and the higher level of familiarity more recent graduates have with EBM methods.

The freeform comments reflected some of the current concerns and interests of family physicians. The most common suggestion, that guidelines should be accessible via the Internet for ease of access and to keep them up to date, actually reflects directions being taken in modern medical technology and computerization. For example, after publication of an article in 2001 advising that CPGs require updating at least every 3 years,14 the AHRQ archived the otitis media with effusion guideline and noted at its National Guideline Clearinghouse web site that the guideline is no longer viewed as guidance for current medical practice.15 Several physicians mentioned that they used the National Guideline Clearinghouse web site ([http://www.guidelines.gov](http://www.guidelines.gov)) to keep up-to-date. The suggestion that current recommendations be integrated into a computerized system has been noted in other studies.9

We believe our survey is representative of American family physicians, and our response rate of 71% was extremely high. However, whereas other surveys have questioned family physicians about the general use of CPGs, we questioned our sample about only 3 specific guidelines, which could limit generalizations from our results. At least one respondent commented that he used clinical guidelines but did not happen to use any of the 3 we selected for our study.

What recommendations can we make based on these results? First, over half of family physicians studied said they have changed their medical practices based on CPGs, and only 3% said they do not believe in guidelines and would not use them, which should give some encouragement to those promoting the use of guidelines. Second, roughly 50% to 60% of physicians studied expressed a desire to have the guidelines delivered in a variety of forms, including flowsheets, algorithms, and outlines, supporting the idea that CPGs issued should be formatted in a variety of styles to reflect the varied ways in which different physicians prefer to learn. EBM seems to be increasingly important to a younger generation of family physicians and efforts should be made to incorporate this in future guidelines. Finally, it should be noted that accessibility was the main drawback expressed about guidelines. A recent survey of members of the AAFP found that 94% of family physicians have internet access, 85% of whom use the internet to look up clinical information.10 Whereas a survey of American family physicians in 1997 ranked the Internet as one of the least effective methods of promoting CPG adoption, in the present survey, the use of the internet to access CPGs was the most frequent suggestion made in the “Additional comments” section of the questionnaire. The Internet is probably the ideal medium to disseminate guidelines and will probably become increasingly important, both to reduce the paper overload in physicians’ offices16 and to make it easier to keep CPGs current.1,14

## Acknowledgments

We thank research assistant Cynthia O’Toole.

## Notes

*   This research was supported by Research Stimulation grant G0003RS from the American Academy of Family Physicians Foundation.

*   Received for publication September 10, 2003.
*   Revision received September 10, 2003.

## References

1.  Field MJ, Lohr KN, editors. Clinical practice guidelines: directions for a new program. Washington DC: National Academy Press; 1990.
    
    

2.  Christakis DA, Rivara FP. Pediatricians’ awareness of and attitudes about four clinical practice guidelines. Pediatrics 1998; 101: 825–30.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTAxLzUvODI1IjtzOjQ6ImF0b20iO3M6MjA6Ii9qYWJmcC8xNy8yLzE1MC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

3.  Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999; 318: 527–30.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTgvNzE4Mi81MjciO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzE3LzIvMTUwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  Kimura S, Pacala JT. Pressure ulcers in adults: family physicians’ knowledge, attitudes, practice preferences, and awareness of AHCPR guidelines. J Fam Pract 1997; 44: 361–8.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9108833&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1997WU01200009&link_type=ISI) 

5.  Lawler FH, Viviani N. Patient and physician perspectives regarding treatment of diabetes: compliance with practice guidelines. J Fam Pract 1997; 44: 369–73.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9108834&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1997WU01200010&link_type=ISI) 

6.  Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? N Engl J Med 1989; 321: 1306–11.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1056/NEJM198911093211906&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=2677732&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1989AY17100006&link_type=ISI) 

7.  Gupta L, Ward JE, Hayward RSA. Clinical practice guidelines in general practice: a national survey of recall, attitudes and impact. Med J Aust 1997; 166: 69–72.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9033560&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=A1997WE11200006&link_type=ISI) 

8.  James PA, Cowan TM, Graham RP, Jameroni BA. Family physicians’ attitudes about and use of clinical practice guidelines. Fam Pract 1997; 45: 341–7.
    
    

9.  Wolff M, Bower DJ, Marbella AM, Casanova JE. US family physicians’ experiences with practice guidelines. Fam Med 1998; 30: 117–21.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9494802&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 

10. Practice Management Profile: a survey of members of the American Academy of Family Physicians. Leawood (KS): American Academy of Family Physicians; 2003.
    
    

11. Pichichero ME. Acute otitis media: Part I. Improving diagnostic accuracy. Am Fam Physician 2000; 61: 2051–6.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=10779248&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000086474100010&link_type=ISI) 

12. Dowell SF, Schwartz B, Phillips WR. Appropriate use of antibiotics for URIs in children: part 1. Otitis media and acute sinusitis. Am Fam Physician 1998; 58: 1113–23.
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=9787277&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000076378600010&link_type=ISI) 

13. Semchenko A, Baroody F, Culpepper L. Management of acute sinusitis and acute otitis media. American Family Physician Monograph (No 1, 2001).
    
    

14. Shekelle PG, Ortiz E, Rhodes S, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 2001; 286: 1461–7.
    
    [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1001/jama.286.12.1461&link_type=DOI) 
    
    [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=11572738&link_type=MED&atom=%2Fjabfp%2F17%2F2%2F150.atom) 
    
    [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000171188600023&link_type=ISI) 

15. Agency for Healthcare Research and Quality. Clinical practice guidelines archive. Available at: URL: [http://www.ahcpr.gov/clinic/cpgarchv.htm](http://www.ahcpr.gov/clinic/cpgarchv.htm).
    
    

16. Hibble A, Kanka D, Pencheon D, Pooles F. Guidelines in general practice: the new tower of Babel? BMJ 1998; 317: 862–3.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTcvNzE2Mi84NjIiO3M6NDoiYXRvbSI7czoyMDoiL2phYmZwLzE3LzIvMTUwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)