Appendix B:

Aspects of Ambulatory Antimicrobial Stewardship Interventions as Applied to Components of the Ambulatory Work System (SEIPS 2.0).¹³^*

Reference	SEIPS 2.0 Element
Reference	Study Type	Tools / Technology Components	Person(s) Components	Organizational Components	Task Components	Physical Environment Components	External Environment Components
Litvin et al²⁰	Qualitative analysis of quasi-experimental study	• CDSS through EHR note template • Hyperlink to patient education • EHR audit and feedback • Limited printing capabilities • Not tailored to multiple complaints • Facilitates discussion • Tool adaptability • Network issues	• Patient symptoms • Patient age • Commitment to use CDSS • Agreement with CDSS • Technically savvy staff • Comfort with prior ARI templates	• CDSS alters practice workflow • Role of person initiating CDSS is different in different clinics (ie, triage nurse, medical assistant, and clinician)	• Improves efficiency	• Computer location	• External clinician guidelines
Persell et al²¹	2 × 2 × 2 RCT of patients with ARIs comparing (1) accountable justification, (2) suggested alternatives, and (3) peer comparison with educational module	• EHR used for accountable justifications and suggested alternatives (pop-up of alternatives and educational material)		• Monthly peer comparison emails • Other clinicians see accountable justification	• Ordering an antibiotic in EHR
Hux et al²²	RCT of PCPs in Ontario randomized to mailed intervention combining prescribing feedback with educational bulletins	• Mailed intervention	• Prescribing feedback
Hallsworth et al²³	Randomized 2 × 2 factorial trial of GPs who prescribed antibiotics in the upper 20th percentile randomized to (1) patient educational leaflet and feedback and (2) patient-focused intervention	• Patient education leaflet • Patient-focused information	• Feedback intervention given letter saying they prescribed >80% of local practices
Strykowski et al²⁴	Spanish RCT of GPs: acute exacerbation of COPD or chronic bronchitis, comparing (1) multifaceted intervention and CRP POC, (2) multifaceted intervention alone, and (3) usual care	• CRP POC testing		• Both groups had follow-up meetings with individual feedback • Both groups had training on ARIs		• Brochures and posters in waiting rooms
Foxman et al²⁵	RCT of nonelderly patients in 6 areas in United States assigned to insurance plans varying by level of cost sharing						• Insurance policies regarding cost sharing
Long et al²⁶	Shanghai RCT of patients with suspected CAP, randomized to procalcitonin or guidelines alone	• Procalcitonin testing					• Guidelines for CAP treatment
Arakaki et al²⁷	Nonblinded RCT of adults diagnosed with cellulitis by PCP, randomized to outpatient dermatologic consultation			• Dermatologic consultation
Taylor et al²⁸	RCT of educational intervention aimed at parents of children <2 years of age	• Pamphlet, videotape with local pediatrician promoting judicious antibiotic use
Torres et al²⁹	Observer-blinded RCT parallel-group study of clinical prediction rule of which children have bacterial pneumonia	• More chest X-rays in the control group		• Alternative: management based on institutional guidelines
Le Corvoisier et al³⁰	RCT of GPs: interactive seminar presenting evidence-based guidelines for ARIs		• Subgroups: 2-day evidence-based medicine course only or with 1-day problem-solving course				• Specific French guidelines
Meeker et al³¹	RCT in 5 adult primary care clinics to posted commitment letter or usual care in ARIs	• Letters/posters about avoiding inappropriate antibiotic prescribing included clinician photos and signatures				• Poster-sized commitment letters in exam rooms
Brittain-Long et al³²	Swedish randomized open label study to receive a rapid (next day) or delayed (8 to 12 days) result of PCR throat swab	• Multiplex PCR for respiratory viruses
Christakis et al³³	RCT of primary care pediatricians for POC evidence delivery on otitis media	• Computerized patient flow manager and online prescription manager	• NPs, physicians, and housestaff	• Computerized patient flow manager designed by clinics		• Computer workstations
Dahler-Eriksen et al³⁵	Danish randomized crossover trial of CRP measured within 3 minutes or with results in 1 to 2 days	• Intervention group: measure CRP within 3 minutes and got results 1 to 2 days later		• Needed hospital lab
De la Poza Abad et al³⁶	Pragmatic open label RCT among 23 Spanish primary care centers and ARIs to (1) delayed patient-led prescription, (2) delayed prescription collection strategy requiring patients to collect prescription from PCP, (3) immediate prescription, or (4) no antibiotics	• All patients given information about antibiotics • Delayed prescription: prescribe antibiotics to take only if symptoms worsen	• Patients told it was normal to feel worse at first, depend on patient to decide to take antibiotics		• Patients told it was normal to feel worse at first
Hemkens et al³⁷	Swiss pragmatic randomized trial of PCPs with highest antibiotic prescribing randomized to quarterly prescription feedback over 2 years or usual care	• Website required physicians to log in with individual access code	• Personalized feedback by mail and online				• 1 time provision of evidence-based guidelines for ARIs and UTIs
Samore et al³⁸	Cluster RCT in rural communities in Utah and Idaho comparing community intervention with CDSS on paper and CDSS with handheld computer	• Mailings to parents • Refrigerator magnets • Spiral bound flip chart for ARI self-management • 3 CDSS tools made for ARIs • Patient-initiated documentation tool • CDSS generated diagnostic and therapeutic recommendations		• Local clinicians led initiative at local hospital medical staff meetings • Continuing medical education • Clinicians compensated for each visit and could keep handheld computers		• Exam room posters and brochures	• Meetings with community leaders • News releases on self-management of ARIs • Self-care materials distributed at health fairs • Education at pharmacies and offices
Gerber et al³⁹ and Gerber et al⁴⁰	Cluster randomized trial of pediatric practices focusing on ARIs,³⁹ and follow-up for 18 months after intervention terminated⁴⁰	• ARI guidelines available as links in EHR	• Quarterly personalized feedback and audit	• 1 hour on-site clinician education • Practice-specific antibiotic prescribing rates			• Used prescribing guidelines
Gonzales et al⁴¹	3-arm cluster randomized trial, with 11 practices in each group: (1) printed CDSS, (2) computerized CDSS, and (3) control.	• Arm 1: printed CDSS for acute cough • Arm 2: EHR CDSS • Arm 3: no decision support • Arm 1 and 2: printed patient education material • Arm 1 and 2: template for ARI clinic visits	• Provider education and feedback in both intervention groups	• Half-day training session	• Provider education • Practice guidelines • Clinical champions • Audit and feedback
Jenkins et al⁴²	Cluster randomized study of 8 primary care clinics randomized to CDSS pathways and patient education or usual care	• 1 page CDSS pathway incorporated into EHR encounter template • Patient education		• Peer champion provider at each clinic
Finkelstein et al⁴³	Cluster randomized trial: 12 pediatric practices in 2 MCOs in eastern Massachusetts and Washington State	• Parents mailed CDC brochure on antibiotic use		• Physician peer leaders reviewed prescribing guidelines in practice meetings • Given feedback on practice prescribing		• Education displayed in waiting rooms	• CDC campaigns for judicious antibiotic use
Meeker⁴⁴	Cluster randomized study of 47 primary care practices randomized to receive 0, 1, 2, or 3 behavioral interventions: (1) suggested alternatives, (2) accountable justification, and (3) peer comparison	• Suggested alternatives: electronic order sets suggesting nonantibiotic treatments • Accountable justification: prompts to enter justifications for prescribing antibiotics for nonindicated diagnoses	• Peer comparison: emails comparing antibiotic prescribing rates with those of top performers	• Accountable justification: written justification would be seen in EHR as a note
		• All groups received online ARI diagnosis and treatment education
Yang et al⁴⁵	Matched-pair cluster randomized trial in China						• Disclosed prescribing status monthly to patients, health authorities, and health workers
Schnoor et al⁴⁶	Cluster randomized trial of 8 German clinical centers and 4 computer-based interventions	• All received printed versions of poster and electronic version of guideline	• Every other month, GPs given peer comparison			• All received poster
Finkelstein et al⁴⁷ and Ackerman et al⁷⁵	Controlled, community-level cluster randomized trials in 16 Massachusetts communities: physician behavior change strategy⁴⁷ with follow-up⁷⁵	• Discussion guide at well-child visits • Prescription pads for treatment of viral infections • Stickers, lapel pins, and otoscope insufflators • Bimonthly educational briefs • Patient brochure	• Introductory letter to pediatricians and family practitioners	• Educational coordinator visited practices • Kick-off dinners			• Advertisements at child care centers and pharmacies
Huang et al⁴⁸	Cluster randomized trial of 3-year community-wide intervention for parents of children in 16 communities¹¹	• Educational newsletters					• Educational materials in pediatrician offices, pharmacies, and child care settings
Vervloet et al⁴⁹	Matched randomized study of meeting with PCPs and pharmacists, 4 groups with intervention, and 4 matched controls	• PCP communication skills training • Antibiotic prescribing agreements in EHR		• Feedback session about group's prescribing • More variation between physicians than within practices
Linder et al⁵⁰	Cluster randomized trial of 27 primary care practices to receive an EHR-integrated ARI CDSS	• CDSS tool integrated into EHR
Linder et al⁵¹	Cluster RCT of 27 primary care practices, by using an ARI quality dashboard	• ARI quality dashboard	• Audit and feedback
Cals et al⁵² and Cals et al⁵³	Pragmatic cluster randomized trial of Dutch PCPs of the impact of POC CRP testing and communication skills on antibiotic use in LRTIs,⁵² with follow-up of 3.5 years⁵³	• POC CRP testing	• Communication training for physicians	• Peer review of colleague transcripts with simulated patients
Mainous et al⁵⁴	Quasi-experimental design with 3-month baseline data collection period and 15-month follow-up in 9 intervention and 61 control practices	• EHR CDSS tool is populated when ARI diagnosis entered • CDSS installed on each practice's EHR progress note template • Hyperlinks to patient and provider education	• Performance review • Audit and feedback	• Quarterly meetings with audit and feedback • Academic detailing • Monthly phone calls with staff			• NIH and CDC guidelines to improve antibiotic prescribing for ARIs • CDC Get Smart program
Slekovec et al⁵⁵	Quasi-experimental study of French GPs on quinolone prescription for UTIs	• Guidelines mailed to GPs and available on website		• 200 GPs attended training sessions			• Regional guidelines
Gonzales et al⁵⁶	Quasi-experimental community-level study of mass media campaign in Colorado with comparison communities		• Providers signed postcard soliciting support				• Mass media campaign including outdoor and radio advertisements, billboards, bus tails, bus stop posters, interior bus signs, health fairs, and opinion pieces
Hurlimann et al⁵⁷	Swiss quasi-experimental study of sustained feedback of antibiotic prescription for ARIs and UTIs		• Sustained individual feedback				• New guidelines written
Vinnard et al⁵⁸	Quasi-experimental study with concurrent control groups of academic detailing and educational mailings	• Educational mailings to providers and patients • Prescription pads for symptomatic treatments • Providers signed letters to patients	• Intensive intervention for highest prescribers	• Academic detailing of providers (visits from pharmacist and antimicrobial stewardship expert)
Gonzales et al⁵⁹	Nonrandomized controlled trial, with (1) full intervention, (2) limited intervention, or (3) usual care	• Full intervention: household and office-based patient education materials • Limited intervention: office-based educational materials • Full intervention: educational refrigerator magnets • Full intervention: patients mailed pamphlet and letter from medical director		• Full intervention: clinician education • Full intervention: clinic-specific antibiotic prescription rate for acute bronchitis • Full intervention: 30 minutes of staff meeting • Full intervention: setting clinic goals	• Full intervention: education on acute bronchitis management • Full intervention: clinicians taught how to refuse antibiotic prescriptions	• Both intervention groups: educational posters in exam rooms
Gonzales et. al⁶⁰	Nonrandomized controlled trial of elderly patients with ARIs: 4 intervention practices and 51 control practices in Denver	• Mailed campaign to patients • Office-based materials • Introductory letter from Department of Public Health • Patient reference card on ARIs • Patient refrigerator magnet • Patient brochures on antibiotic resistance	• Focus groups with elderly volunteers for feedback	• Individual prescribing profiles	• Colorado-wide initiative to provide PCPs with performance feedback	• Office-based materials: CDC posters, patient reference cards, and exam room posters
Formoso et al⁶¹	Community-level, controlled, nonrandomized trial in northern Italy of educational campaign						• Region-wide posters, brochures, and local media advertisements • Region-wide physician and pharmacist newsletter
Yardley et al⁶²	Process analysis of RCT of GPs in study of LRTI antibiotic prescribing, including web-based training in use of CRP	• Training software modifiable • Patient booklet with checklist • CRP intervention helpful	• Symptoms important to patients		• Completing patient booklet checklist
Szymczak et al⁶³	Semistructured interviews with pediatricians, following an intervention³⁹		• Parental pressure as a barrier to stewardship	• Skepticism of audit and feedback reports • One respondent admitted to gaming behavior
Mustafa et al⁶⁴	Semistructured interviews of family doctors on ARIs		• Avoiding confrontation • Negotiating with patients • Hedging • Physical traits • Exaggerated symptoms • Patient expectations	• Clinicians ask patients about antibiotic expectations • Pride • Rapport	• Use physical exam findings to minimize signs • Careful word choice • Thorough exam • Education • Setting expectations
Grondal et al⁶⁵	Semistructured interviews of GPs regarding pharyngitis	• GPs believed rapid Streptococcus test to be unreliable, as it only detects one bacterium • CRP used to diagnose bacterial pharyngitis as way to screen for all bacterial infections	• Clinical presentation outweighs test results • Streptococal pharyngitis should be diagnosed based on appearance		• All bacteria must be treated		• Clinical presentation outweighs guidelines for management of Streptococcal pharyngitis
Munoz-Plaza et al⁶⁶	In-depth interviews of 6 PCPs and 3 urgent care providers about acute sinusitis	• Need to find guideline when needed • Visual cards and graphs to show patients when they will improve • EHR with web-based patient education	• Help patients understand symptoms • Patients want reward for copays and taking off work	• Patients see physicians with different practices • No protected group education time • Large health systems enable system-level alerts • Limited time during patient visits		• Large educational posters in exam rooms	• External pressures for patient satisfaction • Patient can get antibiotics elsewhere • Information overload with updated guidelines • Copays for visits
Tonkin-Crine et al⁶⁷	Semistructured interviews of experts involved in GP ARI guideline development across 5 countries	• Complementary patient education • Computer reminders to reinforce guidelines	• Address GP concerns and explain the need for guidelines • GPs worry about patients becoming ill without prescription • Prescribing feedback helpful	• Academic detailing • GPs assume practice already follows guidelines • GPs may not engage with mandatory meetings	• Flexible interventions increase feasibility • Interventions should engage GPs		• Develop guidelines based on research, not to save money • Guidelines should be consistent • Guidelines difficult to locate and lengthy • Local versions of national guidelines • Governmental funding and national campaigns
Wood et al⁶⁸	Qualitative interviews with PCPs in 9 European countries	• Request access to local resistance data	• Most PCPs did not see antibiotic resistance as a problem in their practice	• Antibiotic resistance seen as issue for secondary care rather than primary care			• Other countries thought to be responsible for resistance • Self-prescribing of antibiotics • Prescribing by dentists and hospital physicians leads to resistance
Mauffrey et al⁷⁰	Qualitative study of French PCPs about preferences for interventions	• Dedicated prescription for antibiotics considered excessive • Limit antibiotic availability in outpatient settings • Computerized prescription aids • Educational resources for patients in office	• Physician training • Patient characteristics • Avoiding prescribing antibiotics is dangerous • Older prescribers: prescriptions influenced by habit	• Practice evaluation is difficult due to competing demands • Prior approval of ID physician would make practice harder	• Restricted susceptibility testing • Explain and repeat key messages • Role of PCPs in educating patients • Too many other health ministry-required tasks		• Performance-based incentives • Public campaigns reinforce what they tell patients • Physician awareness campaigns
Ronnerstrand et al⁷¹	Qualitative semistructured interviews: patients asked to imagine seeing a physician for ARI, and how long they would delay filling prescription	• Given prescription to fill later	• More trust means patients would delay prescription				• Willing to postpone treatment if told others would do the same • Social capital
Dempsey et al⁷²	Semistructured interviews of 13 PCPs	• Prefer of CDSS • Prefer over-the-counter prescription pad • Prefer patient educational materials	• Meet patient expectations	• No accountability for prescribing • Other clinicians' misconceptions about acute bronchitis • Previsit triage and education by nurses may prevent visits	• Prescribe antibiotics to save time and money • Diagnostic uncertainty		• Reporting of quality measures may help
Huddy et al⁶⁹	Qualitative focus group study of barriers and facilitators to use of CRP for diagnosing CAP among European GPs	• Want not just POC CRP but other lab testing in the same system • Need to test equipment and stay aware of shelf life of cartridges	• CRP as aid to communication	• Centralized labs • Different people perform test in different clinics (GP, nurse, and in-house lab)	• Quality control tasks • Responsibility of GP		• Although CRP is cost-effective, individual practices may be liable for the cost of the test • Loss of income to central labs
Rowbotham et al⁷³	Semistructured interviews and focus groups of UK NPs about experience with ARIs	• Used drawings or information leaflets to teach patients	• New NPs worried about making mistakes • Patients exaggerated symptoms	• Peer support helpful with demanding patients • Previous experiences with getting antibiotics from other providers		• Protocols support decision making • Visits are time consuming and complex • Provide self-management education • Educate patients to prevent return visits	• Legal protection of NPs versus GPs
Kuzujanakis et al⁷⁴	Parenteral surveys		• Parents demand antibiotics • Privately insured and with more or older children had more knowledge		• Alternatives to antibiotics		• Media information about antibiotics • School or day care requires antibiotics
Murphy et al⁷⁶	Qualitative interview of GPs reviewing own charts of ARI encounters		• Children less likely to receive antibiotics		• Amoxicillin as placebo if physician did not think they need antibiotics • Uncertainty in prescribing antibiotic		• External guidelines
Anthierens et al⁷⁷	Qualitative study using think-aloud approach with European GPs about using online tool for CRP ordering	• Online intervention • CRP used as evidence for patients on condition's seriousness • Tailoring for individual countries and languages	• Considering needs of patients • Level of attention and time taken needs to be brief	• Managing patient demands, fewer visits, decreased income •	• GPs use communication skills • Clarity of intervention • Physical exam used to communicate to patients		• Countries differ in how involved patients expect to be in the decision • Study in one country may not help in other countries

PCP, primary care provider; CDSS, clinical decision support system; RCT, randomized controlled trial; POC, point of care; ARI, acute respiratory infection; EHR, electronic health record; NIH, National Institutes of Health; CDC, Centers for Disease Control and Prevention; GP, general practitioner; LRTI, lower respiratory tract infection; CRP, C-reactive protein; ID, infectious diseases; MCO, managed care organization; CAP, community-acquired pneumonia; COPD, chronic obstructive pulmonary disease; NP, nurse practitioner; UTI, urinary tract infection; PCR, polymerase chain reaction; GAS, Group A Streptococcus; CAP, community-acquired pneumonia; UTI, urinary tract infection.
↵* Studies described include rigorously designed quantitative and qualitative studies describing antibiotic stewardship interventions and the context around antibiotic prescribing decisions, included through November 7, 2016.