Article Figures & Data
Tables
Search strategy for Medline (network*(ti) AND (health care(ti) OR practice*(ti) OR research(ti) OR primary care(ti) OR primary health(ti)) OR pbrn(tw) OR (“Community Networks” (MeSH) OR “Group Structure”(MeSH)) AND ("Primary Health Care"(MeSH) OR “Family Practice”(MeSH) OR “Physicians, Family”(MeSH) OR gp(tw) OR ([general OR family] AND [physician* OR practi* OR doctor*]) OR “primary care” OR “primary health care”) English key words* “network” combined with “primary care” or “research” or “practice-based” or “general practice” or “pcrn” Dutch key words† “academisering” or “netwerk” or “werkplaats” combined with “academisch” or “huisarts” * Used for Embase and Google.
† Used for Picarta and Google.
Content validity 1. Do authors report that they reviewed relevant literature to identify previous research that have been done in the area and/or previous tools? 2. Do authors report the involvement of experts/peer groups in the development of the tool and/or item-selection/item-reduction process? 3. Do authors report the outcomes of the level of agreement among experts reflecting the content relevance of the proposed measurement areas/items? Reliability 4. Do the authors describe the tool and measurement method in sufficient detail to permit replication? 5. Do authors report the results of intra and/or inter-reliability tests? Feasibility 6. Do authors report that the acceptability and measurability of the measurement items and feasibility of the tool were tested in a pilot study 7. If the authors did not perform a pilot study, has the tool been applied once otherwise? 8. Do authors report other (empirical) evidence that the measurement quality of the tool has been examined? Practice investment 9. Do authors provide sufficient information to evaluate the time investment? 10. Do authors provide information about the costs of an assessment/evaluation using this tool? All questions were answered as yes or no.
- Table 3.
Description of the Identified Tools for the Evaluation of Primary Care Research Networks
Name/Author Concept of Interest Proposed Domains of Interest for PCRNs Evaluation Indicators (n) Objective-based framework/ Clement et al. (2000) PCRN effectiveness in terms of realizing predefined and agreed objectives Predefined and agreed objectives for PCRNs: To develop a network infrastructure
To develop research capacity of primary care participants (PCPs)
To increase the number/quality of research projects led by PCPs
To increase the use of research findings by PCPs
To increase the number/quality of research projects in which PCPs collaborate
To increase the number/quality of research projects in which PCPs participate
To provide a network that PCPs find acceptable
45 Evaluation tool kit/Harvey et al. (2000) PCRN suitable for purpose and potential productivity in terms of producing knowledge and creating ideas and intellectual capital Organizational and management dimensions which create social and intellectual capital for PCRNs: Strategic emphasis
Winner focus capacity-building
Practitioner capacity-building
Policy
Composition of executive
Direction of decision making
Network structure
Strength of center
Hierarchy (direction of power and resources)
Research infrastructure
Governance of research
Mechanisms for identifying research needs
Mechanisms for encouraging research ideas
Route of research ideas into projects
Mechanisms for supporting research
Mechanisms for evaluating research
Network processes
Leadership style
Cultural cohesion
Trust relationships
Process facilitation
Ease of joining the network
Mechanisms for enabling inter-relationships
Multidisciplinary
Mechanisms for publicizing the network
Mechanisms for allowing feedback
Ease of contacting support
Network boundaries
Relationships with partner organizations
Relationships with other networks
Self-evaluation by network
Data collection
44 PCRN, primary care research network; PCP, primary care physician.
- Table 4.
Description of the Identified Tools for Practices Involved in Primary Care Research Networks or Research
Name/Author Concept of Interest Proposed Domains of Interest for Evaluation Indicators (n) Huisartsgeneeskundige Academiserings Lineaal Maastricht (HALMA)/Doorn et al. (1999) Output of academic general practices in university-linked general practice networks Practice performance/output (qualitative and quantitative) in: Research and registration/data collection
Regular research activities
Research development
Registration/data collection (for research database)
Quality of health care
Academic medical teaching/education
Education/training in practice
Faculty teaching/medical education
70 Primary Care Research Team Assessment/ Carter et al. (2002) Research infrastructure of (general) practices involved in research (2 levels of accreditation) Does research infrastructure in practice meet quality standards in: Practice organization
Practice profile
Records and register
Strategic planning
Practice research infrastructure
Practice as a learning organization
Individual development
Team development
Teaching others
Research resources and infrastructure
Research resources
Computerization and data-handling
Links with other organizations
Project funding and management
Project funding
Project management
Involvement of patients
Consumer participation
Ethical issues
Dissemination of research
Dissemination strategy
78 Framework/Tool↓ Checklist items Content validity Reliability Feasibility Practice investment Literature Reviewed Expert Involvement Outcomes of Content Validity Study Such as Level of Agreement Described in Sufficient Detail to Permit Replication? Inter-reliability/Intrareliability Pilot Study If Not, Was the Tool Otherwise Applied Other (Empirical) Evidence of Measurement Quality? Time Investment Costs Huisartsgeneeskundige Academiserings Lineaal Maastricht (HALMA),† Doorn et al. (1999) + + + − − + − − − Objective-based framework,* Clement et al. (2000) + − − − − − − − − − Tool kit,* Harvey et al. (2000) + + − + − − +‡ − − − PCRTA,† Carter et al. (2002) + + − + −§ + + + + * Primary care research network level.
† Practice level.
‡ The Tool kit was used once in a comparative case study of 5 primary care research networks in the United Kingdom.
§ Carter et al reported that reliability and validity were qualitatively evaluated after the Primary Care Research Team Assessment (PCRTA) pilot study by an independent researcher, but reported no outcomes of inter-reliability or intrareliability.
