Summary of Findings from Data Extraction
Title | Author | Publication Year | Location | Key Takeaway |
---|---|---|---|---|
A metric for academic performance applied to Australian universities 2001 to 2004 | Sandstrom and Sandstrom13 | 2007 | Australia | Use of a performance-related model that combines productivity with quality measures using a single database. Measured bibliometric data such as number of publications. |
A simple, generalizable method for measuring individual research productivity and its use in the long-term analysis of departmental performance, including between-country comparisons | Wootton6 | 2013 | Norway | Development of an indicator of individual research output based on grant income, publications, and numbers of PhD students supervised. |
Assessing research activity and capacity of community-based organizations: refinement of the CREAT instrument using the Delphi method | Humphries et al14 | 2019 | US | Development of the Community REsearch Activity Assessment Tool (CREAT) instrument using a structured Delphi panel. Most metrics are subjective. Objective, numeric measurements include staff and budget. |
Assessing research capacity in Victoria’s south-west health service providers | Gill et al15 | 2019 | Australia | Implementation of the Research Capacity and Culture (RCC) tool which had previously been developed by Holden et al, 2012.16 |
Assessment of health research capacity in western Sydney local health district (WSLHD): A study on medical, nursing and allied health professionals | Lee et al17 | 2020 | Australia | Implementation of the RCC tool, demonstrating differences between various professionals.16 |
Biomedical research productivity: factors across the countries | Rahman and Fukui18 | 2003 | Japan | Analyzed country of origin for published articles to determine significant factors relating to research output defined as publications per million population per year. Significant factors included gross national product per capita, research and development expenditure, number of science and engineering students, and number of physicians. |
Building research collaboration networks: an interpersonal perspective for research capacity building | Huang19 | 2014 | Singapore | Highlights the value of research collaboration networks as evidence of research capacity. |
Common metrics to assess the efficiency of clinical research | Rubio20 | 2013 | US | Identification of metrics to assess the efficiency of clinical research processes and outcomes. They identified 15 metrics in 6 categories. Objective, numeric metrics include time for IRB submission to approval, time to publication, and number of technology transfer products. Categories included processes, careers, services, economic return, collaboration, and products. |
Developing indicators for measuring Research Capacity Development in primary care organizations: a consensus approach using a nominal group technique | Sarre and Cooke21 | 2009 | England | Development of a list of indicators to measure research capacity development at an organizational level using workshops and modified nominal group technique. Individual metrics include research personnel, funding, membership in research alliances, number of projects, and awards. They were grouped by category according to the model developed by Cooke.22 |
Development and use of a research productivity assessment tool for clinicians in low-resource settings in the Pacific Islands: a Delphi study | Ekeroma et al4 | 2016 | Fiji, Samoa, Tonga, Vanuatu, Cook Islands, Solomon Islands | Focus group discussions to obtain viewpoints on meaningful research indicators. They developed a tool of 21 subjective and objective indicators. Example metrics include bibliometrics, funding, recognition, collaboration, and personnel. |
Evaluating health research capacity building: an evidence-based tool | Bates et al23 | 2006 | Ghana | Development of a tool to measure clinical research capacity–building programs. The framework was based on reported literature then adapted to the local context through an internal working group. Their resulting tool consisted of a mix of 12 objective and subjective measurements. Sample numeric metrics include bibliometrics, research funding, and researcher remuneration. |
Evaluation of the research capacity and culture of allied health professionals in a large regional public health service | Matus et al24 | 2019 | Australia | Evaluation of research among allied health professionals working in a large regional health service using the Research Capacity and Culture (RCC) tool.16 Principal component analyses to determine key components that influence differences between various professional groups. |
How has healthcare research performance been assessed?: a systematic review | Patel et al25 | 2011 | Articles from several countries were included | Systematic review of indicators of health care research, along with evidence supporting their use. Indicators include publications, citations, impact factor, funding, authorship, population size, h-index, peer reviews, presentations, patents, doctoral students, and editorial responsibilities. |
Indicators for tracking programs to strengthen health research capacity in lower- and middle-income countries: a qualitative synthesis | Cole et al26 | 2014 | Canada, UK, Switzerland | Qualitative evaluation of research evaluations to identify key indicators of research productivity. Quantitative indicators include awards, trainees with a mentor, workshop attendance, courses run by educational institutions, course attendance, collaboration activity attendance, joint projects, and bibliometrics. |
Measuring research capacity development in healthcare workers: a systematic review | Bilardi et al27 | 2021 | UK, Australia, Italy | Systematic review and narrative synthesis of articles containing tools to measure health care workers’ individual research capacities. Many articles contained data on team and organizational level. Many domains of assessment were identified, including skills, motivations, bibliometrics, informatics, communication, collaboration activities, studies, ethics, quality, support, skills, infrastructure, leadership, efficiency, dissemination, culture, and sustainability. |
Measuring, analysis and visualization of research capacity of university at the level of departments and staff members | Kotsemir and Shashnov28 | 2017 | Russia | Literature review on methods of research capacity in the university. Their analysis focuses primarily on bibliometrics, including number of publications, h-index, impact factor of published studies, and articles with evidence of collaboration. |
Nine criteria for a measure of scientific output | Kreiman and Maunsell29 | 2011 | US | Identification of qualities that define an effective research metric. They advocate that metrics should be quantitative, based on robust data, rapidly updated and retrospective, presented with CIs, normalized by number of contributors, career stage and discipline, impractical to manipulate, and focused on quality over quantity. |
Rehabilitation Medicine Summit: building research capacity | Frontera et al5 | 2006 | US | Outcomes of a summit convened to advance and promote research in medical rehabilitation. They identified several important domains of research capacity, including research environment, infrastructure, and culture. Objective indicators they identified include bibliometrics and funding. |
Research capacity building frameworks for allied health professionals - a systematic review | Matus et al30 | 2018 | Australia | Systematic review of 5 databases to identify models and frameworks for research capacity building. They identified 3 main themes: supporting clinicians in research, working together, and valuing research for excellence. |
Validation of the research capacity and culture (RCC) tool: measuring RCC at individual, team and organization levels | Holden et al16 | 2012 | Australia | Development of the Research Capacity and Culture (RCC) tool based on literature review and expert guidance. Validation performed for internal consistency and test-retest reliability. Indicators include funding, bibliometrics, age of researchers, evidence of partnerships and dissemination. |
Abbreviation: IRB, institutional review board.