Article Figures & Data
Figures
- Figure 1.
PRISMA flow diagram.
Tables
Title Author Publication Year Location Key Takeaway A metric for academic performance applied to Australian universities 2001 to 2004 Sandstrom and Sandstrom13 2007 Australia Use of a performance-related model that combines productivity with quality measures using a single database. Measured bibliometric data such as number of publications. A simple, generalizable method for measuring individual research productivity and its use in the long-term analysis of departmental performance, including between-country comparisons Wootton6 2013 Norway Development of an indicator of individual research output based on grant income, publications, and numbers of PhD students supervised. Assessing research activity and capacity of community-based organizations: refinement of the CREAT instrument using the Delphi method Humphries et al14 2019 US Development of the Community REsearch Activity Assessment Tool (CREAT) instrument using a structured Delphi panel. Most metrics are subjective. Objective, numeric measurements include staff and budget. Assessing research capacity in Victoria’s south-west health service providers Gill et al15 2019 Australia Implementation of the Research Capacity and Culture (RCC) tool which had previously been developed by Holden et al, 2012.16 Assessment of health research capacity in western Sydney local health district (WSLHD): A study on medical, nursing and allied health professionals Lee et al17 2020 Australia Implementation of the RCC tool, demonstrating differences between various professionals.16 Biomedical research productivity: factors across the countries Rahman and Fukui18 2003 Japan Analyzed country of origin for published articles to determine significant factors relating to research output defined as publications per million population per year. Significant factors included gross national product per capita, research and development expenditure, number of science and engineering students, and number of physicians. Building research collaboration networks: an interpersonal perspective for research capacity building Huang19 2014 Singapore Highlights the value of research collaboration networks as evidence of research capacity. Common metrics to assess the efficiency of clinical research Rubio20 2013 US Identification of metrics to assess the efficiency of clinical research processes and outcomes. They identified 15 metrics in 6 categories. Objective, numeric metrics include time for IRB submission to approval, time to publication, and number of technology transfer products. Categories included processes, careers, services, economic return, collaboration, and products. Developing indicators for measuring Research Capacity Development in primary care organizations: a consensus approach using a nominal group technique Sarre and Cooke21 2009 England Development of a list of indicators to measure research capacity development at an organizational level using workshops and modified nominal group technique. Individual metrics include research personnel, funding, membership in research alliances, number of projects, and awards. They were grouped by category according to the model developed by Cooke.22 Development and use of a research productivity assessment tool for clinicians in low-resource settings in the Pacific Islands: a Delphi study Ekeroma et al4 2016 Fiji, Samoa, Tonga, Vanuatu, Cook Islands, Solomon Islands Focus group discussions to obtain viewpoints on meaningful research indicators. They developed a tool of 21 subjective and objective indicators. Example metrics include bibliometrics, funding, recognition, collaboration, and personnel. Evaluating health research capacity building: an evidence-based tool Bates et al23 2006 Ghana Development of a tool to measure clinical research capacity–building programs. The framework was based on reported literature then adapted to the local context through an internal working group. Their resulting tool consisted of a mix of 12 objective and subjective measurements. Sample numeric metrics include bibliometrics, research funding, and researcher remuneration. Evaluation of the research capacity and culture of allied health professionals in a large regional public health service Matus et al24 2019 Australia Evaluation of research among allied health professionals working in a large regional health service using the Research Capacity and Culture (RCC) tool.16 Principal component analyses to determine key components that influence differences between various professional groups. How has healthcare research performance been assessed?: a systematic review Patel et al25 2011 Articles from several countries were included Systematic review of indicators of health care research, along with evidence supporting their use. Indicators include publications, citations, impact factor, funding, authorship, population size, h-index, peer reviews, presentations, patents, doctoral students, and editorial responsibilities. Indicators for tracking programs to strengthen health research capacity in lower- and middle-income countries: a qualitative synthesis Cole et al26 2014 Canada, UK, Switzerland Qualitative evaluation of research evaluations to identify key indicators of research productivity. Quantitative indicators include awards, trainees with a mentor, workshop attendance, courses run by educational institutions, course attendance, collaboration activity attendance, joint projects, and bibliometrics. Measuring research capacity development in healthcare workers: a systematic review Bilardi et al27 2021 UK, Australia, Italy Systematic review and narrative synthesis of articles containing tools to measure health care workers’ individual research capacities. Many articles contained data on team and organizational level. Many domains of assessment were identified, including skills, motivations, bibliometrics, informatics, communication, collaboration activities, studies, ethics, quality, support, skills, infrastructure, leadership, efficiency, dissemination, culture, and sustainability. Measuring, analysis and visualization of research capacity of university at the level of departments and staff members Kotsemir and Shashnov28 2017 Russia Literature review on methods of research capacity in the university. Their analysis focuses primarily on bibliometrics, including number of publications, h-index, impact factor of published studies, and articles with evidence of collaboration. Nine criteria for a measure of scientific output Kreiman and Maunsell29 2011 US Identification of qualities that define an effective research metric. They advocate that metrics should be quantitative, based on robust data, rapidly updated and retrospective, presented with CIs, normalized by number of contributors, career stage and discipline, impractical to manipulate, and focused on quality over quantity. Rehabilitation Medicine Summit: building research capacity Frontera et al5 2006 US Outcomes of a summit convened to advance and promote research in medical rehabilitation. They identified several important domains of research capacity, including research environment, infrastructure, and culture. Objective indicators they identified include bibliometrics and funding. Research capacity building frameworks for allied health professionals - a systematic review Matus et al30 2018 Australia Systematic review of 5 databases to identify models and frameworks for research capacity building. They identified 3 main themes: supporting clinicians in research, working together, and valuing research for excellence. Validation of the research capacity and culture (RCC) tool: measuring RCC at individual, team and organization levels Holden et al16 2012 Australia Development of the Research Capacity and Culture (RCC) tool based on literature review and expert guidance. Validation performed for internal consistency and test-retest reliability. Indicators include funding, bibliometrics, age of researchers, evidence of partnerships and dissemination. Abbreviation: IRB, institutional review board.
Item Description Time frame The time frame intended for monitoring is up to each department to determine. Bibliometrics Each publication, presentation, or patent is counted once regardless of the number of authors. 1. Peer-reviewed publications Number of new original research articles published in the peer-reviewed literature. 2. Publications other than peer-reviewed Number of new original research contributions published outside of the peer-reviewed literature (eg, book chapters). 3. Presentations (oral and poster) Number of new oral and poster presentations given at regional, national, or international meetings or conferences. Presentations may be counted more than once if they are delivered more than once. 4. Number of published faculty Total number of faculty who were listed authors on a publication in the peer-reviewed literature. 5. Number of presenting faculty Total number of faculty who gave an oral or poster presentation at a regional, national, or international meeting or conference. 6. Patents filed Number of new patents filed. 7. Patents issued Number of new patents issued. Impact Researchers include doctoral level and other research faculty as defined under “Personnel.” 8. New citations Number of new citations in peer-reviewed literature of articles written by researchers in the department. This includes new citations for all articles of current researchers, regardless of when the article was published. 9. Median h-index Median h-index for researchers in the department. Ongoing research Ongoing research includes projects approved or deemed exempt by an IRB. 10. New projects with IRB approval Number of projects newly approved or deemed exempt within the past year. 11. Active projects with IRB approval Number of projects actively under way. This includes new projects listed above. Collaboration activities Activities involving participation with organizations outside the department. 12. Joint activities with other research organizations Number of activities as described under “Bibliometrics” or “Ongoing research” which involved direct participation from researchers outside the department (eg, other departments, other schools, or other organizations). 13. Peer-review panels for research funding proposals Number of department faculty who have served on a peer-review panel at the national or international level for extramural/external research or research training funding proposals in the past year. 14. Personnel participating in national/international research leadership Number of department faculty serving in leadership roles in national or international research-focused organizations. This can include committee service with regular meetings (at least twice yearly), committee chair, board of directors, or similar level of leadership. Funding Funding is defined as total direct dollar or in-kind support for activities intended to lead to external and peer/editorially reviewed presentations, publications, and dissemination. This includes start-up costs, bridge funding, core funding, pilot project funding, staff time, investigator support, consultation, and supplies. 15. Internal funding Funding that the department or institution contributed to research activities. 16. External funding (including grants) Funding-derived sources external to the department and external to the institution such as outside grants, industry funding, contracts, or philanthropy designated for research. 17. Other funding Funding that does not fit in the above categories (eg, endowments, royalties). 18. Total funding Sum of the 3 funding sources listed above. Personnel One research FTE includes 40 hours of work per week from personnel in the department whose time is intended to lead to external and peer/editorially reviewed presentations, publications, and dissemination. 19. Doctoral level research FTE Total research FTE of doctoral-level faculty (not including trainees) with primary academic appointments in the department. This includes FTE (paid time designated or paid effort allocated) directed toward research, regardless of the funding source, for their salary compensation in the specified time frame. 20. Other research faculty FTE Total research FTE of other research faculty with bachelor’s or master’s level degree (not including trainees) with primary academic appointments in the department. This includes FTE (paid time designated or paid effort allocated) directed toward research, regardless of the funding source, for their salary compensation in the specified time frame. 21. Nonresearch faculty FTE Total nonresearch FTE of all department personnel at or above master’s level education. This can include time spent for administration, teaching, patient care, or other activities. 22. Total research administration FTE Total FTE for administrative time of all staff with research leadership roles. 23. Total faculty FTE Total of the above 4 items 24. Total faculty Total FTE for research activities of all faculty who perform or support research activities (even if not their whole job, not including trainees). This includes only faculty directly reporting within the department and does not include research faculty in other departments or organizations paid for with grant funds. 25. Total research support staff FTE Total FTE for research activities of all staff who support research activities (even if not their whole job, not including trainees). This includes only staff directly reporting within the department and does not include research support staff in other departments or organizations paid for with grant funds. This may include statisticians, study coordinators, or research aides. Education/academics Trainee publications and presentations are included in this section, as well as in the “Bibliometrics” section. Each publication or presentation is counted once in this section regardless of the number of trainee authors. 26. Research trainees Number of trainees who were actively involved in research during the past year, even if research is not the primary focus of their education. This includes trainees at all graduate levels who are actively contributing to ongoing research or publication activities and does not include trainees not participating in any such activities. 27. Trainee publications Number of publications (peer-reviewed or other than peer-reviewed as defined above under Bibliometrics) with a trainee as a listed author. 28. Trainee presentations Number of presentations (oral or poster) with a trainee as a listed author. 29. Faculty with rank of Assistant Professor Number of research faculty with the academic rank of Assistant Professor or equivalent. 30. Faculty with rank of Associate Professor Number of research faculty with the academic rank of Associate Professor or equivalent. 31. Faculty with rank of Professor Number of research faculty with the academic rank of Professor or equivalent. Abbreviations: FTE, full-time equivalent; IRB, institutional review board.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.