Delphi methodology in health research: how to do it?

Abstract

Introduction

Delphi technique is widely used to develop consensus on group opinion. However, no strict guidelines exist and various methods are often employed. The aim of this article was to reflect on Delphi methodology and provide guidance useful to researchers in integrative medicine.

Methods

Two parallel Delphi studies were undertaken to achieve consensus on how to treat phantom limb pain with acupuncture. Whilst completing these studies methodological issues relating to Delphi technique were identified which may be of use to other researchers.

Results

Ten areas were identified;, use of the term ‘expert’, sample size and sample heterogeneity/homogeneity, iteration, structure of round one, optimal number of response categories, inclusion/exclusion of data in subsequent rounds, participant feedback, defining consensus, stability of response and agreement, attrition.

Conclusions

Defining and using the term ‘expert’ is problematic. Three rounds are optimal. Round one data collection and analysis need structuring to avoid generation of unmanageable amounts of data. Subsequent rounds should consider using Likert Scales with four to seven categories, with even number of categories eliminating the problems associated with midpoints. To ensure rigour, data should not be excluded from round three. Participant feedback should include both central tendency and a measure of dispersion and be presented graphically. Consensus should be clearly defined and not confused with stability of response or agreement. Attrition can be minimised by ensuring participants are well informed and through a short time frame between rounds. It is intended that this guidance may help future researchers.

Introduction

The Delphi technique is now a widely used methodology which has the advantage over other consensus methods of not requiring face to face contact [1] whilst still guiding group opinion [2]. It can facilitate wider group participation than other consensus techniques (such as nominal group technique and consensus development conference) so avoiding recruitment bias due to participants geographical location.

Delphi technique evolved due to limitations of traditional methods used to gain group opinion for policy development [3] and was founded on the premise that unstructured, face to face group predictions were weaker than individual statistical predictions [4] The original Delphi method was developed in the 1950’s by Olaf Helmer, Norman Dalkey and Nicholas Rescher of the Rand Corporation to forecast the impact of technology on warfare [4]. It has subsequently been used in healthcare, marketing, education, information systems, transportation engineering [5] and complementary and alternative medicine (CAM) to establish guidelines and establish key components of an intervention [6], [7], [8].

Delphi technique is defined by its use of ‘experts’, and its use of a series of questionnaires interspersed with controlled feedback and provides information on group opinion [5]. It is an appropriate methodology when there is lack of agreement, incomplete knowledge, uncertainty or lack of evidence [9]. This technique does not intend to challenge statistical or model based procedures but instead is intended for use in situations where statistical methods are not practical or possible [5].

Delphi technique has four main characteristics; anonymity between participants, iteration with controlled feedback of group opinion, statistical aggregation of group response and expert input [3]. Anonymity allows views to be expressed and changed privately [5]. Iteration with controlled feedback allows ‘communication’ between participants and perspectives to be shared [9] and allows participants to change views [10]. Statistical aggregation of group responses allows for data to be analysed and interpreted [10]. Expert input means each participant is informed in the area under study [11].

The aim of this article was to reflect on the Delphi technique as a research tool in healthcare. It draws upon a practical example from two studies carried out by the authors and highlights areas for consideration when using this technique [12]. The paper identifies a number of areas which need careful consideration, which could be useful for researchers when deciding whether to use this methodology and provides a list of the main issues (drawn from the above) to consider when using this technique (Table 1). The paper attempts to address areas not commonly considered in other reviews, such as what to consider when designing round one, the size of Likert scale, items to include in subsequent rounds and differentiating between consensus, agreement and reliability. The paper identifies areas needing consideration for the future use of the technique to inform integrative medicine research and provides a ‘how to’ guide.