Abstract
Introduction: Obtaining a representative patient sample for research purposes can be challenging. Classic probability sampling can be trusted, but these approaches are not always feasible; yet alternatives may introduce bias. We summarize relevant literature, the need for new approaches, and illustrate a practical hybrid approach that could consistently produce representative patient samples.
Methods: Valid approaches shift sampling decisions from fallible interviewers to less fallible reproducible processes. In lieu of the interviewer's inclination to select particular patients, we used the day of the week, the appointment time, and the sequence of the patient's last name in the alphabet to select a sample for a consent process and a survey.
Illustrative Use: Characteristics of the study sample (n = 225) were compared with the characteristics of the population (N = 1964) that had an office visit during the recruiting period. The data suggested that the study sample was highly representative of the population in this illustrative case.
Discussion: A hybrid sampling approach, in the context of a brief consent process, and a nonthreatening interview produced a representative study sample, but formal evaluation via simulation is needed to validate the hybrid approach. Convenience samples of consecutive patients should be avoided to minimize bias.
- Health Services
- Primary Health Care
- Research Methods
- Sampling Studies
- Selection Bias
- Survey Methods
- Sampling Bias
Obtaining an unbiased or a representative sample of primary care patients for cross-sectional research purposes is challenging. Resources for patient recruiting are usually limited, the research process must not interfere with patient care, and research is rarely a priority for patients. The classic procedures for obtaining an unbiased simple random sample from a patient population involve constructing a sampling frame (or list) of eligible patients. However, simple random sampling is rarely used in large-scale surveys because data collection costs, sample sizes, and statistical power must also be considered.1 Thus, compromises are made and combinations of sampling techniques (simple random sampling, stratified sampling, systematic sampling, or multistage cluster sampling) are used to identify the patients who will be recruited for a given cost and to achieve a specified sample size.1 In primary care, construction and adherence to the sampling frame for a simple random sample may be compromised by real-life office practice. Unscheduled drop-in patients may be seen each day and no-shows occur, thus compromising a list based on patients scheduled the previous day. Convenience samples are used because they are “convenient” and the costs are low, but they are likely to yield biased results.
A key word search (survey sampling and primary care, surveys and primary care, sampling methods, and primary care) of the PubMed and Google Scholar databases (6/24/2010) produced a large number of hits, but the first listed papers were generally not relevant to the issue of methods for selecting patients or visits for cross-sectional research in primary care. We also searched family medicine journal web sites. A dozen reports over a 16-year period were found.2–13 Four studies compare probability and nonprobability sampling.2,6,10,11 The research of Cumming supports the proposition that a systematic sample can yield valid results, but the reported study was done in shopping centers rather than in primary care.2 Smith et al report that a convenient treatment-specific sample (analgesia for chronic pain) yielded a demographic profile similar to that produced by a general practice population sampling scheme.10 However, the patients were offered only hypothetical rather than actual participation in a randomized trial, which changes the context. Rothemich et al describe a self-administered, office-based survey in great detail and report that the nonprobability waiting room sample was similar to the target population; however, the sample was quite large (844/1085) relative to the population, which undercuts the rationale for sampling.6 Feild et al report that probability sampling and nonprobability sampling of hemodialysis patients and their spouses yielded similar demographic profiles, but probability sampling was more efficient. However, the setting was outpatient specialty care, in which the patients are more homogenous and scheduling processes are likely to be more orderly.11 Woolf et al document that the consent process for a survey may introduce moderate to large selection bias, depending on the demographic characteristic under consideration.7 Other literature addresses issues peripheral to this paper.3–5,8,9,12,13 This relatively sparse literature does not adequately address hybrid sampling techniques in primary care. We illustrate a practical hybrid systematic and multistage patient sampling and recruiting scheme that adds to this sparse literature on sampling in primary care.
Description of Sampling Method
The method combined features of a systematic and a multistage sampling approach. In the context of patients and office visits, a systematic sampling scheme might involve selecting a random patient on the first day of the study (a random start) and then attempting to recruit every 5th or 10th patient (a fixed interval related to the desired sample size) across all subsequent study days.14 This assumes that there is a natural way that patients can be counted across days such as on a single sign-in sheet. A multistage (clustered) sampling scheme might involve randomly selecting half-days when patients are seen, then appointment times within the half-days, and then all patients within the selected appointment times.14 The method used in the current study was a hybrid “systematic” and “multistage” sampling approach that was based on the day of the week, clock time, the patient appointment schedule, and the alphabet. Briefly,
-
Index clock times were first selected depending on whether the day of the week was Monday, Wednesday, or Friday (Set 1) or Tuesday, Thursday, or Saturday (Set 2).
-
The index clock times for Set 1 were the hours (9:00, 10:00, 11:00, etc) and the half hours (9:30, 10:30, 11:30, etc).
-
The index clock times for Set 2 were 15 minutes past the hour (9:15, 10:15, 11:15, etc) and 45 minutes past the hour (9:45, 10:45, 11:45, etc).
-
After taking note of the day of week (Set 1 or Set 2), the interviewer checked the time, determined from the patient appointment schedule which (adult) patient had the appointment for the next index time, and approached that patient for a brief informed consent process and an interview.
-
If there was more than one patient scheduled for the index time the interviewer selected the patient whose last name came first in the alphabet. Any arbitrary alphabet rule could be used to break ties, as long as the rule is followed. If the patient uses a non-Latin alphabet, a discrete roll of dice in a cup (or a table of random numbers) could be used as a tie breaker. We assumed that an alphabet rule was more convenient than dice or a table of random numbers to break appointment ties.
Because our family medicine center patients are usually given 15-, 30-, or 60-minute appointments starting at the hour, 15 minutes past, 30 minutes past, or 45 minutes past the hour, the proposed sampling scheme fits with the flow of patients through our office. Other scheduling practices such as 10-, 20-, or 40-minute appointments could be accommodated by the proposed method. If the dominant pattern is six 10-minute appointments per hour, then half could be sampled on Monday-Wednesday-Friday and the other half could be sampled on Tuesday-Thursday-Saturday. If there is a concern about a hidden selection bias related to specific combinations of clock times and days of the week, the index clock times (on the hour or 15 minutes after) can be alternated between Set 1 and Set 2 every other week. Post hoc statistical modeling can also be used to account for specific features of the study design.
Illustrative Use
The illustrative site was an urban family medicine resident training clinic affiliated with a multihospital health system and a medical school. Forty percent of the registered adult patients are African-American, 70% are female, 75% are unmarried, 40% have Medicaid insurance, 10% receive organized charity care, and 6% have no insurance. There has been a recent (after the study) influx of patients from North Central Asia with limited English proficiency who use a non-Latin alphabet.
Data Collection Method Used in This Example
The population and sample data described in this paper were part of a study focused on sleep habits and obesity.15 The study was conducted from mid-May to mid-July, 2009, when a student research assistant was available. Eligible patients (adults) were identified in the family medicine center waiting room. The informed consent process and the interview were conducted either in a semi-private corner of the waiting room, in the examination room, or in a nearby conference room. The local institutional review board approved the use of a one-page statement of research rather than a multipage consent document. The interview concerned an innocuous topic (sleep habits), which is reported elsewhere.15 The questionnaire was administered by the interviewer because of the relatively low educational or literacy levels of some patients.
The target population was all patients who were actually seen on days when the survey data were collected. The data for this sampling example were obtained from the electronic medical record. Prevalent characteristics are emphasized because the sample estimates are more stable.
The proportional distributions of the demographic and clinical variables for the sample and the population were calculated along with 95% (adjusted Wald) confidence intervals for each sample subgroup.16 We also used single-sample z-tests, based on the normal approximation to the binomial distribution and exact binominal tests when the confidence intervals and z-tests yielded contradictory results. As a final check, we performed a global χ2 goodness-of-fit test to compare the entire distribution (of age, ethnicity, etc) in the sample with the corresponding population distribution, in which the population distribution is assumed to be fixed.
Table 1 displays the demographic and clinical characteristics of the target population and the sample patients who were selected and interviewed under the clock and appointment-based the sampling protocol. In every instance except two, the adjusted Wald 95% confidence interval included the population parameter and the single-sample z-test yielded a large P value (P > .05), supporting the conclusion that the sample estimates and the population parameters were statistically identical. The exceptions were a possible underestimation of the frequency of normal weight patients (14.6% relative to 20.1%) and a possible overestimation of the frequency of organized charity care patients (18.8% relative to 12.5%). Snedecor and Cochran point out that the normal approximation to the binomial test can be misleading if the population proportion is 20% or less and the sample size is less than 50, which was the case for the two exceptions.17 Moreover, exact binomial tests for identity between the sample proportions and the population proportions yielded large P values (P > .05) for the two exceptions. Finally, we used χ2 goodness-of-fit tests to ascertain whether the sample distributions were identical to the population distributions, while assuming that the population distributions are fixed. Each test yielded a large P value (P > .05), again suggesting that the sample and the population are statistically identical. The student research assistant reported that few patients (< 10%) refused to participate in the nonthreatening (sleep habits) interview. These data suggest that the clock and alphabet-based sampling approach (in the context of a nonthreatening interview) achieved a sample that is representative of the target population without increasing costs, which were fixed by the time commitment of the student research assistant to the project.
Population and Sample Characteristics
Discussion
The goal under any sampling scheme is to obtain a representative sample of the target population. Standard approaches such as attempting to recruit all successive eligible patients may not be practical if there are only a few patient interviewers available, the inclusion criteria are broad (all adult patients who have an office visit during some time interval), and the office is busy. Moreover, giving the interviewer latitude to select patients is problematic because friendly, healthy patients will be oversampled and grouchy, ill patients will be undersampled. The illustrative procedure deliberately limits interviewer decision-making and produced a representative, unbiased sample for about the cost of a convenience sample.
The method we illustrate shares features with both systematic sampling and multistage sampling.1 Selecting the first patient according to an approximate random process (clock time) and then selecting patients at fixed (30-minute) intervals thereafter is analogous to standard descriptions of systematic sampling.14 However, selecting patients according to day of the week (1 of 2 possibilities), clock times within days (1 of 12 possibilities), and position of name in the alphabet (1 of N possible patients at each clock time) is analogous to multistage sampling. Based on our single global comparison between the sample and the population shown in Table 1, the proposed method should produce less bias than convenience samples, but possibly more bias than true probability sampling, albeit at a lower cost than true probability sampling. A formal evaluation of the alternative sampling methods, associated data collection costs, and the final sample size that can be obtained for a given design is standard procedure for well funded large-scale surveys.12,13 These issues are beyond the scope of this paper. Sampling textbooks and review papers address these issues in more detail.3,14
For our cross-sectional research, the theoretical sampling frame for a simple random sample was the list of eligible patients who made at least one office visit during the data collection period. However, the complete list of these patients could not be known until after the data collection period was over. Thus, a simple random was not feasible. A more complex probability design might be a multistage, clustered sampling scheme based on randomly selected weeks, days, half-days, clock times, and patient characteristics. A formal multistage clustered probability design was not attractive because it apparently would increase data collection costs and reduce the final sample size under a small fixed budget. Our nonprobability approach shares features (day, time, patient characteristic) with a multistage probability design. Investigators could chose to include these design features in their post hoc data analysis models to adjust for possible bias. We chose not to model these theoretical design effects in our substantive analyses of sleep habits because the sample appeared to be highly representative and similar to previous samples from our population.
Previous authors have addressed sampling schemes that could be used for selecting patients or visits for office-based research.2–13 Our study adds to this sparse literature on the pros and cons of alternative ways of sampling patients from primary care. More experience with practical but valid hybrid samples in primary care is needed, especially in the context of a greater emphasis on evidence-based primary care. There are no perfect sampling solutions for all circumstances. Compromises between practicality, costs, statistical power, and a tolerable level of (selection) bias are usually made.12,13
The ultimate success of any sampling approach depends not only on sampling mechanics but also on inherent study risk, study appeal, the complexity of the consent process, the length or content of any research interviews, patient compensation, and the clinical context within which the data collection occurs. Thus, we have described the clinical context around data collection for our study in sufficient detail so that readers can draw their own interferences. More work is needed on practical and valid procedures for selecting primary care samples for research and factors that encourage and discourage participation. Traditionally accepted probability sampling methods should be used wherever possible. We have illustrated a clock-based systematic and multistage sampling procedure that yielded a representative sample, and may be preferable to other nonrandom procedures after further testing.
Notes
-
This article was externally peer reviewed.
-
Funding: Department of Family Medicine, Summa Health System, Summa Foundation, Northeast Ohio Medical University, Akron.
-
Conflict of interest: none declared.
- Received for publication March 8, 2010.
- Revision received December 7, 2010.
- Accepted for publication December 20, 2010.
