Abstract
Objective: Aviation accidents have been reduced substantially by training pilots to avoid high-risk behaviors caused by cognitive errors. To determine whether similar cognitive cause factors or errors are involved in medical malpractice cases, and to evaluate the reliability of identifying such factors, physicians reviewed state and federal malpractice cases in a legal database.
Method: Reviewing physicians evaluated 30 cases meeting inclusion criteria from state and federal malpractice cases for the year 2004 in the 8th US Circuit Court of Appeals region, using criteria for cognitive factors derived from aviation. The cases were categorized into three classes based on the degree of agreement between the reviewers. The results as to prevalence of the factors and reliability of identifying the factors were then analyzed statistically.
Results: Fifty-nine percent of the cases met criteria for strong and good correlation with the factors. Cognitive factors involving IMSAFE (illness, medications, stress, alcohol, physiogical (f), and emotional) and medical team management/leadership can be reliably identified. Other factors, such as hazardous attitudes and loss of situational awareness, were identified with minimal reliability.
Conclusion: The aeronautical cognitive causative approach can be translated into a medical approach to reliably identify cognitive causes of errors in a significant proportion of medical malpractice cases from a legal database.
In the 1980s the aviation community under the direction of the Federal Aviation Administration (FAA) found that 85% of all accidents and 52% of all fatal general aviation accidents were due to pilot error, specifically poor decision making or risk management. After 12 years of research and development, it was determined that most errors were cognitive in nature and were the result of attitudes, behavioral traps, stresses, and other influences. Since 1987, the aviation community has been teaching pilots risk-management techniques that include cognitive evaluation of factors such as pilot’s condition before and during a flight, condition of the aircraft, and the environment. The goal has been for pilots to apply this cognitive model to their aviation activities and thereby decrease the accident rate. It has been largely successful, with up to 50% decrease in aviation accidents.1
Applying the criteria of aviation cognitive errors and risks to medicine may yield similar results. The 1999 Institute of Medicine Report found that approximately 44,000 to 98,000 mortalities were attributed directly to preventable medical error each year.2 A recent published study concluded that malpractice litigation is responsible for as much as $97.5 billion to hospital and physician service costs each year.3 The current approach has been to apply a systems method to catch or prevent medical error, for example, electronic medical records, color-coded arm bands, policies, and so forth. Systems do catch errors, but not all. An error in the system in and of itself can be the cause of more error. An approach in understanding the cause of medical errors by providers themselves is essential to effective interventions. Physicians often make medical decisions that are influenced by fatigue, emotion, medications, attitudes, external pressures, and other factors. A cognitive approach combined with a system method together is needed to reduce errors and, in turn, contain both the human and monetary cost of medical errors.
To test the hypothesis that the cognitive causes of errors or risks in medicine are similar to that in aviation, we adapted the criteria defining the aforementioned aviation factors and applied them to actual malpractice cases. The goal was to determine: (1) whether faculty and resident physicians could reliably identify these factors in malpractice cases, and (2) prevalence of errors in the cases included in the review.
Method
Two faculty and two residents were selected as reviewers for the project. One of the participants (SCS) has had more than 20 years of experience in medicine, and 15 years experience as a pilot with a commercial rating. This participant adapted the aviation risk management criteria to medicine. Another participant has had more than 5 years of experience postresidency in medicine. One of the residents was at postgraduate level 3. The other resident was at postgraduate level 2, with additional formal graduate training in public health and health policy.
The initial development of the aviation model was funded by the FAA and was conducted by Ohio State University. It involved several categories of evaluation with emphasis on the pilot. The four fundamental risk elements are the pilot, the airplane, the environment, and external pressures. The pilots were trained to evaluate their personal performance according to an acronym called IMSAFE.1,4–6 I, illness in the pilot that could affect decision making and judgment; M, medications that could affect reflexes and judgment; S, stress in the pilot’s life, environment, and so forth, that could potentially affect judgment; A, alcohol or any other substance ingested that could affect judgment; F, physiologic factors such as fatigue, hunger, and the need to relieve oneself that could influence judgment; E, important emotional factors, such as anger or depression, which are well known to affect judgment. In addition, there are also five hazardous attitudes that affect judgment and outcomes.1,4–7 The hazardous attitudes are as follows: (1) antiauthority, (2) macho (is an exaggerated sense of power or pride), (3) impulsivity, (4) invulnerability, and (5) resignation. The hazardous attitudes were studied and found to be involved in a number of aviation accidents. Other factors involved in poor aviation outcomes were “loss of situational awareness” (LOSA), poor team resource management/leadership, and external pressures.1,4–8
Some adaptation was necessary to allow the risk factors of IMSAFE hazardous attitudes, LOSA factors, external pressures criteria, and team resource management/leadership factors to be properly applied to medical situations. The four fundamental risk elements in medicine became the physician, the patient, environment (hospital, clinic, etc.) and external pressures.9 We addressed two fundamental questions. First, could the translated Medical Decision Making Risk Management (MDMRM) cause criteria be reliably identified in actual malpractice cases by different reviewers? Second, what was the prevalence of these factors in actual cases? The most objective data were felt to be legal databases.
Westlaw is a Web-based law database that can be easily accessed with a password and the assistance of an attorney.10 The malpractice cases obtained involved cases from both federal and state courts in the 8th US Circuit Court of Appeals region in 2004. Cases to be included had to meet the following criteria. (1) There was information describing the facts substantiating the commission of medical malpractice; (2) cases filed by prison inmates were excluded; (3) cases involving only legal technicalities were excluded (such as those exceeding the statute of limitations, or whether certain parties were to be included in the lawsuit); and (4) cases that did not reflect injury sustained by the practice of medicine were excluded (e.g., lawsuits resulting from injuries, such as “slipped or fell on the sidewalk of a clinic”).
Many of the cases in the Westlaw database have no “background and facts.” Those records could not be used as the “background and facts” of the cases were needed to analyze for the criteria. The facts of the cases have already been determined in court and therefore are not in dispute. Also, a large proportion of cases involve primarily legal technicalities, such as statute of limitations questions, and who can be named, etc. These cases were not useful because they usually did not have background and facts and were not concerned with answering the question of the cause of malpractice. Many of the prison cases were felt to be nuisance or frivolous in nature and therefore were not included. Some cases were not actual malpractice cases but were personal injury cases of the type that involved falls on medical facility premises, etc. Due to peculiarities in the system, these cases were included in the database. These types of cases were not included in the review.
The Westlaw is not a comprehensive library of all the malpractice cases that are filed in the 8th US Circuit Court of Appeals region. Many of the cases filed were settled out of court, and these are not recorded here. Some cases were dismissed or dropped and therefore are not recorded. This study was not meant to be a comprehensive review of cognitive causation of medical malpractice cases. Rather, this study was designed to examine cases that went to trial and met the criteria set forth.
The medical malpractice criteria were derived from the FAA training manuals4–8 developed in 1987 and adapted to evaluate medical practice by one of our authors (SCS) (Table 1). The reviewers were then given printed summaries of the cases and the criteria to review. The reviewers then indicated on separate reporting papers, whether the case met any of the criteria included in Table 1. Each reviewer was unaware of the activities and evaluations of the other reviewers. The cases were then classified according to the number of criteria judged as met by the reviewers. Cases meeting three or more criteria were placed in Class A (strong correspondence), those meeting two criteria were assigned to Class B (good correspondence), and cases in Class C met one or no criteria. To test the inter-reviewer agreement of these assessments, a Kappa analysis was conducted.11 A Kappa of 1 shows perfect agreement, whereas a Kappa of 0 indicates agreement no better than chance. Overall Kappa values, across criteria, were also calculated to compare the primary rater with the others using SPSS statistical software. T values were calculated to determine significance of these Kappa values.
Malpractice Case Criteria
Results
A total of 126 cases were reviewed for inclusion into the study. Of those, 30 cases met inclusion criteria; 14 (46%) met criteria for Class A; 5 (13%) met criteria for Class B; 11 (35%) met criteria for Class C; and 19 (59%) met criteria for Class A or B. The specific criteria (as listed in Table 1) met by each case are shown in Table 2. As indicated in Table 3, the individual reviewers could reliably identify IMSAFE and medical team management/leadership. However the concordance with LOSA criteria was only minimally to moderately good. IMSAFE criteria were reliably identified to a Kappa of 0.71, with a standard deviation of 0.23. Medical team management/leadership was reliably identified to a Kappa of 0.63, with a standard deviation of 0.22. Other criteria were less reliably identified as seen in Table 3. External pressures did not have a enough data to statistical analyze as to reliability. Overall Kappa values (summed across situations) were statistically significant for comparisons between all the raters: (SxF, Kappa = 0.290, N = 30, P < .001; SxB, Kappa = 0.495, N = 30, P < .001; SxW, Kappa = 0.463, N = 28, P < .001; BxF, Kappa = 0.408, N = 36, P < .001; BxW, kappa = 0.615, N = 30, P < .001; FxW, kappa = 0.333, N = 29, P < .001)
Malpractice Cases Criteria Classification
Kappa of Criteria Values (Correlation Between Observers)
Discussion
Since the Institutes of Medicine seminal publication “To Error is Human,”2 medical error and patient safety has become a priority. It seems that much work has been done using primarily a systems approach. Work has been done by others in classification and taxonomy of medical errors12–14; this classification system describes them as process errors. As described by Elder and Dovey,12 the process errors clarify how something went wrong, not the causation. The classification defines only the deficiencies in the system that allows cognitive errors to proceed to patient injury. They do not explain the causation of those cognitive errors.
It is known that aviation has been using a very successful cognitive model for twenty years, along with a systems approach. Converting the cognitive part of the model to medicine could provide a valuable tool that has been tested previously by the aviation community. By testing the reliability of evaluating malpractice cases, we determined that aviation and medicine have common cognitive causes of errors. The aviation model provided an approach to look at cognitive errors in actual malpractice court cases.
However, when medical malpractice cases have been analyzed previously, they only quantified the relative risk of a malpractice suit being filed based on patient complaints, factors (other than cognitive) that prompted the filing, the types of claims, and contributing factors involved in the claims.15–17 There appeared to be little correspondence between actual occurrences of adverse events, negligence, and the initiation of malpractice, and outcomes of those lawsuits.17 However, the data from the present study indicate an alternative hypothesis. Cognitive causes of errors on the part of the provider may be the basis for a malpractice suit, even if technically there was no evidence of gross negligence or error during the medical care of the patient.
The data show that some MDMRM criteria, notably IMSAFE and medical team management/leadership, can reliably be identified in court cases by reviewers with different levels of experience. Indeed, we should look at medical malpractice cases in a systematic fashion to determine the cognitive causation of errors. Additional support for a cognitive basis for medical error comes from an article by Landrigan et al,18 which showed that one of the IMSAFE criteria, fatigue, was involved in errors made by medical interns in an ICU environment.
The prevalence of cases in this study involving these cognitive causative factors within the limited sample ranged from 13% for Class B alone to an impressive 59% for both Class A and B, in the year and region reviewed. The prevalence may vary in other databases, for other years and regions of the country, and further studies should be done to verify the results presented here. Further refinements in the criteria can be and should be made for medical decision making risk management based on the aeronautical model. Reliability of identifying those criteria should be studied further. Training in preventing errors based on cognitive factors has effectively reduced pilot errors. A similar training program for physicians may have the same result.
Conclusions
The aeronautical, cognitive approach can be translated into a medical approach that can provide a reliable method to evaluate and identify cognitive causes in a significant proportion of medical malpractice cases that go to trial.
Acknowledgments
We thank Lynn Boughey, J.D. of the Boughey Law Firm for assistance in data collection; Dr. Milton Smith, Chairman of the Department of Family Medicine (University of North Dakota), Dr. Wade Talley, Assistant Professor Family Medicine, Center for Family Medicine (University of North Dakota), and Dr. Brenda Cary, Center for Family Medicine (University of North Dakota) for participating in the study.
Notes
This article was externally peer reviewed.
Conflict of interest: none declared.
- Received for publication March 9, 2006.
- Revision received June 2, 2006.
- Accepted for publication June 12, 2006.
Jump to section
Related Articles
Cited By...
- No citing articles found.