You are here

Article Text

Article menu

Download PDFPDF

SPECIAL PAPER
Decision support and safety of clinical environments
Free
  1. A H Morris
  1. Correspondence to:
 Dr A H Morris, Pulmonary Division, LDS Hospital, Salt Lake City, Utah 84143, USA;
 amorris{at}ihc.com

Abstract

Safety in the clinical environment is based on structures that reduce the probability of harm, on evidence that enhances the likelihood of actions that increase favourable outcomes, and on explicit directions that lead to decisions to implement the actions dictated by this evidence. A clinical decision error rate of only 1% threatens patient safety at a distressing frequency. Explicit computerised decision support tools standardise clinical decision making and lead different clinicians to the same set of diagnostic or therapeutic instructions. They have favourable impacts on patient outcome. Simple computerised algorithms that generate reminders, alerts, or other information, and protocols that incorporate more complex rules reduce the clinical decision error rate. Decision support tools are not new; it is the new attributes of explicit computerised decision support tools that deserve identification. When explicit computerised protocols are driven by patient data, the protocol output (instructions) is patient specific, thus preserving individualised treatment while standardising clinical decisions. The expected decrease in variation and increase in compliance with evidence-based recommendations should decrease the error rate and enhance patient safety.

  • decision support tools
  • error
  • safety

https://doi.org/10.1136/qhc.11.1.69

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Safety in the clinical environment is based on structures that reduce the probability of harm, on evidence that enhances the likelihood of actions that increase favourable outcomes, and on explicit directions that lead to decisions to implement the actions dictated by this evidence. Fundamental to all appropriate action are the skills of operators to execute the proper tasks correctly. The successful translation of evidence-based research results into clinical practice has the potential to reduce errors in healthcare delivery, both near misses and errors leading to injury (adverse events),1, 2 including both execution (slips and lapses) and planning errors.3, 4 Medicine is a domain “... in which there are available many more wrong responses than correct ones...”.5 Left to our own devices, we clinical decision makers are too likely to make well intended but erroneous decisions. The healthcare delivery system has a well established need for innovative methods to translate research results and evidence-based guidelines into practice. The human behaviour modification required for some of these new methods is a barrier to implementation. Providers commonly fail to adopt and implement evidence-based interventions. For completeness, one must acknowledge that some error may enhance favourable patient outcomes. For example, an error in administration of agents to suppress premature ventricular contractions following myocardial infarction 15 years ago would have violated the consensus recommendations. Yet these recommendations were later proved to be incorrect and the omission error would have increased the patient's probability of survival.6–8 Given the imperfect state of our clinical knowledge, we must remain alert to the need to formally and systematically test the effects of interventions intended to reduce error. Their impact on ultimate clinical outcomes should be an important determinant of their role in clinical practice.

    Reason dictates that we reduce errors. However, human limitations guarantee that clinical error will not disappear. Human error and injury are unavoidable.3, 4, 9, 10 Clinical error rates are common (1–50%).9–37 Most human errors are not as egregious as those that lead to the occasional wrong limb amputations so dramatically touted in the news and are therefore difficult to detect. Adverse drug events are generally undetected. Traditional screening for in-hospital adverse drug events detects only 1% and voluntary reporting only 12% of the adverse drug events detected by automated computerised screening of an integrated electronic clinical database.38 In addition, the impact of clinical error can be important, even when the error rate is as low as 1%.36 Many, if not most, clinical errors result from system problems. Humans cannot be relied upon consistently to render decisions that comply with evidence-based recommendations.

    Humans have limited ability to deal effectively with large amounts of information. A clinical decision error rate of only 1% in the intensive care unit (ICU) threatens patient safety at a distressing frequency.27, 36 This is probably also true of other, non-ICU, environments. This provides a benchmark for the standardisation of clinical decisions and the reduction of unnecessary variation in practice. The goal for standardisation should be an error rate of <1% if we are to make important progress towards elimination of threats to patient safety. The impact of most efforts, including guidelines and education, falls far short of this goal. A systems approach to this problem has clear potential to ameliorate the current unacceptable clinical error rate. The use of computerised protocols at the point of care to establish an explicit method seems to be a reasonable means of iteratively improving treatment, of reducing error, and of increasing quality.39–41

    IMPORTANCE OF EXPLICIT METHODS

    An explicit decision support tool standardises clinical decision making and leads different clinicians to the same set of diagnostic or therapeutic instructions. By standardising clinical decision making, it establishes an explicit clinical method and provides a basis for comparison with alternative clinical methods. In the parlance of the continuous quality improvement movement, an explicit method is part of the “stabilisation of process” necessary to improve quality.42–45 Both paper based and computerised protocols can theoretically contain the detail necessary to include most clinical scenarios and patient data combinations. Clinicians draw inferences with paper based protocols. An inference engine (computer program) draws inferences with computerised protocols. In practice, the different methods of drawing inferences produce different levels of detail in the rule sets of the protocols. The computerised protocol contains much more detail than is humanly possible to expect in the paper based protocol. Clinicians must use their judgment to fill in the gaps in logic, even when the paper based protocol contains much detail.46 This fosters variation among clinical decision makers and unnecessary variation in clinical practice and research. It does not seem possible for humans, without the stimulus of computerisation, to include the detail necessary for explicit protocols for complex clinical problems.

    Clinical care (the treatment a patient receives) is determined by both the clinical caregiver's decisions and by the patient's individualised response to the illness and interventions.39, 40 Using explicit decision support tools to standardise the clinician's response to the patient's expressions of disease, we can approach more closely the patient's contribution to this patient-caregiver relationship. Variability in response among clinicians29, 47–52 is an important contributor to “noise” in the patient-caregiver system.28, 39, 40, 53 Decision support tools decrease clinician induced “noise” because they decrease unnecessary variation. An increase in the signal-to-noise ratio for clinical outcomes can follow and thus enhance our ability to recognise changes in clinical outcome. Unexpectedly, individualisation of patient treatment is preserved when clinical decisions are standardised with explicit detailed patient data driven computerised protocols.39, 40 This is one of the most attractive attributes of the point of care use of computerised protocols. An essential element in achieving this unexpected result is the use of patient data—that is, the patient's unique expression of the disease54–61—to drive the decision support tool (protocol) rules. In contrast, time driven decision support tools—for example, a clinical path that requires discharge of the patient after 3 days of care—raise legitimate concerns about patient invariant (“cookbook”) care.

    EVIDENCE THAT DECISION SUPPORT TOOLS INCREASE SAFETY

    Control of the process of medical care is beneficial.62, 63 Decision support tools can be categorised as “reminders”, “consultants”, or as “educational” products.64 Many forms of guidelines, clinical paths, and protocols are available. They can support clinical decision making65 and influence clinician performance and patient outcome favourably.62, 66–68 They can avoid vexing problems in different areas of clinical medicine47, 69 and thus appear generally applicable. They respond to limitations of human decision making that are largely independent of the task. Benefit can therefore be expected in many healthcare domains.

    Protocols with the greatest potential to standardise clinical decisions explicitly articulate the rules for selecting patients, for identifying patient states, or for generating specific instructions for treatment. Of these, explicit computerised protocols contain the greatest detail. They may achieve the upper limit of uniformity of clinical decision making, short of the application of the closed loop controllers70 that eliminate humans from the decision making process. Computer based clinical decision support systems have favourable impacts on patient outcome.68 Simple computerised algorithms that generate reminders, alerts, or other information12, 58, 71–73 and protocols that incorporate more complex rule based and extensive data based instructions59, 61, 74 reduce the clinical decision error rate. Paper based and computerised decision support tools that provide explicit point of care (point of decision making) instructions to clinicians have achieved clinician compliance rates of 90–95%.40, 46, 61

    Evidence has long supported the conclusion that standardisation of clinical decision making with computerised protocols is desirable and productive in both clinical research and clinical care. Computerised protocols have favourable impacts on important clinical outcomes in hospital pharmacy and infectious disease departments58, 59, 75–77 and in both outpatient and inpatient hospital practice.12, 38, 61, 71, 75, 76, 78–89 Computerised protocols have controlled the intensity of care of patients with acute respiratory distress syndrome in both treatment arms of a randomised clinical trial.55 Three benefits follow the use of such explicit computerised protocols: (1) precise description of the method (process) of patient care (the rules and logic for clinical decision making); (2) assurance of equal intensity of care (experimental group equation or equivalence5); and (3) achievement of common intermediate end points (for example, treatment regulated to produce the same Pao2 and pH). While a large body of evidence underpins the use of explicit guidelines and their favourable impact on important clinical outcomes, these protocols have only controlled a small part of the unnecessary variation in critical care and even less in medical care in general. Only a small number of important non-experimental cointerventions—such as mechanical ventilation, antibiotic choices, intravenous fluid therapy, and haemodynamic support—have yet been systematically addressed with explicit decision support tools.

    “Evidence has long supported the conclusion that standardisation of clinical decision making with computerised protocols is desirable and productive in both clinical research and clinical care”

    Explicit decision support might also reduce safety, for several reasons. Firstly, guidelines (or more explicit detailed protocols) might induce harm if they were inflexible and failed to respond to patient differences. They would lead to “cookbook medicine” and, without proper clinical oversight, result in errors. They would probably not be appropriate for the management of more complex clinical issues. However, they can be used for simple tasks such as oxygen therapy following surgery with general anaesthesia or the use of penicillin or its equivalent for group A streptococcal pharyngitis in children. Secondly, clinical trainees might abandon critical thinking and guidelines might contribute to the production of clinicians less prepared for the rigorous intellectual challenge of healthcare delivery. However, a wisely used explicit method can be an effective teaching tool for both decision making and clinical practice. Explicit decision support tools articulate the variables considered and the decision rules, unlike much traditional clinical teaching. When training is valued, explicit methods can be an asset. When training is not valued they could be a disadvantage. Like any tool they can be misused. Thirdly, some fear that clinical innovation might be stifled, particularly if clinician reimbursement were linked to standard practice guidelines. Finally, many physicians are concerned that their role will be diminished and that they may become disenchanted with medicine if widespread mandatory guideline and protocol use is instituted.90

    ATTRIBUTES OF SUCCESSFUL GUIDELINES/PROTOCOLS

    Thousands of decision support tools with different names, foci, and outputs are currently available to clinical practitioners. Many of these lack specific instructions for commonly encountered clinical practice scenarios and are useful only in a conceptual sense.88, 90–95 They neither standardise clinical decisions nor lead to uniform implementation of clinical interventions, although that is their intended purpose (table 1).88, 95, 96 Inexplicit guidelines that omit important details force the clinician to fill in the gaps in logic or identified variables.97–99 Judgment, background, and experience will vary among clinicians and so, therefore, will the choices of rules and variables they use to fill in the gaps of inexplicit guidelines and algorithms. In addition, a single clinician will probably fill in the logic gaps with different choices at different times, even though faced with the same clinical expression of the disease.

    View this table:
    Table 1

    Comparison of four tools (developed elsewhere) available for withdrawing mechanical ventilation from patients following cardiac surgery showing both the variability between decision support tools and the inexplicit nature of most of them

    Clinicians and patients expect treatment to be tailored to the patient's specific needs. If decision support tools are perceived to be unable to accommodate relevant variations among patients, clinicians will reject them. It seems axiomatic to demand that clinicians respond to the patient's individualised expression of disease. The anticipated patient responses are major determinants of clinical decision making. However, this focus can be misleading. While the understanding of patient response is our goal, the important unit of analysis for advancement of our clinical knowledge is not the patient, but the combined patient-caregiver unit.39, 40, 55 The patient-caregiver unit is one example of the holistic “transactional unit” articulated by Altman and colleagues.100, 101 One cannot study patient response effectively in the absence of the clinician and the surrounding clinical environment, just as one cannot study the clinician's response in the absence of the patient.

    WHAT NEW ATTRIBUTES DO EXPLICIT COMPUTERISED DECISION SUPPORT TOOLS CONTRIBUTE?

    Decision support is a broadly applicable concept that embraces many activities in human life. Physicians have for years used tools brought to the point of care (point of decision making) as aids. These include a myriad of pocket editions of texts, antibiotic therapy guides, diagnostic algorithms, and treatment synopses. Decision support tools are therefore not new. It is the new attributes of explicit computerised decision support tools that deserve identification here. These include point of care application with stationary or hand-held devices, incorporation of enough detail to be explicit, capture and storage of patient data in an electronic format, identification of temporal changes, consistency, and reproducibility. When explicit computerised protocols are driven by patient data, the protocol output (instructions) is patient-specific, thus preserving individualised treatment while standardising clinical decisions.39, 40 This is an important non-intuitive property that deserves emphasis among clinicians.

    WHY WOULD ONE EXPECT DECISION SUPPORT TOOLS TO IMPROVE PATIENT SAFETY?

    Clinical decision makers are faced with more variables than they can easily manage.9, 10, 18, 26, 27, 29, 71, 102, 103 Our interpretation of physiological data is frequently inaccurate.49 Ill defined terms or statements—such as the advice from the Society of Critical Care Medicine that “... caution should be exercised when pulmonary artery occlusion pressure becomes increased to the extent that pulmonary edema is a risk”104—contribute to variability and error.39, 40, 48 Clinicians cannot reasonably be expected systematically to generate therapeutic decisions that are coherent, that consider all appropriate options, and that are consistent with the relevant scientific evidence.105

    Compliance of physicians with evidence-based treatments or guidelines is low across a broad range of healthcare topics.54, 60, 106–114 The links between compliance, quality, error, and ultimate clinical outcome are complex and difficult to tease out of the literature. Physician compliance with quality indicators is higher than compliance with evidence-based practice guidelines, and reached 75% when using intense physician feedback with achievable benchmarks.115 However, this does not illuminate the issues that surround the standardisation of clinical decision making at the point of care.56, 61, 116 The goals of quality assessment and those of clinical care need not coincide. One must be circumspect when reviewing quality indicator reports if one's interest lies in clinical decision making. Computerised protocols for mechanical ventilation have been associated with clinician compliance of 94% in 11 different hospitals.40, 55, 61 Many factors, including cultural issues and health beliefs, influence compliance.117, 118 Widespread distribution of evidence-based guidelines119, 120 and education programmes121–125 have had only a limited impact on compliance.

    CULTURAL CONDITIONS THAT ENHANCE ADOPTION OF EXPLICIT DECISION SUPPORT TOOLS

    One must incorporate cultural concerns to fully grasp the determinants of safe care. Humans resist the adoption of innovations when they threaten the status quo. Decades of persistent work by innovative thinkers have produced products that appear to provide satisfactory decision support in clinical practice but these have not been widely adopted.126–131 In fact, the healthcare profession may be unusually resistant to the adoptions of “disruptive innovations”, innovations perceived to be a threat to the institutions and providers of current practice.132 The absence of requisite infrastructure in the clinical environment is an important obstacle to the adoption of clinical decision support tools. Computer systems require major investments. The unstable information technology business with its frequent takeovers and failures (such as Emtek©, Oasis©) presents additional barriers to the practising community and its clinicians.

    The cultural environment at LDS Hospital and Intermountain Health Care Inc has fostered progress in decision support development during the past 25 years. The LDS Hospital enjoys a striking tradition of collaboration and group development which may be a reflection of the attitudes of the majority Mormon culture. It set the stage for collaboration between front line clinicians, researchers, physicians, nurses, respiratory therapists, and administrators. The other central building block was the integrated electronic medical record (HELP system) established in about 1970 at LDS Hospital.133–137 We began in critical care in the 1970s with development of monitoring, data management and reporting, and reminder outputs from the integrated electronic HELP database.138 We then developed more complex outputs using logical arguments and data from multiple sources. One such example is the following alert (paraphrased) generated as part of a daily routine respiratory care quality assurance activity using data from the HELP database: “Patient ## in W820 grew Pseudomonas aeruginosa from a sputum culture. This organism was also recovered from patient ## in E630; respiratory therapist #### cared for both patients.”138 We then developed complex protocols in paper and then computerised versions139 to meet a research need.55 The collaborative environment and our mutual commitment to the research question launched our work in decision support with the unexpected outcome that decision support appeared to be more important than the investigation for which it had been initially invoked.39–41, 56, 61, 139–141 Our collaborations with decision support tools evolved into a broader interest in safety. We use protocols to reduce unnecessary variation, reduce error, and increase patient safety through systems approaches.142–145

    SYSTEM VERSUS INDIVIDUAL APPROACH

    Human error and injury have their roots in the organisation itself.27, 146, 147 The common focus on the individual clinician as the pertinent element of analysis27, 148, 149 is at odds with multiple lines of reasoning and scholarship, including systems theory,150 scaling,150, 151 the understanding of emergent properties that appear with higher levels of integration,152 and the transactional world view of environmental psychology.100, 101

    Systems can be configured to support human decision makers. Systems configured with checks (for example, data consistency checks in industrial plants and internal consistency checks for clinical data such as systolic, diastolic and mean blood pressure, heart rate from ECG and pulse oximetry), reminders at the point of decision making (automobile seat belt warning lights), and automatic interlocks (automobile brake ignition interlocks) all reduce human error rates. Systems without these basic decision support tools have been associated with disaster—for example, the Chernobyl nuclear power plant meltdown.

    The evidence that guidelines and explicit methods improve clinical outcomes and safety in healthcare delivery systems is compatible with results from systems approaches in other human activities. In the manufacturing sector an understanding of the performance of the system (stabilisation of process) is a prerequisite to quality improvement.42–45, 153 In the airline and aerospace industry, in the nuclear power and its regulatory industry, and in the military the adoption of systems approaches to error control and the use of root cause and accident precursor analyses are widespread and successful. Human behaviour in healthcare delivery is qualitatively similar to that in these other human activities. Experience in these other fields probably applies to health care.3, 4, 154, 155 Applications of root cause and accident precursor analyses in healthcare delivery systems appear promising.1

    CONCLUSIONS

    Medicine, like social science, enjoys an “ecology of science ... in which there are available many more wrong responses than correct ones ...”.5 Individualised decision making unsupported by outcome data is likely to be both variable and incorrect in complex clinical circumstances. Computerised protocols possess a unique potential for increasing the rigour of clinical care and of experimental clinical research by providing explicit methodology with the highest level of detail while preserving individualised (patient specific) treatment. The expected decrease in variation and increase in compliance with evidence-based recommendations should decrease error and enhance patient safety. Achievement of this end will require a culture that supports collaboration and consensus, one that fosters a synthesis of thought and consensus as a complement to the individual decision making freedom of the past.

    Key messages

    • Safety in the clinical environment is based on structures that reduce the probability of harm, on evidence that enhances the likelihood of actions that increase favourable outcomes, and on explicit directions that lead to decisions to implement the actions dictated by this evidence.

    • Explicit computerised decision support tools standardise clinical decision making and lead different clinicians to the same set of diagnostic or therapeutic instructions.

    • Simple computerised algorithms that generate reminders, alerts, or other information, and protocols that incorporate more complex rules reduce the clinical decision error rate.

    • Decision support tools are not new; it is the new attributes of explicit computerised decision support tools that deserve identification.

    • When explicit computerised protocols are driven by patient data, the protocol output (instructions) is patient specific, thus preserving individualised treatment while standardising clinical decisions.

    • The expected decrease in variation and increase in compliance with evidence-based recommendations should decrease the error rate and enhance patient safety.

    Acknowledgments

    The author is indebted to the medical, nursing, and respiratory therapy staffs of the Intermountain Respiratory Intensive Care Unit and of the Shock-Trauma/Intermountain Respiratory ICU and the Respiratory Care Department of the LDS Hospital for collaboration during the past 25 years. The author acknowledges the contributions of Drs James Orme Jr, Terry Clemmer, Lindell Weaver, Frank Thomas, Tom East, Jane Wallace, George Thomsen, James Pearl, Nat Dean, and Brad Rasmusson for their collaboration in protocol development and implementation; Drs Paul Barach, Steven Small, Brent James, Jonathan Nebeker, Charlene Weir, and Polly Wiessner for insights into errors in health care delivery; and Drs Roberta Goldring, Robert Rogers, and Waldemar Johanson who, through their vision and insight, have enabled the decision support effort for the past 16 years.

    REFERENCES

    1. ↵
      Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ2000;320:759–63.
      OpenUrlFREE Full Text
    2. ↵
      Berman S. The AMA clinical quality improvement forum on addressing patient safety. Jt Comm J Qual Improv2000;26:428–33.
      OpenUrlPubMed
    3. ↵
      Reason J. Human error. Cambridge, UK: Cambridge University Press, 1990.
    4. ↵
      Reason J. Human error: models and management. BMJ2000;320:768–70.
      OpenUrlFREE Full Text
    5. ↵
      Campbell D, Stanley J. Experimental and quasi-experimental designs for research (reprinted from Handbook of research on teaching, 1963). Boston: Houghton Mifflin Co, 1966.
    6. ↵
      Cardiac Arrhythmia Suppression Trial (CAST) investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med1989;321:406–12.
      OpenUrlCrossRefPubMedWeb of Science
    7. Pratt C, Brater D, Harrell Jr F, et al. Clinical and regulatory implications of the cardiac arrhythmia suppression trial (editorial). Am J Cardiol1990;65:103–5.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      Greene H, Roden D, Katz R, et al, CAST investigators. The cardiac arrythmia suppression trial: first CAST ... then CAST-II. J Am Coll Cardiol1992;19:894–8.
      OpenUrlPubMedWeb of Science
    9. ↵
      Abramson NS, Wald KS, Grenvik ANA, et al. Adverse occurrences in intensive care units. JAMA1980;244:1582–4.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      Wu A, Folkman S, McPhee S, et al. Do house officers learn from their mistakes? JAMA1991;265:2089–94.
      OpenUrlCrossRefPubMedWeb of Science
    11. Williamson J, Goldschmidt P, Jillson I. Medical practice information demonstration project: final report. Contract #282-77-0068GS. Baltimore, MD: Office of the Assistant Secretary of Health, DHEW, 1979.
    12. ↵
      McDonald CJ, Wilson GA, McCabe G Jr. Physician response to computer reminders. JAMA1980;244:1579–81.
      OpenUrlCrossRefPubMedWeb of Science
    13. West J. An autopsy method for evaluating trauma care. J Trauma1981;21:32–4.
      OpenUrlPubMedWeb of Science
    14. Morris AH, Chapman RH, Gardner RM. Frequency of technical problems encountered in the measurement of pulmonary artery wedge pressure. Crit Care Med1984;12:164–70.
      OpenUrlPubMedWeb of Science
    15. Morris AH. Elimination of pulmonary wedge pressure errors commonly encountered in the ICU. Cardiologia (Italy)1985;30:941–3.
      OpenUrlPubMed
    16. Browner W, Newman T. The analogy between diagnostic tests and clinical research. JAMA1987;257:2459–63.
      OpenUrlCrossRefPubMedWeb of Science
    17. Physicians ACo. Working conditions and supervision for residents in internal medicine programs: recommendations. Ann Intern Med1989;110:657–63.
    18. ↵
      Iberti T, Fischer E, Leibowitz M, et al. A multicenter study of physician's knowledge of the pulmonary artery catheter. JAMA1990;264:2928–32.
      OpenUrlCrossRefPubMedWeb of Science
    19. Brook R. Using scientific information to improve quality of health care. In: Warren K, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. New York: New York Academy of Sciences, 1993: 74–85.
    20. Chalmers T, Lau J. Randomized controlled trials and meta-anlayses in gastroenterology: major achievements and future potential. In: Warren K, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. New York: New York Academy of Sciences, 1993: 96–106.
    21. Chan L, Schonfeld N. How much information is lost during processing: a case study of emergency department records. Comput Biomed Res1993;26:582–91.
      OpenUrlCrossRefPubMedWeb of Science
    22. McNeil B. Use of claims data to monitor patients over time: acute myocardial infarction as a case study. In: Warren K, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. New York: New York Academy of Sciences, 1993: 63–73.
    23. O'Connor G, Plume S, Wennberg J. Regional organization for outcomes research. In: Warren K, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. New York: New York Academy of Sciences, 1993: 44–51.
    24. Warren K, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. New York: New York Academy of Sciences, 1993.
    25. Wennberg J, Barry M, Fowler F, et al. Outcomes research, PORTs, and health care reform. In: Warren K, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. New York: New York Academy of Sciences, 1993: 52–62.
    26. ↵
      Iberti TJ, Daily EK, Leibowitz AB, et al. Assessment of critical care nurses' knowledge of the pulmonary artery catheter. Pulmonary Artery Catheter Study Group. Crit Care Med1994;22:1674–8.
      OpenUrlPubMedWeb of Science
    27. ↵
      Leape L. Error in medicine. JAMA1994;272:1851–7.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      Tyson NdG. Signal versus noise. Natural History1996;105:72–6.
      OpenUrl
    29. ↵
      Gnaegi A, Feihl F, Perret C. Intensive care physicians' insufficient knowledge of right-heart catheterization at the bedside: time to act?. Crit Care Med1997;25:213–20.
      OpenUrlCrossRefPubMedWeb of Science
    30. Schulz E, Barrett J, Price C. Read code quality assurance: from simple syntax to semantic stability. J Am Med Inform Assoc1998;5:337–46.
      OpenUrlAbstract/FREE Full Text
    31. Sudeep N, Anuradha L, Obasanjo O, et al. Errors in the treatment of tuberculosis in Baltimore. Chest2000;117:734–7.
      OpenUrlCrossRefPubMedWeb of Science
    32. Palevsky PM, Bhagrath R, Greenberg A. Hypernatremia in hospitalized patients. Ann Intern Med1996;124:197–203.
      OpenUrlCrossRefPubMedWeb of Science
    33. Kohn L, Corrigan J, Donaldson M, eds. To err is human: building a safer health system. Washington, DC: National Academy Press, 1999.
    34. Nakhleh RE, Zarbo RJ. Amended reports in surgical pathology and implications for diagnostic error detection and avoidance: a College of American Pathologists Q-probes study of 1,667,547 accessioned cases in 359 laboratories. Arch Pathol Lab Med1998;122:303–9.
      OpenUrlPubMedWeb of Science
    35. Krizek TJ. Surgical error: ethical issues of adverse events. Arch Surg2000;135:1359–66.
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      Gopher D, Olin M, Badihi Y, et al. The nature and causes of human errors in a medical intensive care unit. In: Proceedings of the human factors and ergonomics society annual meeting. 1989: 956–60.
    37. ↵
      Jones JK, Fife D, Curkendall S, et al. Co-prescribing and co-dispensing of cisapride and contraindicated drugs. JAMA2001;286:1607–9.
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      Classen DC, Pestotnik SL, Evans RS, et al. Computerized surveillance of adverse drug events in hospital patients. JAMA1991;266:2847–51.
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      Morris A. Algorithm-based decision making. In: Tobin M, ed. Principles and practice of intensive care monitoring. New York: McGraw-Hill, 1998: 1355–81.
    40. ↵
      Morris A. Developing and implementing computerized protocols for standardization of clinical decisions. Ann Intern Med2000;132:373–83.
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      Morris A, Cook D. Clinical trial issues in mechanical ventilation. In: Marini J, Slutsky A, eds. Physiologic basis of ventilatory suport. New York: Marcel Dekker, 1998: 1359–98.
    42. ↵
      Deming W. Quality, productivity, and competitive position. Cambridge, Mass: Massachusetts Institute of Technology, Center for Advanced Engineering Study, 1982.
    43. Deming W. Out of the crisis. Cambridge, Mass: Massachusetts Institute of Technology, Center for Advanced Engineering Study, 1986.
    44. Shewart W. Economic control of quality of manufactured product. New York: D Van Nostrand, 1931 (republished in 1980 by the American Society for Quality Control, 230 W. Wells St, Milwaukee, Wisconsin 53203).
    45. ↵
      Walton M. The Deming management method. New York: Putnam Publishing Group (Perigee books), 1986.
    46. ↵
      The Acute Respiratory Distress Syndrome Network. Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med2000;342:1301–8.
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      Leiner G, Abramowitz S, Small M. Pulmonary function testing in laboratories associated with residency training programs in pulmonary diseases. Am Rev Respir Dis1969;100:240–4.
      OpenUrlPubMedWeb of Science
    48. ↵
      Morris A. Hemodynamic guidelines. Crit Care Med1993;21:1096 (see editorial by Gay PC. Of pundits, professors, and pollsters. Crit Care Med 1993;21:959–60).
      OpenUrl
    49. ↵
      Ontario Intensive Care Study Group. Evaluation of right heart catheterization in critically ill patients. Crit Care Med1992;20:928–33.
      OpenUrlPubMedWeb of Science
    50. Wennberg JE. Dealing with medical practice variations: a proposal for action. Health Aff1984;3:6–32.
      OpenUrlFREE Full Text
    51. Wennberg JE. Variations in medical practice and hospital costs. Conn Med1985;49:444–53.
      OpenUrlPubMed
    52. ↵
      Wennberg JE, Gittelsohn A. Small area variation analysis in health care delivery. Science1973;142:1102–8.
    53. ↵
      Hulley S, Cummings S. Designing clinical research. Baltimore: Williams and Wilkins, 1988.
    54. ↵
      Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med1998;338:232–8.
      OpenUrlCrossRefPubMedWeb of Science
    55. ↵
      Morris A, Wallace C, Menlove R, et al. Randomized clinical trial of pressure-controlled inverse ratio ventilation and extracorporeal CO2 removal for ARDS. Am J Respir Crit Care Med 1994;149:295–305 (erratum 1994;149:838; letters to the editor 1995;151:255–6, 1995;151:1269–70, and 1997;156:1016–7).
      OpenUrlCrossRefPubMedWeb of Science
    56. ↵
      East TD, Böhm SH, Wallace CJ, et al. A successful computerized protocol for clinical management of pressure control inverse ratio ventilation in ARDS patients. Chest1992;101:697–710.
      OpenUrlCrossRefPubMedWeb of Science
    57. East T. Role of the computer in the delivery of mechanical ventilation. In: Tobin M, ed. Principles and practice of mechanical ventilation. New York: McGraw-Hill, 1994: 1005–38.
    58. ↵
      Classen DC, Evans RS, Pestotnik SL, et al. The timing of prophylactic administration of antibiotics and the risk of surgical wound infection. N Engl J Med1992;326:281–6.
      OpenUrlCrossRefPubMedWeb of Science
    59. ↵
      Pestotnik S, Classen D, Evans R, et al. Implementing antibiotic practice guidelines through computer-assisted decision support: clinical and financial outcomes. Ann Intern Med1996;124:884–90.
      OpenUrlCrossRefPubMedWeb of Science
    60. ↵
      Classen DC, Pestotnik SL, Evans RS, et al. Adverse drug events in hospitalized patients: exess length of stay, extra costs, and attributable mortality. JAMA1997;277:301–6.
      OpenUrlCrossRefPubMedWeb of Science
    61. ↵
      East T, Heermann L, Bradshaw R, et al. Efficacy of computerized decision support for mechanical ventilation: results of a prospective multi-center randomized trial. Proc AMIA Symp 1999: 251–5.
    62. ↵
      Wirtschafter DD, Scalise M, Henke C, et al. Do information systems improve the quality of clinical research? Results of a randomized trial in a cooperative multi-institutional cancer group. Comput Biomed Res1981;14:78–90.
      OpenUrlCrossRefPubMedWeb of Science
    63. ↵
      Dawes R, Faust D, Meehl P. Clinical versus actuarial judgement. Science1989;243:1668–74.
      OpenUrlAbstract/FREE Full Text
    64. ↵
      Miller R, Goodman K. Ethical challenges in the use of decision-support software in clinical practice. In: Goodman K, ed. Ethics, computing, and medicine: informatics and the transformation of health care. Cambridge, UK: Cambridge Univeristy Press. 1998: 102–15.
    65. ↵
      Grimshaw J, Russell I. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet1993;342:1317–22.
      OpenUrlCrossRefPubMedWeb of Science
    66. ↵
      Grimm RH, Shimoni K, Harlan WR, et al. Evaluation of patient-care protocol use by various providers. N Engl J Med1975;292:507–11.
      OpenUrlPubMedWeb of Science
    67. Safran C, Rind D, Davis R, et al. Effects of a knowledge-based electronic patient record on adherence to practice guidelines. MD Comput1996;13:55–63.
      OpenUrlPubMed
    68. ↵
      Johnston M, Langton K, Haynes B, et al. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. Ann Intern Med1994;120:135–42.
      OpenUrlCrossRefPubMedWeb of Science
    69. ↵
      Hirsh J. Oral anticoagulant drugs. N Engl J Med1991;324:1865–75.
      OpenUrlPubMedWeb of Science
    70. ↵
      Blesser W. A systems approach to biomedicine. New York: McGraw-Hill, 1969.
    71. ↵
      McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med1976;295:1351–5.
      OpenUrlCrossRefPubMedWeb of Science
    72. Dexter PR, Perkins S, Overhage JM, et al. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med2001;345:965–70.
      OpenUrlCrossRefPubMedWeb of Science
    73. ↵
      Kuperman GJ, Teich JM, Gandhi TK, et al. Patient safety and computerized medication ordering at Brigham and Women's Hospital. Jt Comm J Qual Improv2001;27:509–21.
      OpenUrlPubMed
    74. ↵
      Morris A. Evaluating and refining a hemodynamic protocol for use in a multicenter ARDS clincial trial. Am J Respir Crit Care Med2000;161(Suppl 3):A378.
      OpenUrl
    75. ↵
      Evans RS, Burke JP, Pestotnik SL, et al. Prediction of hospital infections and selection of antibiotics using an automated hospital database. In: Proceedings of 14th Annual Symposium on Computer Applications in Medical Care, Baltimore, MD. Los Alamitos, CA: IEEE Computer Soc Press, 1990: 663–7.
    76. ↵
      Evans RS, Pestotnik SL, Burke JP, et al. Reducing the duration of prophylactic antibiotics use through computer monitoring of surgical patients. Ann Pharmacother1990;24:351–4.
      OpenUrlPubMed
    77. ↵
      Evans R, Classen D, Pestotnik S, et al. A decision support tool for antibiotic therapy. In: Safran C, ed. Proceedings of 19th Annual Symposium on Computer Applications in Medical Care (SCAMC), New Orleans. New York: McGraw-Hill, 1995: 651–5.
    78. ↵
      McDonald CJ. Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med1976;84:162–7.
    79. Tierney W, McDonald C, Martin D, et al. Computerized display of past test results. Ann Intern Med1987;107:569–74.
    80. McDonald C, Tierney W. Computer-stored medical records: their future role in medical practice. JAMA1988;259:3433–40.
      OpenUrlCrossRefPubMedWeb of Science
    81. Tierney W, McDonald C, Hui S, et al. Computer predictions of abnormal test results: effects on outpatient testing. JAMA1988;259:1194–8.
      OpenUrlCrossRefPubMedWeb of Science
    82. Tierney W, Miller M, McDonald C. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med1990;322:1499–504.
      OpenUrlCrossRefPubMedWeb of Science
    83. Tierney W, Miller M, Overhage J, et al. Physician inpatient order writing on microcomputer work stations. JAMA1993;269:379–83.
      OpenUrlCrossRefPubMedWeb of Science
    84. Pestotnik SL, Evans RS, Burke JP, et al. Development of a computerized infectious disease monitor (CIDM). Comput Biomed Res1990;18:103–13.
    85. Evans RS, Burke JP, Classen DC, et al. Computerized identification of patients at high risk of hospital-acquired infection. Am J Infect Control1992;20:4–10.
      OpenUrlCrossRefPubMedWeb of Science
    86. Classen DC, Pestotnik SL, Evans RS, et al. Intensive surveillance of midazolam use in hospitalized patients and the occurrence of cardiordespiratory arrest. Pharmacotherapy1992;12:213–6.
      OpenUrlPubMedWeb of Science
    87. Evans R, Pestotnik S, Classen D, et al. Prevention of adverse drug events through computerized surveillance. In: Proceedings of 16th Annual Symposium on Computer Applications in Medical Care, Baltimore, MD. Los Alamitos, CA: IEEE Computer Soc Press, 1992: 437–41.
    88. ↵
      Tierney WM, Overhage JM, Takesue BY, et al. Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Inform Assoc1995;2:316–22.
      OpenUrlAbstract/FREE Full Text
    89. ↵
      McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med1996;124:56–62.
    90. ↵
      Tierney WM, Overhage JM, McDonald CJ. Computerizing guidelines: factors for success. In: Cimino J, ed. Proceedings of 1996 AMIA Annual Fall Symposium (formerly SCAMC). Washington: Hanley & Belfus, 1996: 459–62.
    91. Audet A-M, Greenfield S, Field M. Medical practice guidelines: current activities and future directions. Ann Intern Med1990;113:709–14.
    92. Fletcher R, Fletcher S. Clinical practice guidelines. Ann Intern Med1990;113:645–6.
    93. Hadorn D, McCormick K, Diokno A. An annotated algorithm approach to clinical guideline development. JAMA1992;267:3311–4.
      OpenUrlCrossRefPubMedWeb of Science
    94. Miller P, Frawly S. Trade-offs in producing patient-specific recommendations from a computer-based clinical guideline: a case study. J Am Med Inform Assoc1995;2:238–42.
      OpenUrlAbstract/FREE Full Text
    95. ↵
      Fridsma D, Gennari J, Musen M. Making generic guidelines site-specific. In: Cimino J, ed. Proceedings of 1996 AMIA Annual Fall Symposium (formerly SCAMC). Washington: Hanley & Belfus, 1996: 597–601.
    96. ↵
      Field M, Lohr K, eds. Clinical practice guidelines: directions for a new program (summary). Washington: National Academy Press, 1990.
    97. ↵
      Armstrong R, Bullen C, Cohen S, et al. Critical care algorithms. New York: Oxford University Press, 1991.
    98. Don H, ed. Decision making in critical care. Philadelphia: B C Decker Inc, 1985.
    99. ↵
      Karlinsky J, Lau J, Goldstein R. Decision making in pulmonary medicine. Philadelphia: B C Decker, 1991.
    100. ↵
      Altman I, Rogoff B. World views in psychology: trait, interactional, organismic, and transactional perspectives. In: Stokols D, Altman I, eds. Handbook of environmental psychology. New York: John Wiley & Sons, 1987: 7–40.
    101. ↵
      Werner C, Brown B, Altman I. Planning and doing transactionally-oriented research: Examples and strategies. In: Stokols D, Altman I, eds. Handbook of environmental psychology II. New York: John Wiley & Sons, 2002 (in press).
    102. ↵
      Tversky A, Kahneman D. Availability: a heuristic for judging frequency and probability. In: Kahneman D, Slovic P, Tversky A, eds. Judgment under uncertainty: heuristics and biases. Cambridge, UK: Cambridge University Press, 1982: 163–78.
    103. ↵
      Jennings D, Amabile T, Ross L. Informal covariation assessment: data-based versus theory-based judgments. In: Kahneman D, Slovic P, Tversky A, eds. Judgment under uncertainty: heuristics and biases. Cambridge, UK: Cambridge University Press, 1982: 211–30.
    104. ↵
      Guidelines Committee, Society of Critical Care Medicine. Guidelines for the care of patients with hemodynamic instability associated with sepsis. Crit Care Med1992;20:1057–9.
      OpenUrlPubMedWeb of Science
    105. ↵
      Pocock SJ. Clinical trials: a practical approach. New York: John Wiley & Sons, 1983.
    106. ↵
      Nelson E, Splaine M, Batalden P, et al. Building measurement and data collection into medical practice. Ann Intern Med1998;128:460–6.
      OpenUrlCrossRefPubMedWeb of Science
    107. Schacker T, Collier AC, Hughes J, et al. Clinical and epidemiologic features of primary HIV infection. Ann Intern Med1996;125:257–64 (erratum 1997;126:174).
      OpenUrlCrossRefPubMedWeb of Science
    108. Kiernan M, King AC, Kraemer HC, et al. Characteristics of successful and unsuccessful dieters: an application of signal detection methodology. Ann Behav Med1998;20:1–6.
      OpenUrlCrossRefPubMedWeb of Science
    109. Galuska DA, Will JC, Serdula MK, et al. Are health care professionals advising obese patients to lose weight? JAMA1999;282:1576–8.
      OpenUrlCrossRefPubMedWeb of Science
    110. Dickerson JE, Hingorani AD, Ashby MJ, et al. Optimisation of antihypertensive treatment by crossover rotation of four major classes. Lancet1999;353:2008–13.
      OpenUrlCrossRefPubMedWeb of Science
    111. Avorn J. Putting adverse drug events into perspective. JAMA1997;277:341–2.
      OpenUrlCrossRefPubMedWeb of Science
    112. Bates D, Spell, N, Cullen D, et al. The costs of adverse drug events in hospitalized patients. JAMA1997;277:307–11.
      OpenUrlCrossRefPubMedWeb of Science
    113. Lesar T, Briceland L, Stein D. Factors related to error in medication prescribing. JAMA1997;277:312–7.
      OpenUrlCrossRefPubMedWeb of Science
    114. ↵
      Johnson J, Bootman J. Drug-related morbidity and mortality; a cost of illness model. Arch Intern Med1995;155:1949–56.
      OpenUrlCrossRefPubMedWeb of Science
    115. ↵
      Kiefe CI, Allison JJ, Williams OD, et al. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA2001;285:2871–9.
      OpenUrlCrossRefPubMedWeb of Science
    116. ↵
      Henderson S, Crapo R, Wallace C, et al. Performance of computerized protocols for the management of arterial oxygenation in an intensive care unit. Int J Clin Monit Comput1992;8:271–80.
    117. ↵
      Cochrane G. Compliance in asthma. Eur Respir Rev1998;8:239–42.
    118. ↵
      Jones P. Health status, quality of life and compliance. Eur Respir Rev1998;8:243–6.
      OpenUrl
    119. ↵
      Lomas J, Anderson GM, Domnick-Pierre K, et al. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med1989;321:1306–11.
      OpenUrlCrossRefPubMedWeb of Science
    120. ↵
      Greco P, Eisenberg J. Changing physicians' practices. N Engl J Med1993;329:1271–4.
      OpenUrlCrossRefPubMedWeb of Science
    121. ↵
      Singer M, Haft R, Barlam T, et al. Vancomycin control measures at a tertiary-care hospital: impact of interventions on volume and patterns of use. Infect Control Hosp Epidemiol1998;19:248–53.
      OpenUrlPubMedWeb of Science
    122. Pritchard RS, Fisher ES, Teno JM, et al. Influence of patient preferences and local health system characteristics on the place of death. Study to Understand Prognoses and Preferences for Risks and Outcomes of Treatment (SUPPORT) Investigators. J Am Geriatr Soc1998;46:1242–50.
      OpenUrlPubMedWeb of Science
    123. Teno J, Lynn J, Connors AFJ, et al. The illusion of end-of-life resource savings with advance directives. Study to Understand Prognoses and Preferences for Risks and Outcomes of Treatment (SUPPORT) Investigators. J Am Geriatr Soc1997;45:513–8.
      OpenUrlPubMedWeb of Science
    124. Teno JM, Licks S, Lynn J, et al. Do advance directives provide instructions that direct care? Study to Understand Prognoses and Preferences for Risks and Outcomes of Treatment (SUPPORT) Investigators. J Am Geriatr Soc1997;45:508–12.
      OpenUrlPubMedWeb of Science
    125. ↵
      Lo B. Improving care near the end of life: why is it so hard? JAMA1995;274:1634–6.
      OpenUrlCrossRefPubMedWeb of Science
    126. ↵
      Weed L. Physicians of the future. N Engl J Med1981;304:903–7.
      OpenUrlPubMedWeb of Science
    127. Weed L. New connections between medical knowledge and patient care. BMJ1997;315:231–5.
      OpenUrlFREE Full Text
    128. Weed L. Reengineering medicine: questions and answers. Fed Bull1995;82:24–36.
      OpenUrl
    129. Weed L. Reengineering medicine. Fed Bull1994;81:149–83.
      OpenUrl
    130. Burgher C. The use of problem knowledge couplers in a primary care practice. Healthc Inform Manage1997;11:13–26.
    131. ↵
      Weed L, Weed L. Opening the black box of clinical judgment. BMJ (eBMJ edition)1999;319:1–32, 1–11, 1–11, 1–12, 1–8.
      OpenUrlFREE Full Text
    132. ↵
      Christensen C, Bohmer B, Kenagy J. Will disruptive innovations cure health care? Harvard Business Rev2000;78:102–12.
      OpenUrl
    133. ↵
      Warner H, Olmsted C, Rutherford B. HELP: a program for medical decision-making. Comput Biomed Res1972;5:65–74.
      OpenUrlCrossRefPubMedWeb of Science
    134. Hulse R, Clark S, Jackson J, et al. Computerized medication monitoring system. Am J Hosp Pharm1976;33:1061–4.
      OpenUrlAbstract
    135. Gardner RM, West BJ, Pryor TA, et al. Computer-based ICU data acquisition as an aid to clinical decision-making. Crit Care Med1982;10:823–30.
      OpenUrlPubMedWeb of Science
    136. Pryor TA. The HELP medical record system. MD Computing1988;5:22–33.
    137. ↵
      Kuperman GJ, Garder RM, Pryor TA. HELP: a dynamic hospital information system. New York: Springer-Verlag, 1991.
    138. ↵
      Morris AH, Gardner RM. Computer applications. In: Hall J, Schmidt G, Wood L, eds. Principles of critical care. New York: McGraw-Hill, 1992: 500–14.
    139. ↵
      East T, Morris A, Gardner R. Computerized management of mechanical ventilation. In: Grenvik A, Ayres S, eds. Textbook of critical care. 3rd ed. Philadelphia: W B Saunders, 1995: 895–911.
    140. Morris A. Protocol management of ARDS. New Horizons1993;1:593–602.
      OpenUrlPubMed
    141. ↵
      Morris A. Rational use of computerized protocols in the ICU. Crit Care2001;5:249–54.
      OpenUrlCrossRefPubMedWeb of Science
    142. ↵
      Clemmer TP, Spuhler VJ, Berwick DM, et al. Cooperation: the foundation of improvement. Ann Intern Med1998;128:1004–9.
    143. Clemmer TP, Spuhler VJ, Oniki TA, et al. Results of a collaborative quality improvement program on outcomes and costs in a tertiary critical care unit. Crit Care Med1999;27:1768–74.
      OpenUrlCrossRefPubMedWeb of Science
    144. Oniki TA, Clemmer TP, Arthur LK, et al. Using statistical quality control techniques to monitor blood glucose levels. Proc Annu Symp Comput Appl Med Care 1995:586–90.
    145. ↵
      Clemmer TP. Nutrition support teams: role in the new health care environment. Nutr Clin Pract1994;9:217–20.
      OpenUrlAbstract/FREE Full Text
    146. ↵
      Staender S, Davies J, Helmreich B, et al. The anaesthesia critical incident reporting system: an experience based database. Int J Med Inf1997;47:87–90.
      OpenUrlCrossRefPubMedWeb of Science
    147. ↵
      Galletly DC, Mushet NN. Anaesthesia system errors. Anaesth Intensive Care1991;19:66–73.
      OpenUrlPubMedWeb of Science
    148. ↵
      Joint Commission on Accreditation of Healthcare Organizations. What every hospital should know about sentinel events. Oakbrook Terrace, IL: Joint Commission on Accreditation of Healthcare Organizations, 2000.
    149. ↵
      Joint Commission on Accreditation of Healthcare Organizations. Root cause analysis in health care: tools and techniques. Oakbrook Terrace, IL: Joint Commission on Accreditation of Healthcare Organizations, 2000.
    150. ↵
      Miller J. Living systems. New York: McGraw-Hill, 1978.
    151. ↵
      Mandelbrot B. The fractal geometry of nature. New York: W H Freeman, 1983.
    152. ↵
      Schultz S. Homeostasis, humpty dumpty, and integrative biology. News Physiol Sci1006;11:238–46.
      OpenUrl
    153. ↵
      Imai M. Kaizen: the key to Japan's competitive success. New York: McGraw-Hill, 1986.
    154. ↵
      Prochaska JO, DiClemente CC, Norcross JC. In search of how people change. Applications to addictive behaviors. Am Psychol1992;47:1102–14.
      OpenUrlCrossRefPubMedWeb of Science
    155. ↵
      Rogers EM. Diffusion of innovations. 4th ed. New York: Free Press, 1995.

    Footnotes

    • Supported by the NIH (RO1-HL-36787, NO1-HR-46062), AHCPR (HS 06594), Deseret Foundation, Respiratory Distress Syndrome Foundation, LDS Hospital, and IHC Inc.