Article Text
PDF
Abstract
Background With the introduction of Computerised Physician Order Entry (CPOE) in routine hospital care, a great deal of effort has been put into refining Clinical Decision Support Systems (CDSS) to identify patients at risk of preventable medication-related harm.
Objectives This study compared a CPOE with basic CDSS and 16 clinical rules with a manual pharmacist medication review to detect overdose and drug–drug interactions that actually required a change in medication.
Methods The study involved the review of 313 patients admitted over 5 months at an internal medicine ward where a change in medication as a result of dosing of therapeutic errors was detected by a manual medication review by a trained pharmacist. Subsequently, all these patients' medication orders (MOs) were entered into the authors' CPOE with basic CDSS. Medication orders with a safety alert indicating overdose and drug–drug interactions generated by the authors' CPOE with basic CDSS were compared with the same type of medication errors identified through manual review. The positive predictive value (PPV), sensitivity and specificity compared with manual review were determined. Second, a set of 16 clinical rules was applied to the patient and prescribing data. The overlap between the clinical rules and manual review was determined by comparing patients triggered by the clinical rule with patients with a corresponding error in the manual medication review.
Results Manual medication review identified 57 medication errors involving overdose and 143 therapeutic errors of which 46 were drug–drug interactions. The CPOE with basic CDDS generated 297 safety alerts involving overdose (PPV 0.06, sensitivity 0.32, specificity 0.92) and 365 safety alerts involving drug–drug interactions (PPV 0.12, sensitivity 0.96, specificity 0.91). The clinical rules generated 313 safety alerts identifying 39% of all the overdoses and therapeutic errors found in the manual review at which they were targeted. In 23% of the alerts generated by a clinical rule, the patients actually required a change of medication as indicated by the manual review. When CPOE with basic CDSS and the rules were combined, 66% of the overdoses and therapeutic errors were identified.
Conclusions The authors' CPOE with basic CDSS and the clinical rules are useful early strategies for preventing medication-related harm. They could be a first step towards more advanced decision support. These computerised systems will be even more useful in daily practice, once they are further fine-tuned to decrease the number of alerts that need no clinical action.
- Medication errors
- clinical decision support
- medication review
- medication safety
- patient safety
- quality of care
Statistics from Altmetric.com
- Medication errors
- clinical decision support
- medication review
- medication safety
- patient safety
- quality of care
Introduction
A substantial proportion of hospitalised patients experience medication-related harm that is preventable—for example due to incorrect dosing, contra-indicated drug choice or drug–drug interactions (DDIs).1–4 Strategies to prevent such problems are being developed. One such strategy is the structured review of patient medication (medication review) by physicians or pharmacists to identify patients with medication errors (MEs) that may lead to harm. In some settings, for example where clinical pharmacists do not routinely participate in ward rounds, this approach may have a retrospective character which implies late intervention, which may be too late to be effective. Moreover, this system is very labour-intensive, since all medication for all patients has to be systematically reviewed. The advantage is that the complete clinical status of each patient is taken into account when identifying problems. A less labour-intensive strategy is the use of computerised trigger systems. These systems can identify patients at risk of medication-related harm (adverse drug events, ADEs) using either data on the prescribed medication alone or the combination of medication with certain patient characteristics or clinical laboratory values.5–9 An example of such a system is the Clinical Decision Support system (CDSS) within Computerised Physician Order Entry (CPOE) systems.10 In The Netherlands, the CDSS integrated into most types of CPOE system is basic; only drug overdose and DDI alerts are generated. For successful identification of high-risk patients, more is required, such as identification of patients at risk of dosing problems in cases of clinical deviation from chemistry parameters or determined blood drug concentrations, or cases where a specific medicine for a specific disease needs to changed.11 ,12 Currently, some hospitals in The Netherlands are developing more advanced support in addition to their basic CDSS by creating defined clinical rules—basically computerised algorithms that look for specific medication orders, patient characteristics and/or laboratory values that identify patients at risk of suboptimal therapy and of medication harm.13 The advantage of such computerised systems is that they limit labour input dramatically. Such systems should be sensitive enough to identify patients at risk, but also specific enough to generate clinically relevant alerts and thus prevent alert fatigue.
This study compared a CPOE with basic CDSS and 16 advanced clinical rules with a manual pharmacist medication review to detect overdose and DDIs that required a change in medication.
Methods
Setting and study population
This study was performed in two general internal medicine wards and one gastroenterology/rheumatology ward at the UMCG. All patients admitted for more than 24 h to these wards were included (313 patients). A waiver from the Medical Ethical Committee was obtained for this study, as the study fell within the boundaries of normal hospital routine for quality improvement. During the study period the system of medication ordering was a conventional paper-based system.
Study design and data collection
A trained research pharmacist (JEvD) visited the ward daily to collect the following data: patient characteristics, medical history, diseases, medication orders (MOs), and laboratory values. Data were extracted from the hospital information system, medical charts and administration charts.
Methods for identifying medication errors or patients at risk
Medication review method to identify medication errors
All MOs were reviewed by a trained research pharmacist (JEvD) with regard to the presence of medication errors (MEs) according to the classification scheme of The Netherlands Association of Hospital Pharmacists14 and considering the complete clinical situation of the patient.
In this study, we included only dosing and therapeutic errors. These errors, if not corrected, have a high probability of leading to medication-related harm2 ,15–18 and are therefore the prime target for CDSS.
CPOE with basic CDSS
All MOs were manually entered into a test environment in our CPOE with basic CDSS, the commercially available Medicator (iSOFT, Leiden, The Netherlands). The Medicator CDS(S) system is basic: safety alerts are generated only for overdoses or DDIs.19 These safety alerts are shown to physicians during the prescribing phase when used in functioning systems. This medication surveillance is based on a national drug database for community pharmacies (the ‘G-standard,’ Z-index BV, The Hague, The Netherlands). After entering MOs into the system, all safety alerts generated were collected, and both MOs and safety alerts (overdose or DDI) were recorded in an SPSS database (version 14; SPSS, Chicago, Illinois).
Computer-based clinical rules
Leiden University Medical Center (LUMC) has developed a computerised alert system that uses clinical rules to detect patients with a potential ADE or who are at risk of an ADE. The system uses data combined from the CPOE, the hospital information system (eg, laboratory values) and the national drug information database (‘G-Standard’) to detect potential patients at risk. Detection is based on defined algorithms, so-called clinical rules. Currently, more than 100 clinical rules have been defined and agreed on by a multidisciplinary team including a pharmacist, a hospital pharmacist, an internal medicine specialist and a clinical pharmacologist. The clinical rules and the computer system have been tested and validated.20 A 5-month pilot study at a general internal medicine ward was performed in the LUMC to compare this new computerised alert system with conventional medication surveillance in their CPOE/CDSS to assess its additional value. Twenty different clinical rules led to an alert in the small patient population admitted to this ward.21
In the current study, which compared this computerised approach with the patients identified as having medication errors, we excluded four rules that were not defined as medication errors in the medication review, resulting in a set of 16 rules (see table 5). A query was designed in MS Access 2003 (Microsoft, Seattle, Washington) for each clinical rule. These queries were applied to the patient data to assess how many patients were triggered by the clinical rules.
Analysis
SPSS version 14 was used for the analysis. Safety alerts generated by CPOE with basic CDSS were compared with overdose or DDI errors detected by the medication review method for all the MOs. The overlap between CDSS and medication review method was analysed by calculating sensitivity, specificity and positive predictive value (PPV) of the support on overdoses and the support on DDIs. The overlap between clinical rules and medication review method was analysed for those patients identified by the clinical rules as being at risk, and limited to patients with an identical medication error. Sensitivity and specificity were not calculated, since patients without an alert and with a related medication error were not included. The overlap was manually reviewed and subsequently analysed by calculating the percentage of patients who were identified as being at risk by both systems. Those patients with an error that corresponded to the related clinical rule were identified only by the medication review method.
Results
The 313 patients making up our study population covered a wide age range of adult patients with diverse clinical conditions, as expected on a general medicine ward. They ranged from young adults with Crohn's disease to the frail elderly with poly-pharmacy (table 1).
Study population
Using the medication review method, 622 dosing errors and 143 therapeutic errors were found. The different types of dosing and therapeutic errors are shown in table 2. The ‘overdose’ and ‘DDI’ subtypes were detected 57 and 46 times respectively.
Frequency of different types of errors: medication review method
In total, 297 overdose safety alerts were generated by our CPOE with basic CDSS. The PPV of this type of support was low (0.06), that is few of the generated safety alerts were indeed indicated as actual overdoses by the medication review method. The sensitivity of the support was higher but still not optimal (0.32). (Table 3)
Computerised Physician Order Entry with basic Clinical Decision Support Systems: support on overdoses
In total, 365 safety alerts on DDI safety alerts were generated by the CPOE with basic CDSS. Although the PPV was low (0.12), the sensitivity of the support was high (0.96; table 4). Almost all DDIs resulted in an alert by the system, but the majority of the problems were not considered as medication errors by the medication review method when other patient data were taken into account.
Computerised Physician Order Entry with basic Clinical Decision Support Systems: support on drug–drug interactions
The set of 16 clinical rules generated a total of 313 alerts, 72 (23%) of which also had one or more related ME identified by the medication review method. These were 78 MEs in total (data not shown). Accordingly, 23% of the alerts identifying patients at risk of medication-related harm actually required follow-up—a change in medication or some other action to prevent an ME. The percentage of patients actually requiring a change in medication for two rules could not be determined because no patients were appropriately triggered, and this percentage was zero for seven clinical rules. For the other clinical rules, this percentage varied between 10 and 58% (table 5). The percentage was highest for the rule ‘use of an opioid and no prescription for a laxative’ (58%). The main focus of the rest of the clinical rules set was to prevent potential therapeutic errors and potential overdoses with relation to reduced renal function. The medication review method found 143 therapeutic errors and 57 overdoses (table 2). The set of 16 clinical rules thus identified 78 MEs—that is 39% of the 143 therapeutic errors and 57 overdoses identified by the medication review. Together, our CPOE with basic CDSS and the clinical rules detected 18 overdoses+44 DDIs+69 clinical rule alerts (excluding rule 14, which triggered patients who had already been detected by basic CDSS)=131 (66%) of the 200 overdose and prescribing errors found using the medication review method.
Selected set of 16 clinical rules
Table 6 provides some examples of why patients found to be at risk of medication harm by the basic CDSS within the CPOE or the clinical rules were not considered to have medication errors according to the medication review method.
Signal with Computerised Physician Order Entry/Clinical Decision Support Systems (CPOE/CDSS) or clinical rule but no medication error in medication review
Discussion
A considerable number of patients identified as at risk of medication-related harm by the two computerised systems, the CPOE with basic CDSS and the clinical rules, were found not to be so using the medication review method. Nevertheless, the sensitivity and specificity of the CPOE with basic CDSS in signalling DDIs were good, despite the low PPV. This study also shows that with a small set of clinical rules, a fair proportion (39%) of the medication errors detected by medication review can be prevented, and when the two systems are combined, this result increases to 66%.
CPOE/CDSS
In their review of medication-related clinical decision support in CPOE systems, Kuperman et al22 showed that CDSS can be divided into two stages: basic support, which covers the basic principles of support such as DDI checking and basic dosing guidance; and more advanced support, which also covers more complex support such as dosing support for susceptible patients or guidance for medication-related laboratory testing. The Medicator CDSS can be considered as basic. Our set of clinical rules are a first step towards more advanced clinical decision support combining basic CDSS (eg, DDIs) and advanced CDSS (eg, providing dosing support for patients with renal insufficiency). Because both our CPOE with basic CDSS and the clinical rules set focus only on a part of the spectrum of medication-related problems, they should be developed further to cover more potential problems. However, it is first important to ensure that the current support is optimised.
Our findings show that this CPOE with a basic CDSS package generates far fewer relevant signals (PPV≤0.12) reporting overdoses or DDIs which do not actually need a subsequent change in medication. Nevertheless, CPOE with basic CDSS missed a considerable number of overdoses (sensitivity=0.32) identified through medication review. One reason for this low sensitivity may be the lack of dosing support for susceptible patients (patients with renal failure or geriatric patients), one of the features of more advanced support systems such as the clinical rules. The low PPV could be explained by the fact that the alerts are based on a database designed for community pharmacies (the ‘G-standard’) rather than for hospital pharmacies. This leads to a number of irrelevant alerts for the hospital setting, such as overdose alerts for doses, which are perfectly acceptable in hospital but not in ambulatory care. To increase the PPV, this database should be further adapted to the hospital setting to prevent alert fatigue in hospital physicians.23
Despite the high sensitivity and specificity of the DDI alerts, many alerts were generated that did not need a subsequent change in medication (low PPV). The challenge is thus to strike an optimal balance between the number of alerts that do not need a follow-up and preserving sufficient sensitivity to catch serious DDIs or overdoses. The most relevant determinant for including an alert should be the severity of the consequences of the overdose or DDI.24 These considerations have led to the development of the clinical rules discussed below.
Clinical rules
In this study, we tested a small set of clinical rules. Overall, the clinical rules meant an improvement in identifying patients at risk and needing an actual change in medication. Whereas only up to 12% of the alerts generated by our CPOE with basic CDSS required a subsequent change in medication, this was 23% of the alerts generated by the clinical rules. When the two were combined, two-thirds of the medication errors were identified.
Like the basic CDSS, some of the signals generated by the clinical rules did not require a subsequent change in medication. For example, rules 1–9 on the use of medication and reduced renal function (table 5) could be made more efficient by incorporating a cut-off dose below which no action and thus no alert are required. Other trigger tools have been developed with the same intention.5–8 25–27 Some of these studies compared their tools with other methods to identify medication errors and ADEs such as manual review or voluntary reports.5 ,7 Others only verified the signals generated on the presence of medication errors or ADEs.6 ,8 ,26 Although these studies are positive in their conclusions, they all showed that additional information usually needs to be collected about the individual patient before the actual need to change medication can be known.
Our study was limited by the fact that the medication review method was performed by only one investigator. However, a strict classification scheme was used to identify medication errors. This scheme precisely distinguished different between subtypes of medication errors and did not allow much room for differences in interpretation. The investigator was extensively trained in using this classification scheme. Another limitation of this study was that the set of clinical rules studied was small (only 16 rules). The majority of these rules focused on support for patients with renal failure and thus covered a narrow therapeutic area. Other studies have assessed more diverse rules, which provide further information about the effect of computerised rules in the field of different therapeutic areas.7 ,27 ,28
Conclusions
We conclude that our CPOE with basic CDSS and the clinical rules are useful early strategies to prevent medication-related harm. They could be a first step towards more advanced decision support. These computerised systems will be even more useful in daily practice when they are further fine-tuned to decrease the number of alerts that require no clinical action. Currently, however, computerised systems should still be combined with a manual review approach to guarantee medication safety.
References
Footnotes
Funding This study was supported by an unconditional grant from The Netherlands Organization for Health Research and Development (ZonMw).
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.