Abstract
BACKGROUND: We assessed the impact of clinical decision support (CDS) rules within the electronic health record for ionized calcium (iCa), serum magnesium (Mg), and N-terminal pro-B-type natriuretic peptide (NT-proBNP) in intensive care unit (ICU) inpatients at a large academic center.
METHODS: A repeat order for measurement of iCa or Mg placed within 24 (iCa) or 48 (Mg) h of a previously nonactionable result, or additional orders for NT-proBNP beyond 1 within a single hospitalization, triggered a CDS pop-up alert showing the prior result and offering the opportunity to cancel the order or to place the order after entering an indication for repeat testing. The number of tests performed for each of these analytes and incidence of adverse clinical outcomes potentially associated with hypocalcemia or hypomagnesemia were compared between the 90-day period before CDS implementation and two 90-day periods immediately following.
RESULTS: iCa test volumes decreased by 48%, Mg by 39%, and NT-proBNP by 28% in the 90-day period immediately following implementation and remained decreased by 54%, 49%, and 22%, respectively, during the following 90-day period (all P values <0.0002). Adverse clinical outcomes potentially associated with hypocalcemia or hypomagnesemia did not increase (all P-values >0.17).
CONCLUSIONS: Implementation of CDS dramatically decreased repeat testing of iCa, Mg, and NT-proBNP without adversely impacting clinical outcomes in the ICU. Expansion of the rules from the ICU units to include the entire hospitalized patient population and expansion to additional analytes is expected to lead to further reductions in testing.
As healthcare in the US changes from a fee-for-service model increasingly toward value-based payments, strategies to avoid unnecessary laboratory tests will become progressively more important. A recent metaanalysis found the overall mean rates of laboratory over- and underutilization to be 20.6% and 44.8%, respectively (1). In addition to the cost of the test itself, overutilization also leads to increased phlebotomy, which can lead to patient discomfort and hospital-acquired anemia. In turn, hospital-acquired anemia has been associated with increased blood transfusions, length of stay, and mortality (2–6). Furthermore, the results of laboratory tests must be reviewed by the clinician and can trigger additional downstream tests and interventions, each of which carries costs and risks. A variety of strategies aimed at reducing overall healthcare costs without compromising patient outcomes have been investigated in recent years. One area of increasing interest is in the development of laboratory utilization management programs.
At our institution, ionized calcium (iCa),4 serum magnesium (Mg), and N-terminal pro-B-type natriuretic peptide (NT-proBNP) are often ordered several times per inpatient admission. Although hypocalcemia is common in septic and critically ill patients, it may result from a shift of calcium to the intracellular compartment and indicate disease severity without directly contributing to patient outcome (7). Furthermore, in critically ill patients with iCa values over 4.0 mg/dL (1.0 mmol/L), supplementation has not been shown to improve outcome (8). Repeat testing of iCa therefore may lead to more unnecessary supplementation, which leads to more repeat testing.
Total serum magnesium (Mg) is measured commonly as an estimate of magnesium status because the physiologically active form of magnesium, Mg2+, is difficult to measure. However, total Mg and Mg2+ are not highly correlated with total body and tissue magnesium (9–12). Although hypomagnesemia is not uncommon at the time of intensive care unit (ICU) admission, previous studies have demonstrated a lack of signs and symptoms until Mg is below 1.2 mg/dL (0.5 mmol/L) (9, 12).
NT-proBNP is a useful marker in the diagnosis of heart failure in clinical scenarios for which the diagnosis is unclear, and NT-proBNP concentrations also predict cardiovascular morbidity and mortality in heart failure (13, 14). However, owing to shifts in fluid status and wide biological variability, serial or repeat measurements during hospitalization are of questionable clinical utility and not recommended (15, 16). To avoid confusion or misinterpretation of fluctuating NT-proBNP concentrations during hospitalization, measurement once at admission (to confirm diagnosis of heart failure if unclear) and once before discharge may be optimal (17).
Efforts to reduce redundant ordering for iCa, Mg, and NT-proBNP have been successful at other institutions, although the strategies employed have differed (18–20). Here we describe a quality improvement initiative at the Mayo Clinic aimed at reducing repeat testing of iCa, Mg, and NT-proBNP among hospital inpatients in 5 ICU areas using “soft” stop clinical decision support (CDS) rules.
Materials and Methods
This quality improvement initiative was implemented at Mayo Clinic Hospital, Rochester, MN, and included only adult patients in 5 ICUs. The ICUs included were cardiovascular surgical, medical, trauma/general surgical, vascular/thoracic surgical, and medical/surgical/transplant, for a total of 114 beds. Residents and fellows place the vast majority of laboratory orders for patients in the ICU, although other providers (attending physicians, nurse practitioners, physician assistants) may also place orders. The Mayo Clinic in Rochester, MN, is a tertiary referral center with 2 hospital campuses, the Methodist Campus and Saint Marys Campus.
CDS was implemented for iCa, Mg, and NT-proBNP in the 5 ICU areas. Specifically, a middleware system (Blaze) was used to create an order entry pop-up alert in the electronic medical record system (MICS Lastword, GE Healthcare) when a healthcare provider placed an order for 1 of the 3 analytes within a set time limit. The CDS rule triggered for these analytes when ordered with a frequency of “once,” regardless of whether the order was placed within an order set, individually, or through one of many ordering short-cut tools. The iCa alert was triggered if a second order was placed within 24 h of a prior test with a result of ≥4.8 mg/dL (1.2 mmol/L) for iCa for an individual patient [reference interval for iCa, 4.8–5.7 mg/dL (1.2–1.42 mmol/L)]. These limits were determined by analyzing sets of repeat iCa values for all inpatients over a 3-month period. There were 10428 total iCa tests performed over this period, of which 2206 were repeated within 24 h of a previous iCa value within the reference interval (≥4.8 mg/dL or 1.2 mmol/L). In only 3 of the 2206 instances was the repeat value <4.0 mg/dL (1.0 mmol/L) (an actionable value based upon electrolyte replacement protocol), and all 3 patients had indications (renal failure or dialysis, abnormal heart rhythm, or massive transfusion or apheresis) for repeat measurement. Indications for repeat measurement were determined from communication with pharmacy and critical care providers. Thus, we determined that implementing a rule preventing repeat iCa measurement within 24 h of a previous value ≥4.8 mg/dL (1.2 mmol/L), except for the indications identified, would reduce iCa inpatient utilization without adversely impacting patient care.
In a manner analogous to that described for iCa, we determined that over 3 months there were 8828 orders for repeat serum Mg (reference interval 1.7–2.3 mg/dL or 0.7–0.9 mmol/L) requested within 48 h of a previous Mg value ≥1.4 mg/dL (the actionable value for intravenous Mg replacement). In only 33 instances was the repeat value actionable (Mg <1.4 mg/dL [0.6 mmol/L]), again with almost all instances meeting 1 of the defined indications for repeat measurement (renal failure or dialysis, abnormal heart rhythm, hypocalcemia, or digoxin or diuretic use).
The NT-proBNP alert was triggered any time the test was ordered on the same patient more than once during a 14-day hospital length of stay. Ordering providers were asked to specify an indication for the repeat NT-proBNP measurement. Indications for repeat NT-proBNP were major intervention or change in therapy, rule-out of heart failure, or an order at the time of hospital discharge. For all 3 CDS alerts, ordering providers could answer “Yes” to manually cancel the test or “No” to proceed with ordering (for an example of a pop-up alert see Fig. 1 in the Data Supplement that accompanies the online version of this report at http://www.clinchem.org/content/vol62/issue6). A duplicate order was allowed only if the ordering provider answered “No” and provided the indication for repeat measurement in a free text field. Educational materials related to the intervention consisted of a memo explaining the rationale for the intervention, and a training guide demonstrating pop-up functionality, which were emailed to ICU medical directors, nurse managers, and program directors of residency and fellowship programs with primary practices in the ICUs. There was no direct communication to the primary ordering providers (residents and fellows), although program directors were asked to share the information with all ordering providers.
Data were collected from both Mayo Clinic electronic medical records and laboratory information systems. The number of tests performed for each analyte for patients in the 5 ICU areas was collected for the 90-day period before implementation of the CDS rules, as well as the next two 90-day periods immediately following the intervention. We collected data over two 90-day periods postintervention to assess what impact the change of academic year (July) would have on the effectiveness of the CDS rules. The number of tests performed during each time period was compared as a rate of tests per discharge for each of the 3 time periods. The cost savings were calculated by multiplying the laboratory direct cost to perform each test by the number of tests of each type avoided, which was calculated by taking the difference between the tests ordered at baseline and during each intervention period. To adjust for fluctuations in the numbers of ICU patients, which would impact the number of tests performed, we normalized test volumes to the total number of ICU discharges from the areas for each time period. There were 1924 discharges from the ICU areas studied during the preintervention period, and there were 1997 and 2283 ICU discharges from these areas during the 2 postintervention periods. In addition, ICU test volumes for other routinely ordered tests not targeted by this intervention were obtained for comparison. These test volumes included an electrolyte panel (ELPN), phosphorus (Phos), aspartate aminotransferase (AST), and alkaline phosphatase (Alk Phos). Differences in the test ordering rates were assessed using generalized estimating equations with a Poisson distribution and a log link, with the number of discharges as the offset for each time period.
To evaluate possible adverse effects from hypocalcemia and hypomagnesemia that may have gone undetected and untreated because of decreased testing, we searched the electronic medical record for International Classification of Diseases Ninth Revision (ICD-9) codes related to potentially electrolyte-related conditions acquired during ICU stays. ICD-9 codes queried included cardiac arrest (ICD-9 427.5), ventricular fibrillation (427.41), tachycardia (785.0), hypocalcemia (275.41), tetany (781.7), and seizures not associated with epilepsy or stroke (780.39). Differences in the adverse events were assessed using generalized estimating equations with a Poisson distribution and a log link, with the number of discharges as the offset for each time period. Data were analyzed using SAS 9.4.
Results
The CDS rules for iCa, Mg, and NT-proBNP were implemented on April 24, 2014, in the 5 ICU areas. Compared to the 90-day period immediately before implementation of the tool (01/24/2014–04/23/2014), in the 90-day period immediately after implementation (04/25/2014–07/23/2014) iCa test volumes for patients in the ICU areas decreased by 48%, Mg by 39%, and NT-proBNP by 28% (Table 1). This decrease in testing was maintained through the next 90-day period (07/24/2014–10/25/2014). Testing volumes for other routine tests that were not included in the intervention—electrolyte panel, Phos, AST, and Alk Phos—remained within ±10% of preintervention rates during the first 90-day postintervention period, with the exception of Phos, which decreased by 30% after CDS rule implementation (Table 1).
Tests performed per ICU discharge in pilot ICUs by time period.a
Fig. 1 shows the total number of tests performed on patients in the pilot ICU areas (not normalized to ICU discharges) for iCa, Mg, Phos, electrolyte panel, AST, and Alk Phos by 30-day periods from 01/25/2014 (90 days before intervention) to 10/25/2014 (180 days after intervention). The CDS intervention led to immediate and sustained decreases in tests performed for iCa, Mg, and Phos. Total testing volumes for other tests not subject to the intervention (electrolyte panel, AST, Alk Phos) remained stable or slightly increased (Fig. 1). Volumes for most tests showed a slight increase during the last 30-day period (09/25/14–10/25/14); however, there were also more ICU discharges during this period.
The preintervention period was January 24 to April 23, 2014, the immediate postintervention period was April 25 to July 23, 2014, and the late postintervention period was July 24 to October 25, 2014.
A total of 6110 alerts (516 for iCa, 5160 for Mg, and 434 for NT-proBNP) were generated in the 6-month period after the intervention, with 1 or more tests ordered in 4017 (66%) of those instances. Of the override requests, 3513 (88%) were for serum Mg, 211 (5%) were for iCa, and 293 (7%) were for NT-proBNP. The most common indications for Mg repeat orders were renal failure/diuretic use (28%), arrhythmia/heart rhythm abnormality (18%), monitoring Mg during replacement therapy (17%), and other (37%). The monthly volume of pop-up alerts decreased during the first 3 months of the intervention (Table 2). Pop-up volumes subsequently increased, and override rates increased, during the final 2–3 months of the intervention (Table 2).
Number of pop-up alerts and pop-up overrides, along with percentage of alerts overridden, by month of intervention.
In an attempt to identify any signs of possible patient harm from decreased iCa and Mg testing, we investigated whether instances of symptoms or diagnoses associated with these electrolyte disturbances, including arrhythmia, cardiac arrest, tetany, or seizures, were observed with increased frequency in ICU patients after implementation of the CDS rules. No increase in any of these symptoms or diagnoses was identified after the CDS rules were implemented (Fig. 2, all P values >0.17). For the duration of observation, no instances of tetany or electrolyte-related seizures were identified in patients in the ICU areas studied.
When normalized for discharges per time period, none are statistically significant with all P values >0.17.
Using the combined total number of iCa, Mg, and NT-proBNP tests ordered in the pre- and postintervention periods, cost savings were estimated to be $54800 in the 180-day period following the implementation of the CDS rules. Reduction in Mg testing volume was responsible for approximately 65% of total cost reduction, compared to approximately 30% for iCa and approximately 5% for NT-proBNP. This would save an estimated $111122 throughout the course of a year. This does not include the unanticipated significant decrease in phosphorus ordering that occurred in the ICU areas studied nor cost savings from avoided downstream treatment effects associated with the testing. Further reductions in repeat testing and cost savings are anticipated as the CDS rules are expanded throughout additional inpatient care areas.
Discussion
We implemented “soft-stop” CDS rules for iCa, Mg, and NT-proBNP, using pop-up alerts that could be overridden electronically by the ordering provider. We achieved a 48% decrease in iCa, 39% decrease in Mg, and 28% decrease in NT-proBNP testing in the 90-day period immediately after implementation, which was sustained over the subsequent 90-day period. Order entry interventions have been described previously for iCa, Mg, and NT-proBNP (18–20) and other commonly ordered laboratory tests (18–22), although both our methods and results differed from strategies previously described.
One academic medical center decreased iCa orders by approximately 75% by establishing a reflex calcium test, such that iCa was performed only when total serum calcium was <8.0 mg/dL or >10.2 mg/dL. Direct (without the total calcium reflex) iCa orders were still allowed, but only by special request (18). We chose not to pursue this strategy, primarily because iCa is ordered approximately 3 times more often than total calcium in our ICU practice. Thus, introducing a reflex iCa process may have inadvertently increased our total calcium testing. Our results demonstrate that significant improvement (48%–54% reduction) in iCa utilization can be achieved without performing reflex testing.
A CDS rule aimed at reducing repeat BNP ordering in a community hospital resulted in an approximately 21% reduction in BNP testing across all inpatient areas in the 3 months after the CDS rule intervention (19). In that intervention providers were not required to enter a reason or indication for repeat ordering. Despite this, the observed decrease in testing was similar to what we observed with our CDS pop-up alert that required an indication for repeat testing (23%–27% decrease).
Another institution addressed utilization of iCa, Mg, and Phos by creating an order entry pop-up that addressed all 3 tests on the same alert (20). In the first phase of this study, orders placed for Mg tests occurring more than 72 h into the future were flagged, resulting in a 29% decrease in weekly Mg orders. In the second phase, recent magnesium, calcium, and Phos test results were presented to the ordering provider along with education regarding appropriate indications for testing. Testing was also limited to once per order (no future order entry). This resulted in a paradoxical increase in serum Mg orders. Finally, in the third phase they targeted only magnesium, displayed recent results, limited testing to once per order, provided education regarding appropriate indications for testing, and required users to select an indication. This resulted in a 51% decrease in weekly Mg orders. Our intervention also included recent serum Mg results and education on appropriate test use, and required an indication for repeat testing. We achieved a 39%–49% decrease in Mg orders, similar to the results in phase 3 of the aforementioned study, but without limiting future order entry.
The effectiveness of soft-stop CDS rules remains controversial. A recent study of a soft CDS intervention on repeat troponin orders (outside of the defined serial troponin panel) demonstrated no impact on troponin ordering (21). Another study compared a “hard-stop” CDS rule (phone call necessary to override) vs soft-stop rule for duplicate laboratory orders placed on the same day of an inpatient stay. This study found that the soft CDS rule reduced duplicate testing by only 42.6%, compared to 92.3% for the hard stop (22). However this intervention was aimed only at duplicate tests ordered on the same day of service, rather than testing commonly repeated over multiple days of an inpatient stay (22). Because our intervention included specific tests (e.g., iCa and Mg) that require repeat measurement for some patients, a hard-stop approach would not have been appropriate.
We found that soft-stop CDS rules had a significant impact on test ordering in the pilot ICU areas. Although the overall monthly number of tests avoided (approximately 1400) is similar to the monthly volume of CDS pop-up alerts triggered (approximately 1000–1100), the override rate of approximately 66% suggests that the CDS pop-up alert alone was not responsible for the reduction in test volume. The unanticipated decrease in Phos testing during the same time period suggests that implementation of the CDS rules resulted in a change in test ordering patterns in the ICU. The decrease in pop-up alert volumes during the first 3 months of the intervention also suggests that providers changed test ordering patterns to avoid interacting with the CDS pop-up alert box.
There were many more pop-up alerts and overrides for Mg compared to iCa or NT-proBNP, suggesting that providers were less willing to change ordering patterns for Mg. However, the immediate and sustained decrease in Mg testing volumes (Fig. 1) after CDS implementation demonstrates that the Mg CDS rule was still effective. The increase in pop-ups and overrides (predominantly for serum Mg) during the last 3 months of the intervention may have resulted from inadequately sustained educational efforts as new residents and fellows entered the practice, less outcome-based evidence to support a more conservative Mg replacement strategy (compared to calcium replacement), or other factors.
One limitation to our pop-up alert is that indications were entered in a free text field, such that indications not defined in the rule could be used to proceed with testing. Another limitation to our CDS rule is that it did not directly prevent future order entry (e.g., daily Mg or iCa orders for several days). Future or advance order entry for the duration of an ICU stay is very uncommon in our practice. In environments where future order entry is commonly performed at ICU admission for the duration of an ICU stay, a CDS rule that prohibits or addresses future order entry may be needed to achieve similar results. However, the concepts used in our intervention would still apply, with many CDS systems supporting a rule limiting future order entry.
Different outcomes described for soft-stop CDS rules may reflect differences in the nature of the tests targeted (troponin vs electrolytes), differences in the design of the CDS pop-up alert (difficulty to override the screen, requirement for defined indications vs free text field), differences in institutional culture, or other variables. Despite the high override rate of pop-up alerts (predominantly for Mg), significant reductions in testing volumes were maintained throughout the intervention period.
Decreasing the frequency of testing and associated costs does not result in improved quality of care if patients are experiencing adverse effects as a result of the decrease in testing. To ensure that this was not the case in our patient population, we searched for ICD-9 codes associated with arrhythmias, tetany, electrolyte-related seizures, and hypocalcemia. We did not observe any change in the incidence of these conditions nor receive any feedback from clinicians with concerns that this utilization initiative harmed patients.
In summary, we implemented CDS rules with soft stops in 5 ICU areas, to decrease unnecessary repeat testing of iCa, Mg, and NT-proBNP. Our approach resulted in decreased testing of each of these analytes, along with decreased testing costs, and no increase in adverse patient outcomes. In the future, we will expand the number of inpatient care areas with order entry CDS rules for iCa, Mg, and NT-proBNP, as well as create repeat order rules for additional high volume tests.
Footnotes
↵4 Nonstandard abbreviations:
- iCa,
- ionized calcium;
- Mg,
- serum magnesium;
- NT-proBNP,
- N-terminal pro-B-type natriuretic peptide;
- ICU,
- intensive care unit;
- CDS,
- clinical decision support;
- ELPN,
- electrolyte panel;
- Phos,
- phosphorus;
- AST,
- aspartate aminotransferase;
- Alk Phos,
- alkaline phosphatase;
- ICD-9,
- International Classification of Diseases, Ninth Revision.
(see editorial on page 791)
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest:
Employment or Leadership: None declared.
Consultant or Advisory Role: A.S. Jaffe, Beckman-Coulter, Abbott, Alere, Diadexus, Siemens, Radiometer, Trinity, ET Healthcare, theheart.org, Novartis, and Roche.
Stock Ownership: None declared.
Honoraria: None declared.
Research Funding: None declared.
Expert Testimony: None declared.
Patents: None declared.
Role of Sponsor: No sponsor was declared.
- Received for publication October 7, 2015.
- Accepted for publication February 24, 2016.
- © 2016 American Association for Clinical Chemistry