To the Editor:
The scope of external quality assessment (EQA)1 in laboratory medicine has evolved considerably (1). With the increasing worldwide interest in the use of common reference intervals and/or medical-decision limits, modern EQA schemes need to be better at assessing the standardization status of commercial in vitro diagnostic tests. This need has led to new challenges in the design of EQA surveys.
We report the outcomes of a Norwegian pilot study that investigated the use of commutable, fresh-frozen single donations to assess the current standardization status as part of an initiative toward producing common reference intervals (2). The study covered measurements of calcium, magnesium, albumin, and total protein in serum from 20 single-donation blood samples obtained from Solomon Park Research Laboratories. Serum was generated according to CLSI protocol C37-A, with 2 U human thrombin (Sigma-Aldrich) added per milliliter of plasma to facilitate clotting (3); filtration was not used. Aliquots of these samples were sent to laboratories that used the same test systems (instrument, reagent, and calibrator from the same source). Five peer groups (n ≥ 6 laboratories each; N = 47) were considered: Abbott Diagnostics ARCHITECT, Ortho Clinical Diagnostics VITROS, Roche Diagnostics COBAS, Roche Diagnostics MODULAR, and Siemens Healthcare Diagnostics ADVIA. Target values were calculated for each peer group as the mean of the means of duplicate measurements (2 different runs); grossly outlying replicates were detected visually and omitted from the calculations (calcium, 6 replicates; magnesium, 1; albumin, 5; total protein, 2). The peer groups were checked for outliers with the Grubbs test. Finally, the overall values of target analytes (Meanoverall) were calculated as the mean of the peer group means. Note that the Abbott ARCHITECT peer group for magnesium was excluded from the Meanoverall (outlier in Grubbs test). The concentration ranges for Meanoverall covered by the panels were as follows: calcium, 8.36–9.37 mg/dL (2.09–2.34 mmol/L); magnesium, 1.71–2.20 mg/dL (0.70–0.91 mmol/L); albumin, 3.56–4.58 g/dL; total protein, 5.34–7.13 g/dL. System comparability was assessed by correlation, linear regression, and differences of peer group means from the Meanoverall. The rationale for using this approach was the assumption that globally operating manufacturers have standardized their systems by means of a trueness-based reference measurement system (4).
Table 1 summarizes the regression, correlation, and difference data obtained in the study. Judged against the limits for bias derived from biological variation (http://www.westgard.com/biodatabase-2012-update.htm), the comparability of the systems was excellent for total protein [difference range, −0.8% (Roche COBAS) to 1% (Siemens ADVIA); bias limit from biology, 1.2%], moderate for calcium [difference range, −2.6% (Roche MODULAR) to 2.1% (Siemens ADVIA); bias limit from biology, 0.8%], and problematic for albumin [difference range, −7.4% (Abbott ARCHITECT) to 5.1% (Roche MODULAR); bias limit from biology, 1.3%]. The albumin difference was concentration dependent for the Ortho VITROS system (y = 1.17x − 0.76 g/dL. 95% CIs: slope, 1.10–1.24; y intercept, −1.05 g/dL to −0.46 g/dL) and for the Siemens ADVIA system (y = 0.86x + 0.64 g/dL. 95% CIs: slope, 0.76–0.97; y intercept, 0.21–1.07 g/dL). For magnesium, 4 systems compared well [difference range, −1.5% (Roche COBAS) to 2.4% (Siemens ADVIA)], whereas the Abbott ARCHITECT test showed a difference of 11.2% (bias limit from biology, 1.8%). Note that the Roche COBAS difference is dependent on the magnesium concentration, amounting to −2.5% at 2.20 mg/dL [y = 0.93x + 0.10 mg/dL (0.041 mmol/L). 95% CIs: slope, 0.90–0.97; y intercept, 0.04–0.16 mg/dL (0.014–0.067 mmol/L)]. Although the systems generally had good run-to-run performance (median CV, 1.2%), the study findings suggest some additional concerns. The dispersion of the residuals around the Meanoverall was considerably higher for the Abbott ARCHITECT magnesium test than for the others (r = 0.953; others, r > 0.995). The Siemens ADVIA peer group for magnesium showed relatively high imprecision (CV, 3.3%; CVs for the others, <1.6%). The differences for the replicate outliers obtained in the calcium tests (>6%) were considerably higher than the biological total error limit of 2.4%. This difference may provide incentive for further optimization of calcium tests in general. A limitation of the study was that the target values were not determined by reference measurement procedures, but such information would have had no effect on the differences between methods, which were our primary focus. In view of our results, we emphasize the need for restandardizing the Abbott ARCHITECT magnesium test and all of the albumin tests.
In summary, this study revealed excellent comparability of the total-protein measurements. For calcium, the study demonstrated the need for tighter control of lot variation and suggested the need to further optimize this test. The results of this study reveal that the magnesium test for the Abbott ARCHITECT system requires improved quality and restandardization; the results also reiterate an urgent need for standardization of albumin tests (5). In conclusion, EQA surveys with single-donation samples, peer group formation according to test system, and Meanoverall values for targets can serve as valuable benchmarks for the assessment of system quality and comparability, and they can provide valuable incentives for improvement. Problems with standardization have to be resolved by comparison with reference measurement procedures, be it on an international scale (albumin) or on the scale of the individual manufacturer (magnesium).
↵1 Nonstandard abbreviations:
- external quality assessment;
- overall value of target analyte: mean of the peer group means.
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors' Disclosures or Potential Conflicts of Interest: No authors declared any potential conflicts of interest.
- © 2012 The American Association for Clinical Chemistry