## Abstract

**BACKGROUND:** According to the traditional approach to statistical QC planning, the performance of QC procedures is assessed in terms of its probability of rejecting an analytical run that contains critical size errors (P_{EDC}). Recently, the maximum expected increase in the number of unacceptable patient results reported during the presence of an undetected out-of-control error condition [Max E(N_{UF})], has been proposed as an alternative QC performance measure because it is more related to the current introduction of risk management concepts for QC planning in the clinical laboratory.

**METHODS:** We used a statistical model to investigate the relationship between P_{EDC} and Max E(N_{UF}) for simple QC procedures widely used in clinical laboratories and to construct charts relating Max E(N_{UF}) with the capability of the analytical process that allow for QC planning based on the risk of harm to a patient due to the report of erroneous results.

**RESULTS:** A QC procedure shows nearly the same Max E(N_{UF}) value when used for controlling analytical processes with the same capability, and there is a close relationship between P_{EDC} and Max E(N_{UF}) for simple QC procedures; therefore, the value of P_{EDC} can be estimated from the value of Max E(N_{UF}) and vice versa. QC procedures selected by their high P_{EDC} value are also characterized by a low value for Max E(N_{UF}).

**CONCLUSIONS:** The P_{EDC} value can be used for estimating the probability of patient harm, allowing for the selection of appropriate QC procedures in QC planning based on risk management.

The main objective of the statistical QC (SQC)^{2} planning process is to select QC procedures that allow laboratory personnel to detect instabilities in the analytical system and to prevent reporting of patient results with medically important errors. According to the traditional approach, this objective can be achieved by selecting the appropriate control rule and number of control samples to be analyzed per run for detecting, with a high probability (P_{EDC}), critical size errors that would result in 5% of test results with errors exceeding the quality requirement for the test, while maintaining a low probability of false rejections (1).

However, probability of critical error detection (P_{EDC}) does not allow for the comparison of QC strategies that define analytical runs differently because it does not depend on the frequency of QC testing (2), nor is it influenced by different laboratory testing modes. To overcome this limitation, the expected increase in the number of unacceptable patient results reported during the presence of an undetected out-of-control error condition [E(N_{UF})], has been recently proposed (2) as an alternative QC performance measure. E(N_{UF}) depends on the magnitude of the increase in the analytical error that produces the out-of-control condition but has a defined maximum [Max E(N_{UF})[, corresponding to the worst-case situation for the associated QC strategy. Obtaining a Max E(N_{UF}) value acceptable for the laboratory has been suggested as a goal to guide the SQC planning process (3, 4).

In comparison with P_{EDC}, Max E(N_{UF}) is more directly related to the probability of causing harm to a patient due to the report of an incorrect result, and it is more related to the recent introduction of risk management concepts for QC planning in the clinical laboratory (5). However, the exact relationship between these performance measures is unknown, and the outcome of the traditional approach to SQC planning in terms of the number of erroneous patient results reported is not well established.

On the other hand, it is difficult for a laboratory to develop a SQC plan based on Max E(N_{UF}) because, in contrast with the traditional approach, neither graphical nor computer tools are readily available for the estimation of this performance measure and little guidance is available regarding how to use it in practice.

The purpose of this study was to investigate the relationship between P_{EDC} and Max E(N_{UF}) for simple QC procedures widely used in clinical laboratories and to construct charts relating Max E(N_{UF}) with the capability of the analytical process that allow for SQC planning based on the probability of harming a patient due to the report of an incorrect result.

## Materials and Methods

All investigations were conducted assuming a continuous-mode testing process with bracketed QC, as described previously by Parvin (3). In brief, patient samples were assumed to be analyzed in a continuous stream with periodic analysis of a set of controls. QC results were evaluated and a decision was made about all samples analyzed from the last accepted QC. It was assumed that an out-of-control condition can occur with equal probability anywhere within the stream of samples being analyzed, and that it persists until it is detected and that the test results are not reported until the next set of control results are accepted.

The capability of the analytical process is characterized by its σ value which determines the number of patient results with unacceptable errors [>allowable total error (TE_{a}), the quality requirement for the procedure in terms of a total allowable error] that are produced while the process is stably operating. The σ metric is calculated as (TE_{a} − |bias_{meas}|)/s_{meas}, where bias_{meas} is the analytical bias and s_{meas} is the analytical SD of the procedure during stable operation.

The number of erroneous results produced will increase with the occurrence of additional errors such as a systematic shift or an increase in the imprecision of the analytical process, which leads to an out-of-control condition. In this study, only systematic errors were investigated. The increase in systematic error (ΔSE) should be detected by the QC procedure to prevent the report of erroneous results. A QC procedure is defined as the combination of the QC rule applied and the number of QC samples tested in each QC event (e.g., 1_{2s} n = 3).

When the size of ΔSE is sufficient to produce 5% of patient results with error (i.e., exceeding the TE_{a}), a critical systematic error (ΔSE_{C}) has occurred, and the probability to detect it (P_{EDC}) has been used to characterize the performances of the QC procedures according to the traditional approach of the SQC planning process.

ΔSE_{C} is calculated as multiples of s_{meas} by using the following equation (Eq. 1):
(1) where 1.65 is the *z*-value that corresponds to 5% of the area in 1 tail of a gaussian distribution, or:
The expected increase in the number of unacceptable patient results reported during the existence of an undetected out-of-control error condition, E(N_{UF}), is computed, according to (3), as in Eq. 2
(2) where, ΔP_{E} represents the increase in the probability of producing unacceptable patient results because of the presence of the out-of-control error condition, ARL_{ED} is the average run length to error detection (the average number of batches that are processed before a QC rule rejection occurs), and M is the expected number of patient samples tested between QC events.

The average run length to error detection, ARL_{ED}, is computed as 1/P_{ED} (2), where P_{ED} is the probability of error detection by the QC procedure, computed by integration of the density function of the normal distribution, and ΔP_{E} is computed as the probability that the error for a result exceeds TE_{a} in the presence of an out-of-control error condition minus the probability the error for a result exceeds TE_{a} when the process is in control (6).

The probability of a false rejection, P_{FR}, is the value taken by P_{ED} when ΔSE = 0.

All results were obtained by numerical analysis and most were confirmed by Monte Carlo simulation.

## Results

The probability of a given QC procedure to detect an increase in the analytical error, P_{ED}, depends on the magnitude of the increase, ΔSE, and the capability of the analytical process, σ.

Fig. 1 plots P_{ED} as a function of ΔSE for a 4σ analytical process when 2 QC samples are analyzed and an 1_{3s} rule is applied in each QC event. The probability of error detection increases as does ΔSE, leading to a typical sigmoidal curve (power function curve). In this example, the critical systematic error ΔSE_{C} = 2.35 (σ − 1.65, as multiples of the analytical SD) and the probability to detect it P_{EDC} = 0.450.

The P_{EDC} value for a QC procedure is the same when used to control analytical processes with the same σ and is independent of the number of patient samples analyzed between QC events and the laboratory testing mode.

The expected increase in the number of unacceptable patient results reported during the existence of an undetected out-of-control error condition, E(N_{UF}), depends on the same variables as P_{ED}, but also on the frequency of QC and the laboratory testing mode (3).

In Fig. 1, E(N_{UF}) is plotted as a function of ΔSE for a 4σ analytical procedure when 100 patient samples are analyzed between QC events, 2 QC samples are analyzed in each QC event, and a 1_{3s} QC rule is applied, assuming a continuous-mode testing process with bracketed QC (see Materials and Methods for description). The value of E(N_{UF}) starts at zero, increases as the size of the systematic error increases until it reaches a maximum, Max E(N_{UF}), and then decreases to values close to zero, resulting in a bell-shaped curve.

The value of E(N_{UF}) increases at the initial part of the curve because ΔSE is not large enough to be detected efficiently by the QC procedure (low P_{ED}) so that the few erroneous results that are produced are also reported. At the maximum, the error is large enough to produce many erroneous results but not large enough to be readily detected, which corresponds to the worst-case situation for the QC strategy used. The decline observed afterward occurs because ΔSE is so great that it is immediately detected by the QC procedure (high P_{ED}) such that, of the large number of erroneous results that are produced, relatively few are reported.

For the example shown in Fig. 1, Max E(N_{UF}) = 4.77 at ΔSE = 2.18 when P_{ED} = 0.367. It is noteworthy that the ΔSE at which Max E(N_{UF}) is reached usually does not match with the corresponding ΔSE_{C} value.

For analytical processes with the same capability, controlled with simple QC rules of the form 1_{ks} (k = 2, 2.5, 3, 3.5, 4), analyzing the same number of controls per QC event, and the same number of patient samples between QC events, the calculated Max E(N_{UF}) value depends on the exact combination of imprecision and bias that characterizes the σ of the analytical process. This effect is more apparent at lower values of σ and higher values of k for the QC rule.

In Fig. 2, Max E(N_{UF}) is plotted as a function of the σ of the analytical process under control for a laboratory that works according to a continuous-mode testing process with bracketed QC and 100 patient samples analyzed between QC events and that uses simple QC procedures with different numbers of QC samples tested in each QC event (from 1–3) and different control rules (from 1_{2s} to 1_{4s}).

Each QC rule in the nomograms is represented by 2 lines, with the upper line corresponding to the analytical method with bias = 0.9 TE_{a} and the lower line corresponding to the unbiased method. The lines are practically superimposable, except at σ values below 3, where the effect of bias on the computed value of Max E(N_{UF}) is more evident. For analytical methods with an intermediate bias, the corresponding value of Max E(N_{UF}) is located between these 2 lines.

As expected, for a given QC procedure, the Max E(N_{UF}) values are lower for analytical processes with higher σ values. This is because the magnitude of the increase in systematic error that must occur to produce a significant number of unacceptable patient results is greater for higher σ values and will be detected more easily by the QC procedure, avoiding the report of erroneous results.

For a given value of σ, the computed Max E(N_{UF}) values are lower for QC procedures with tight limits (i.e., 1_{2s}) because they are more sensitive to increments in the systematic error, although they also have a higher probability of false rejections.

The graphical representations in Fig. 2 can be used as nomograms for selecting the appropriate QC procedure to obtain the desired Max E(N_{UF}), when the σ of the analytical process is known. For example, a laboratory that uses an analytical process with σ = 4 and works according to a continuous-mode testing process with bracketed QC and an average of 100 patient samples analyzed between QC events will report a maximum of 4.8 unacceptable patient results, on average, whenever an out-of-control condition occurs if it selects a QC procedure 1_{3s} n = 2. Because Max E(N_{UF}) is proportional to the number of patient samples analyzed between QC events, M, the Max E(N_{UF}) value could be lowered below 1 simply by making M = 20 or changing the QC procedure to 1_{2.5s} when n = 2 and M = 65, but in this last case, the probability of false rejection will increase.

It should be noted that, by definition, the Max E(N_{UF}) value represents the expected increase in the number, but not the total number of erroneous results, reported during the existence of an undetected out-of-control condition, and that all analytical processes in stable operation produce a number of erroneous results according to their own σ values. For these reasons, the total number of erroneous results reported when using analytical processes of poor performance (low σ values) can be significantly higher than the corresponding Max E(N_{UF}) value. For example, for a 2σ analytical process in stable operation that is controlled with a 1_{2s} QC procedure with n = 2, a total of 46.6 erroneous patient results, on average, would be reported between false QC rejections, whereas the Max E(N_{UF}) value is only 23.4 for this combination of QC procedure and σ (if bias=0), according to Fig. 2.

Each QC procedure has its own values of Max E(N_{UF}) and P_{EDC} when used to control an analytical process with a given σ value. In Fig. 3, Max E(N_{UF}) is plotted as a function of P_{EDC} for all QC procedures in Fig. 2. The plot shows that the relationships between these performance measures are very similar for all rules considered, especially for rules with the same N value, so that one can estimate the value of one from the other with a relatively small margin of error, almost regardless of the QC procedure considered. For example, the computed Max E(N_{UF}) value for a QC procedure that has a probability of 0.90 to detect critical systematic errors ranges from 0.30 for 1_{2s} when n = 1–0.41 for 1_{4s} when n = 3. Conversely, to obtain an average of less than 1 unacceptable patient result as a consequence of an out-of-control condition, the plot shows that a QC procedure with a P_{EDC} of about 0.8 is needed, assuming that 100 patient samples are analyzed between QC events in both examples.

## Discussion

The analytical quality of a clinical laboratory depends primarily on the number of erroneous results that are reported to health providers, and it is important to ensure that wrong decisions are not made in the management of patients.

According to the risk management techniques that are currently being introduced in clinical laboratories, the analytical quality of the laboratory can be improved by identifying and evaluating potential causes of failure in the testing process, estimating the risk of harm that they represent to the patient, and designing strategies to prevent failures and to detect errors before incorrect results are reported, through the development of a QC plan (5).

These strategies include, but are not limited to, QC procedures using stable control materials. An increasing proportion of errors affecting the quality of the analytical results has its origins outside the analytical phase of the testing process. These errors are usually random and unpredictable, tend to affect individual samples, and are not detected effectively by QC procedures based on the analysis of control materials (7). Nevertheless, these strategies still have a central role for detecting errors caused by excessive process variation (such as an increase in the systematic error) that affect patient and control samples in a similar fashion. In this study, only systematic errors are addressed; however, increases in analytical imprecision also occur frequently, are difficult to detect and can result in the erroneous reporting of patient results.

Because the risk can never be completely eliminated, the goal of these strategies is to reduce the probability of harming the patient to a residual level that could be considered acceptable from the point of view of patient safety.

The Max E(N_{UF}) value represents the expected number of final erroneous results that could be reported in the worst case scenario, results that are not subjected to further correction and that could be used for making erroneous clinical decisions. Therefore, Max E(N_{UF}) is directly related to the probability of patient harm and can be used to estimate it, provided that both, the mean number of patient results examined between failures and the probability that an incorrect result lead to patient harm, are known (8). The probability of harm can be lowered to an acceptable level by minimizing the Max E(N_{UF}) value through the selection of appropriate QC procedures and the frequency of QC events in the SQC planning process.

We have shown that the Max E(N_{UF}) value can be easily estimated, at least for simple QC procedures, either from the σ value that characterizes the analytical process by using nomograms as shown in Fig. 2, or from the probability of detection of critical systematic errors P_{EDC,} when a traditional approach to the SQC planning is used, as shown in Fig. 3. The same value of Max E(N_{UF}) can be achieved by using different combinations of QC procedures and numbers of patient samples analyzed between QC events.

There is an inverse relationship between the limits of the SQC rules applied and the corresponding value of M that should be used to obtain an acceptable Max E(N_{UF}) value. Rules with wider limits, and thus lower P_{FR}, can be used by decreasing the M value accordingly, and vice versa.

The exact combination selected will determine the efficiency of the SQC strategy, i.e., the number of results reported per sample (patients and controls) analyzed. Low values of M tend to decrease efficiency because the number of controls analyzed per patient sample increases. In contrast, QC procedures with low P_{FR} tend to improve efficiency because they minimize the repetition of patient and control samples.

However, it should be noted that efficiency is a complex issue that depends not only on the capability of the analytical process, but also on other factors such as the reliability of the analytical system (9) and the cost of testing QC material vs the costs associated with false QC rejections.

Max E(N_{UF}) only represents the expected increase in the number of erroneous results reported in the presence of an out-of-control error condition and does not reflect accurately the total number of erroneous results reported when low-σ (σ < 3) analytical processes are used. In these cases, many of the reported erroneous results are produced because the inherent variability of the analytical process, whether the process is out-of-control or not, and Max E(N_{UF}) underestimates the total number of erroneous results that are being reported, and thus, the patient risk. Appropriate selection of QC procedures can reduce the expected number of unreliable final patient results owing to an out-of-control condition but in no way can a QC strategy, regardless of how stringent it is, compensate for the bad quality of an analytical method (10).

The selection of SQC procedures has been done classically with the objective of having 90% detection of critical systematic errors, while maintaining false rejections as low as possible (11). Fig. 3 shows that values for Max E(N_{UF}) in the range from 0.3–0.4 are obtained when simple QC procedures with P_{EDC} = 0.90 are used to control an analytical process. This means that up to about 300 patient samples could be analyzed between QC events for reporting less than 1 erroneous result during an out-of-control condition, which represents a typical daily workload for many tests performed in a small or medium-sized laboratory. Therefore, the use of P_{EDC} = 0.90 as a general criterion appears to be quite reasonable for selecting QC procedures that reduce the probability of harming the patient to a level that could be considered acceptable for many tests conducted in a laboratory.

For the simple QC rules investigated here, obtaining P_{EDC} = 0.90 while maintaining an acceptable rejection rate is only possible in analytical processes with fairly high σ metrics. For processes with lower σ values, the laboratory should select the appropriate number of patient samples to be analyzed between QC events to obtain an acceptable value for Max E(N_{UF}) given the P_{EDC} that can be reasonably achieved.

Regardless, we have shown that there is a practically biunivocal relationship between Max E(N_{UF}) as a percentage of the average number of patients tested between QC events (M), and P_{EDC} over a range of simple QC rules of the form 1ks (k = 2, 2.5, 3, 3.5, 4); therefore, the value of P_{EDC} can be estimated from the value of Max E(N_{UF}) and vice versa. In this way, the P_{EDC} value may be used for selecting appropriate QC procedures during the development of a QC plan based on risk management.

Since the introduction of the Six Sigma concept into clinical laboratories (12), several recommendations have been made for selecting optimal SQC procedures considering only the σ of the analytical process (13–15). Unless there is some exception (13), none of these recommendations take into account the QC frequency or the laboratory testing mode, but most of the SQC procedures recommended for control analytical processes with a certain σ show low Max E(N_{UF}) values (typically below 1 in the nomogram of Fig. 2) because they have been selected to provide a high probability of detection of critical errors (P_{EDC} > 0.90).

In summary, there is a close relationship between P_{EDC} and Max E(N_{UF}) for simple QC procedures generally applied in clinical laboratories that allow the use of P_{EDC} for estimating the probability of patient harm in SQC planning based on risk management. QC procedures selected by their high probability of detection of critical errors, according to the traditional approach for SQC planning, are also characterized by a low number of erroneous patient results reported during the presence of an out-of-control error condition reconciling these 2, apparently different, approaches.

## Footnotes

↵2 Nonstandard abbreviations:

- SQC,
- statistical QC;
- P
_{EDC}, - probability of critical error detection;
- E(N
_{UF}), - expected number of unreliable final patient results;
- Max E(N
_{UF}), - maximum E(N
_{UF}); - TE
_{a}, - allowable total error;
- ΔSE,
- increase in systematic error;
- ΔSE
_{C}, - critical ΔSE.

(see editorial on page 907)

**Author Contributions:***All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.***Authors' Disclosures or Potential Conflicts of Interest:***No authors declared any potential conflicts of interest.***Role of Sponsor:**The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript.

- Received for publication December 31, 2015.
- Accepted for publication March 24, 2016.

- © 2016 American Association for Clinical Chemistry