Background: Antinuclear antibodies (ANAs) are associated with several inflammatory rheumatic diseases. The aim of the present work was to evaluate enzyme immunoassays (EIAs) and compare them with classic immunofluorescent analysis (IFA) for the detection of ANA.
Methods: Seven enzyme immunoassays were used in this study. All assays were applied as described by the manufacturers. Three populations were included in the study: (a) a population of patients with well-established autoimmune inflammatory disease (n = 102); (b) a population in which a rheumatic disease was diagnosed up to 5 years after an IFA was performed (n = 164); and (c) a population of consecutive outpatients suspected to have a rheumatic disease (n = 101). The current clinical diagnoses of the patients served as the standard against which performance of the assays was evaluated.
Results: In patients with well-established rheumatic disorders, the newly developed EIA in which HEp-2 extracts were included had sensitivities and specificities comparable to or in some instances better than the IFA. The assays without HEp-2 extracts included had significantly lower sensitivities and specificities. In the outpatient population, up to 51% of patients had positive ANA tests that did not correspond to classic ANA-associated disease. However, in the assays in which the HEp-2 extracts were not included, the false-positive rate was <10%. The false-negative rate judged against IFA differed from assay to assay and disease to disease and was mostly <10%.
Conclusions: In this study, the sensitivities of EIAs and IFA were largely comparable. However, EIAs without HEp-2 extracts included had a low sensitivity but a high specificity, particularly in nonselected populations. The choice of test is highly dependent on the clinical setting in which the ANA test is to be used and on laboratory policy.
Autoantibodies directed against cell nuclear autoantigens are called antinuclear antibodies (ANAs),1 although several of these autoantigens actually shuttle between nuclei and cytoplasm during the cell cycle or are permanently localized in the cytoplasm. Most ANAs probably do not have a pathogenetic role in autoimmune inflammatory disease but may reflect the inflammatory events taking place in the tissues in such diseases (1).
Positive ANA testing is associated with several inflammatory rheumatic diseases and has been included in the classification criteria for systemic lupus erythematosus (SLE) (2), mixed connective tissue disease (MCTD) (3) and the European classification criteria for Sjögren syndrome (SS) (4) among others. ANAs can also be a critical part in the diagnosis of several nonrheumatic autoimmune diseases, such as autoimmune hepatitis and drug-induced autoimmune syndromes (5)(6).
For decades, immunofluorescent analysis (IFA) has been the standard method for detecting ANA when diagnosing rheumatic autoimmune inflammatory diseases, also called connective tissue diseases. As such, testing for ANAs is intended as an aid in the diagnosis of connective tissue diseases. However, established clinical practice may have moved toward use of a negative test to exclude connective tissue disease (7). This practice may lead to an increase in the number of performed analyses and, as a consequence, a low rate of clinically relevant positive tests and a relative increase in the number of false-positive tests. In addition, the population in the Western world is increasing, and clinicians need to be aware of the fact that ANAs are very common in elderly healthy individuals (8).
Detection of ANAs by IFA is a microscopic technique, in which reader agreement and reliability are of great importance. Furthermore, the manual IFA procedure is relatively labor-intensive. These circumstances have stimulated the development of simpler, high-throughput analyses that can be automated and standardized. Enzyme immunoassays (EIAs) offer some of the desired advantages, and commercial reagent sets have been offered by several manufacturers. However, these have only evolved in the last few years to become potentially useful. Several EIAs have been reported to have limitations in epitope repertoire, sensitivity, and specificity (9)(10). Furthermore, substantial differences in terms of positivity among various EIAs have been described (10)(11). Because ANA testing is a critical part of diagnosis in several autoimmune rheumatic diseases, it is important to know whether initial screening for ANAs by EIA can replace IFA screening on HEp-2 cells without loss of critical information.
The aim of this study was to evaluate the technical and diagnostic performance of current EIAs and IFA for the detection of ANAs.
Materials and Methods
Three patient populations were used for the present evaluation:
Population 1 consisted of 102 patients with established connective tissue disease: 35 with SLE, 21 with SS, 22 with systemic sclerosis, 21 with MCTD, and 3 with rheumatoid arthritis. Of this population, 19% were males, and the mean age was 47 years (range, 15–89 years). This population was used mainly for evaluating the sensitivity of the assays in well-established rheumatic diseases.
Population 2 consisted of 164 patients attending specialized hospital clinics in the Greater Copenhagen area in Denmark with clinical suspicion of connective tissue disease. The included patients had a disease duration of less than 1 year (recent onset). Blood had been drawn from these patients up to 5 years before any diagnosis had been assigned. An ANA-associated diagnosis was found for 148 of the patients (see Table 3⇓ ). Twelve percent of this population were males, and the mean age was 50 years (5–87 years). Sera had been stored at −20 °C in the Department of Autoimmunity at Statens Serum Institut for up to 5 years before the study. These sera were retrieved for use in the present comparative study.
Population 3 included 101 patients at the University Hospital of Copenhagen (Hvidovre, Denmark) referred to the outpatient clinic of rheumatology on the suspicion of a rheumatic disease. Thirty-one percent of this population were males, and the mean age was 55 years (2–94 years). An ANA test was performed as part of the diagnostic routine. This population presented with a wide variety of symptoms, including pain in various joints, muscle pain, skin conditions, neurologic manifestations, and malignancies. Only six patients were eventually diagnosed as having an ANA-associated disease (see Table 3⇓ ).
Six EIA manufacturers were invited, and all agreed to participate. One of the manufacturers (Pharmacia & UpJohn Diagnostics) supplied two test systems. In all, we evaluated seven solid-phase EIAs [Quanta Lite ANA ELISA (INOVA Diagnostics), Diastat ANA ELISA (Shield Diagnostics), Bindazyme ANA Screen (The Binding Site), Microplate Autoimmune ANA Screen (Bio-Rad Laboratories), Relisa ANA (ImmunoConcepts), VarElisa ANA Rheuma (Pharmacia Diagnostics), and UniCap Symphony combined with UniCap dsDNA (Pharmacia Diagnostics)] and an IFA assay using HEp-2 cell slides (ImmunoConcepts) as diagnostic screening tools in the detection of ANAs (autoantibodies) in inflammatory rheumatic diseases.
All of the above assays detect antibodies to double-stranded DNA (dsDNA), SS-A, SS-B, Scl 70, histones, Sm, Sm/RNP, centromeres, and Jo-1. The Quanta Lite ANA ELISA also detects mitochondrial antigens, proliferating cell nuclear antigen, and ribosomal P-proteins; the VarElisa also detects PM-Sl-100 but does not detect dsDNA. Similarly, the UniCap Symphony does not detect dsDNA; for this reason, antibodies to dsDNA were detected by use of the UniCap dsDNA. A positive UniCap Symphony and/or a positive UniCap dsDNA was classified as a positive test.
Other characteristics of the assays are shown in Table 1⇓ . The formulation of the Bindazyme assay was changed during this study; the HEp-2 extract was omitted in the latest formulation of the assay. The previous version of the assay was used to test populations 1 and 3, whereas the new formulation was used to test population 2.
Blood was drawn, and serum was stored at −20 or −80 °C. All analyses were performed as described by the manufacturers. IFA was performed at a 1:160 dilution of the sera, as described, using IgG-specific fluorescein isothiocyanate conjugate (15). All tests were performed in duplicate.
All IFAs were performed at the Department of Autoimmunology, Statens Serum Institut, Copenhagen S, Denmark, and all EIAs were performed at the Department of Clinical Biochemistry and Molecular Biology, University Hospital of Copenhagen, Hvidovre, Denmark.
test classification and calculations
All assays were evaluated against the established clinical diagnosis and compared with a standard IFA. Most of the assays operate with borderline positive test results. Because the assays in this study were tested in a context of screening with the purpose to catch all potential patients, borderline values were classified as positive. The cutoff limits recommended by the manufacturers were used in all comparisons. The influence of the cutoff limits on the sensitivity, specificity, and predictive values of the assays was evaluated in the population with well-established diagnoses and the outpatient population. Results are presented as cutoff limits relative to the cutoffs recommended by the manufacturers. The SPSS (Ver. 11.5) package was used for all statistical calculations.
The intraassay CV did not exceed 4.8% for any of the assays. The interassay CV obtained with the supplied positive controls varied considerably: Quanta Lite (7.9%), Relisa (25%), Bindazyme (11%), Bio-Rad (17%), VarElisa (15% with the negative control), Diastat (38%), UniCap dsDNA (13%), and UniCap Symphony (2.7%).
The test results for patients with previously well-established diagnosis are shown in Table 2⇓ . Only 99 patients were included in Table 2⇓ because only 1 of the 3 patients with rheumatoid arthritis tested positive and only in the Symphony assay. The Quanta Lite, Bio-Rad, Relisa, Diastat, and Bindazyme assays had positive test rates exceeding 90%. The VarElisa and UniCap were less sensitive, whereas IFA had a positive test rate between the two groups of EIAs. Generally, the assays performed well, particularly in patients with SLE, except for the VarElisa and UniCap, which showed lower sensitivity (see Table 1⇑ in the Data Supplement that accompanies the online version of this article athttp://www.clinchem.org/content/vol50/issue11/). The positive test rates for some of the assays were somewhat low in patients with MCTD. Among the tested assays, IFA and the VarElisa had the lowest positive test rates for patients with SS. The VarElisa generally had a low positive test rate, which was also the case for the UniCap except in patients with SS.
The assay performance in the cohort of patients with early suspected inflammatory rheumatic disease is summarized in Table 2⇑ . The sensitivity of the assays, including IFA, in this cohort was considerably lower than the sensitivity for patients with a well-established diagnosis (see Table 2⇑ ); however, the sensitivities were comparable among assays except for the UniCap and VarElisa, which had lower sensitivities. In particular, patients with SLE and MCTD tested positive at a high rate, as did patients with scleroderma in most assays, although to a lesser extent than the population with well-established diagnoses (see Table 2⇑ in the online Data Supplement). In patients with recent-onset or established scleroderma, anti-nucleolar ANAs were missed by most of the EIAs, but many sera were EIA positive while being IFA negative, a finding that warrants further studies to understand the underlying reasons. We observed no significant differences among the assays concerning the duration of the time between blood sampling and establishing of diagnosis (results not shown), nor was there any correlation between the length of this period and the specific clinical diagnoses.
Only 6 of the 101 consecutive patients referred to the outpatient clinic of the rheumatology department actually were found to have an ANA-associated inflammatory rheumatic disease: 2 had SLE, 2 had SS, 1 had polymyositis, and 1 had MCTD. The remaining patients had a wide variety of diagnoses, as shown in Table 3⇓ . IFA gave positive results for all six patients with connective tissue diseases. All of the other assays, except the VarElisa and UniCap, gave positive results for four patients, whereas the latter gave positive results for only two patients. Of particular importance was the number of apparently false-positive tests (see Table 4⇓ ). In this aspect, the VarElisa and UniCap clearly outperformed all the other assays with regard to diagnostic specificity.
We compared the ELISAs with IFA by calculating κ values (Table 5⇓ ). Except for the Bio-Rad assay, all assays gave positive, although low, κ values in the patients with established diagnosis. The κ values increased significantly for all assays (including the Bio-Rad) in the patients tested before diagnosis was established (population 2). The agreement (i.e., cases in which both assays compared showed the same test result) between the IFA and the EIA was >80% except for the VarElisa and UniCap. The lower agreement between IFA and the Bindazyme in population 2 (Table 5⇓ , First sample) can be ascribed to the change in the formulation of the Bindazyme assay, i.e., the HEp-2 extract was omitted from the assay. This new formulation renders this assay similar to the VarElisa and UniCap assays. The serologically false-negative results obtained in the EIAs when judged against IFA could be ascribed to IFA ANA patterns that are less common, e.g., ANAs that recognize nucleoli, nuclear envelope, multiple nuclear dots, or mitotic spindle apparatus (data not shown).
The sensitivity, specificity, positive predictive value, and negative predictive value as a function of the cutoff limits are shown in Figs. 1 and 2, respectively, of the online Data Supplement. The relative cutoff value equal to one is the value recommended by the manufacturers. The sensitivities of all of the assays can be increased by lowering the cutoff limit, but the specificities concomitantly decrease. Similarly, the positive predictive values can be increased by increasing the cutoff limits, but the tradeoff is a decrease in the negative predictive values of the assays. Interestingly, all of the assays had high positive predictive values in patients with established diagnoses. Similarly, the negative predictive values were high when used in a screening procedure, i.e., the assays are very efficient in excluding inflammatory rheumatic disease.
This study clearly illustrates that detection of ANAs is highly dependent on the method used. Some of the EIAs performed poorly, but others had a sensitivity similar to and sometimes slightly better than that for IFA when compared with the clinical diagnoses, and some EIAs also had the lowest number of false-positive results (defined as positive result for ANAs in ANA-unrelated disease). The Relisa assay had the highest rate of diagnostically meaningful positive tests (Table 2⇑ ; also see Fig. 1 in the online Data Supplement), but also had the highest rate of false-positive tests (Table 4⇑ ). The Bindazyme and Bio-Rad assays had equally high rates of positive tests (Table 2⇑ ), but the rates of false-positive tests were within the typical range and were much lower than the Relisa assay. The reason for the rather high false-positive rate in the Relisa assay seems to be the choice of cutoff. Increasing the cutoff from the recommended 10 to 20 decreased the false-positive rate significantly while only slightly influencing the sensitivity of the assay. This would also increase the positive predictive value of the assay with only a minor decrease in negative predictive value.
The ANA-positive rate decreased significantly in the analysis of the patients with tests performed before a connective tissue disorder was established. One reason for this could be that the activity or the extent of the disease may have been lower at the time when blood was drawn, so that tests performed on sera drawn up to 5 years before the diagnosis was established would be ANA-negative to a greater extent than those tests performed close to the time when a diagnosis could be established. However, there was no clear-cut indication of this in this study, at least for ANAs of the IgG class, which most tests are detecting (Table 1⇑ ). It should be mentioned that recent studies on SLE patients showed the presence of ANAs characteristically found in established disease up to 9 years (mean, 3.3 years) before clinical onset of SLE (16).
The Bindazyme assay performed differently from the other assays in both populations 1 and 2 (Table 2⇑ ). The reason for this is probably that the antigen content of the assay was changed during the study in that extracts of HEp-2 cells are no longer included in the Bindazyme assay. This makes the assay more comparable to the VarElisa and UniCap. Unfortunately, it was not possible to test whether the false-positive rate decreased to the same range as VarElisa and UniCap.
The results shown in Figs. 1 and 2 in the online Data Supplement demonstrate the effects of changing the cutoff limits of all of the EIAs. Decreasing the cutoff limits for the VarElisa and UniCap assays could make these assays more comparable to the remaining assays, but this may not be a sound practice. These assays differ in a crucial way from the other assays in that the antigens introduced are recombinant or purified without a HEp-2 cell extract. The assays are therefore antigenically better defined, which is ideal for diagnosis of inflammatory rheumatic diseases and helps ensure lower batch-to-batch variation, but the tradeoff is the lower sensitivity and lack of usefulness in ANA-associated nonrheumatic diseases. The gain, however, is the low false-positive rate. All of the other assays used purified antigens together with extracts of HEp-2 cells (except for the newest formulation of the Bindazyme). The addition of HEp-2 cell extracts is an attempt to include unrecognized as well as nonpurified autoantigens. The penalty for doing so is obviously a high rate of false-positive test results, which means that more analyses have to be performed to confirm or refute presence of specific ANAs. Typically, for every diagnostically true-positive test result, four test results will be falsely positive, i.e., only ∼20% of the patients who test positive will actually have an inflammatory connective tissue disorder. This rate may be in the extreme range, but the frequency of patients with connective tissue disease in the population screened was not much lower than reported in other surveys (1). Probably the only way to reduce the rate of false-positive tests is to change the practice of ordering the tests (17)(18).
The tests performed in population 2 were ordered by the physician in the department the patient first attended, not necessarily by physicians specialized in rheumatology. The false-positive rate could probably be reduced dramatically if the ordering physician was the specialist, but in an overall context this may not be economically sound (e.g., logistic considerations, demand for more specialists, extra time spent by the patient). All of the assays did show discrepancies regarding agreement as to which patients were positive or negative. This is an anticipated result experienced in almost all comparisons among diagnostic tests. These discrepancies theoretically reflect differences in the antigens included in the tests, or rather that antibodies are present that are not detected by all assays. This is not to say that there are no antibodies present in seronegative patients, but there should be room for reducing the size of this particular subgroup of seronegative patients, if they in fact exist. This is beyond the scope of this study but should definitely be explored further.
In this study, IFA was performed at a 1:160 dilution and had a lower sensitivity than several of the EIAs. In putatively normal individuals, approximately one third have a positive ANA test at a 1:40 dilution and 5% at a 1:160 dilution of serum. At the 1:40 dilution, IFA has a high sensitivity for detecting patients with SLE, SS, and systemic sclerosis, but the increase in specificity at the 1:160 dilution is obviously more important. For this reason, it has been suggested that the IFA be performed at both dilutions (19), but this will lead to an increase in costs. The IFA is therefore routinely done at a 1:160 dilution only, but serial titration is commonly done to find the end-point titer.
The EIA ANA tests generally performed at least as well as the traditional IFA with regard to sensitivity. However, there is room for improvement of the EIA ANA tests, which may include the addition of new purified or recombinant antigens in the assays. Furthermore, attention must be drawn to the fact that for certain rheumatic disease diagnoses, such as juvenile rheumatoid arthritis (20), it has been shown that EIA ANA testing cannot reveal the presence of disease-related ANAs. Similar deficiencies in EIA detection of ANAs are found in IFA-positive cases of polymyositis and dermatomyositis patients as well as in subpopulations of the classic inflammatory connective tissue diseases (1)(21).
Most clinical immunology laboratories use a staged approach for ANA testing, starting with a sensitive screening test, which in positive cases is followed by testing for specific ANAs by other methods (18). Hence, the ideal EIA for ANA screening would detect as many of the IFA positives as possible to minimize the ANA-negative population and allow sera to be tested further when a connective tissue disease is clinically suspected. However, in many patients identified as ANA-positive by IFA, the pattern of reactivity by itself is indicative of a connective tissue disease, and diagnosis thus need not be pursued further (1). One should also be aware of the fact that almost all EIA tests for ANAs are produced to support diagnoses of inflammatory rheumatic diseases and that that they may therefore have limited use in other ANA-positive diseases. Thus, if EIA screening for ANAs is chosen as routine, the limitations of the EIA need to be known by the clinicians ordering serology tests.
There were some differences among the EIA tests, but the choice of test will most probably depend not only on the technical performance of the tests but also on issues of cost, service, accessibility, and the overall goals and policies in performing ANA tests. The latter refers to the question of whether the goal is high sensitivity, i.e., identifying most of the patients suspected to have a connective tissue disease, or high specificity, i.e., avoiding false-positive tests. The answers to these questions would be the most important consideration in the decision of which test to choose. The performance of the assays can be changed by manipulation of cutoff limits, but one should be clear about what is measured, i.e., the assays can detect only antibodies for which antigens are present, and no changes in cutoff limits can change this. In any instance, positive tests should be confirmed and further investigated, e.g., as patterns detected in IFA or by use of the EIAs available for specific antibody testing.
1 The formulation of the Bindazyme assay was changed during the study in that HEp-2 extracts are now omitted from the new assay.
1 A positive result for the Symphony assay and/or for dsDNA was classified as combined positive.
2 Patients with well-established rheumatic disease (n = 99), except UniCap (n = 98).
3 Patients with later confirmed rheumatic disease (n = 148).
1 Positive/positive denotes a positive diagnosis/positive test, positive/negative denotes a positive diagnosis/negative test, negative/negative denotes a negative diagnosis/negative test, and negative/positive denotes a negative diagnosis/positive test.
1 Patients with established diagnosis of rheumatic disease.
2 ASE, asymptotic standard error.
3 Patients for whom first sample was tested before rheumatic disease was established (population 2).
↵1 Nonstandard abbreviations: ANA, antinuclear antibody; SLE, systemic lupus erythematosus; MCTD, mixed connective tissue disease; SS, Sjögren syndrome; IFA, immunofluorescent analysis; EIA, enzyme immunoassay; and dsDNA, double-stranded DNA.
- © 2004 The American Association for Clinical Chemistry