BACKGROUND: Zinc transporter 8 (ZnT8) is a recently identified major autoantigen in type 1 diabetes, and autoantibodies to ZnT8 (ZnT8A) are new markers for disease prediction and diagnosis. Here we report the results of the first international proficiency evaluation of ZnT8A assays by the Diabetes Antibody Standardization Program (DASP).
METHODS: After a pilot workshop in 2007, an expanded ZnT8A workshop was held in 2009, with 26 participating laboratories from 13 countries submitting results of 63 different assays. ZnT8A levels were measured in coded sera from 50 patients with newly diagnosed type 1 diabetes and 100 blood donor controls. Results were analyzed comparing area under the ROC curve (ROC-AUC), sensitivity adjusted to 95% specificity (AS95), concordance of sample ZnT8A positive or negative designation, and autoantibody levels.
RESULTS: ZnT8A radio binding assays (RBAs) based on combined immunoprecipitation of the 2 most frequent ZnT8 COOH-terminal domain polymorphic variants showed a median ROC-AUC of 0.848 [interquartile range (IQR) 0.796–0.878] and a median AS95 of 70% (IQR 60%–72%). These RBAs were more sensitive than assays using as antigen either 1 ZnT8 variant only or chimeric constructs joining NH2- and COOH-terminal domains, assays based on immunoprecipitation and bioluminescent detection, or assays based on immunofluorescent staining of cells transfected with full-length antigen.
CONCLUSIONS: The DASP workshop identified immunoprecipitation-based ZnT8A assays and antigen constructs that achieved both a high degree of sensitivity and specificity and were suitable for more widespread clinical application.
The Diabetes Antibody Standardization Program (DASP)8 is a collaborative effort of the Immunology of Diabetes Society and the CDC aimed at the evaluation and improvement of assays for type 1 diabetes–associated autoantibodies, the provision of standard reference samples, and the validation of novel candidate autoantibody antigens (1,2). DASP supervises international workshops in which relatively large sets of coded sera from patients with type 1 diabetes and nondiabetic controls are tested for islet autoantibodies by participating centers, followed by a centralized and independent assessment of autoantibody assay performance.
Recently, the zinc transporter 8 (ZnT8) was identified as a novel autoantibody antigen in patients with new-onset type 1 diabetes (3). ZnT8 is a cation transporter highly expressed on the membrane of pancreatic islet β-cell insulin secretory granules (4), where it performs the uptake of the cytoplasmic zinc required to store insulin in hexameric form (5). Autoantibodies to ZnT8 (ZnT8A) were also observed in the preclinical phase of type 1 diabetes and in patients with latent autoimmune diabetes in adults, suggesting their potential as predictive and diagnostic markers (6,–,11). The ZnT8A humoral response is complex and comprises several distinct autoantibody specificities, directed to epitopes within both NH2- and COOH-terminal cytoplasmic domains of ZnT8. In particular, ZnT8A include antibodies that can be specific for either of the 2 main polymorphic variants of the COOH-terminal domain, which differ by a single amino acid at position 325 (12, 13). After the discovery of these novel major type 1 diabetes autoantibodies, DASP sought for the inclusion of ZnT8A in the panel of markers that are currently the focus of international immunoassay standardization efforts; namely autoantibodies to insulin (14, 15), glutamic acid decarboxylase (GAD), and islet antigen 2 (IA-2) (16). Here we report the results of the first international workshop for the standardization of autoantibodies to ZnT8.
Materials and Methods
A pilot DASP workshop for the evaluation of ZnT8A was held in 2007 followed by a full workshop in 2009. In each workshop, laboratories received uniquely coded sets of sera from patients with newly diagnosed type 1 diabetes, contributed by several centers around the world, and from US blood donors without a family history of diabetes, plus selected reference samples. Because DASP is not a population-based study, the selection of workshop samples did not account for differences in islet autoantibody frequencies between ethnic groups. Although this introduces a potential bias in sensitivity and specificity analyses, it was deemed acceptable because it would not affect the comparison between different laboratories and assay performances. Diabetes was diagnosed according to WHO criteria and classified as type 1 on the basis of clinical characteristics. All samples were collected within 14 days of starting insulin treatment (median 4.5 days). Sera were prepared and frozen in 100-μL aliquots and distributed by the CDC as previously described (16). Laboratories were asked to test samples for ZnT8A using the assay format of their choice, to provide details of their assay protocol, and to report assay results, including raw data, to DASP for analysis. Results for >1 ZnT8A assay or assay format from each laboratory could be reported for evaluation if desired.
In the first pilot 2007 workshop, 16 laboratories from 8 countries (for the list of participants, see the Supplemental Appendix, which accompanies the online version of this article at http://www.clinchem.org/content/vol57/issue12) received 50 patient sera and 100 control sera. The patients included 16 females and 34 males, had a median age of 17 years (range 9–29 years), and were all white. The controls included 51 females and 49 males, had a median age of 20 years (range 18–28 years), and included 80 whites, 4 Hispanics, and 16 blacks. One ZnT8A-positive and 1 negative reference human serum was distributed on request to participating laboratories (courtesy of Prof. J. Hutton). All but 2 laboratories used as antigen source the plasmid clone JH4 that was also distributed on request (courtesy of Prof. J. Hutton). This clone encoded a human ZnT8 protein comprising amino acids 265–369 of the COOH-terminal domain with arginine at residue 325 (CR). One of the patient samples (IDS151) was later found to derive from an insulin-treated patient with longstanding type 1 diabetes, and 1 control sample (N52078) was found to have high-titer autoantibodies against GAD, IA-2, insulin, and IA-2β. Both samples were excluded from the analysis.
In the 2009 workshop, the number of participants rose to 25 laboratories from 13 countries (see the online Supplemental Appendix for the list of participants), and these again received 50 sera from type 1 diabetes patients together with 100 control sera. The patients comprised 17 females and 33 males, had a median age of 24.5 years (range 10–32 years), and included 40 whites, 8 Hispanics, 1 black, and 1 Asian. Controls comprised 50 females and 50 males, had a median age of 20 years (range 18–30 years), and included 80 whites and 20 blacks. In addition, each laboratory was sent a ZnT8A-positive serum from a patient with type 1 diabetes, a ZnT8A-negative control serum from the DASP repository as identified in the pilot 2007 workshop, and 5 serial dilutions of a rabbit polyclonal antiserum obtained after immunization with recombinant human ZnT8 COOH-terminal domain antigen (courtesy of Prof. J. Hutton). Plasmid clones of the ZnT8 COOH-terminal domain polymorphic variants in the pTnT vector were distributed on request to participating laboratories (courtesy of Prof. J. Hutton). These were clones JH5.0, 5.1, and 5.2 encoding for amino acids 265–369 of human ZnT8 and corresponding to the polymorphic variants with arginine (CR) or tryptophan (CW) at residue 325 or their chimeric dimer (CR-CW), respectively. Among the control sera, 6 samples (N51250, N53799, N60573, N61439, S8660, and S8759) were found to have high-titer autoantibodies to other islet antigens and were therefore excluded from the analysis.
We used ROC curves to evaluate the performance of each assay in discriminating health from disease. We calculated the area under the ROC curve (ROC-AUC) with 95% CI assuming a nonparametric distribution of results. In 2009, the statistical significance of differences in the ROC-AUC between assays was evaluated based on the Mann–Whitney U-test, and the threshold at which accuracy (defined as the number of true positives plus true negatives divided by true positives plus false positives, true negatives, and false negatives) was maximal for each quantitative test was calculated using StAR software (17).
We calculated laboratory-assigned sensitivity and specificity as the percentage of type 1 diabetes sera reported as ZnT8A-positive and as the percentage of blood donor sera reported as negative. Concordance of laboratory-assigned designations for positivity was expressed as average pairwise percent agreement between assays using ReCal software (18). We tested the occurrence of agreement by chance by calculating the agreement coefficient (AC1) according to Gwet (19) or the κ coefficient according to Fleiss (20) using the Agreestat macro for Excel (Advanced Analytics).
Adjusted sensitivity 95 (AS95), the level of sensitivity corresponding to a specificity of 95%, was calculated based on the ZnT8A quantitative results using as the threshold the 95th percentile of values observed in the blood donor sera included in the analysis. In both the 2007 and 2009 workshops, common ZnT8A indexes were calculated based on the respective positive reference sera, according to the following formula:
For laboratories that did not test the positive reference serum, a DASP serum that showed similar median levels in other laboratories was used (IDS162 in 2007 and IDS007 in 2009). For comparison of antibody levels between laboratories in patient and control samples, we determined the Kendall W rank correlation coefficient to assess concordance of relative levels of ZnT8A in different assays after ranking (21) and the overall concordance correlation coefficient (OCCC) according to Barnhart (22) for comparison of antibody titers, the latter using the f.analysis macro (23) in the R language and environment for statistical computing and graphics (24).
For all statistical analyses, 2-tailed P values <0.05 were considered significant.
ZNT8A 2007 PILOT WORKSHOP SUMMARY
In the 2007 ZnT8A workshop, all reporting laboratories adopted the radio binding assay (RBA) format based on in vitro transcribed and translated radiolabeled antigen. Sixteen RBAs measured ZnT8A to the human ZnT8-R325 COOH-terminal domain variant (CR assays) (see online Supplemental Table 1). Large differences between assay performances were evident after adjusted sensitivity calculation and ROC analysis (Fig. 1A and Fig. 2A). The median AS95 was 53% [interquartile range (IQR) 48%–59%], and the median ROC-AUC was 0.704 (IQR 0.688–0.737). ROC-AUC differences between individual assays and their respective statistical significance are shown in the online supplementary material (see online Supplemental Table 2). Ranking according to antibody levels for type 1 diabetes samples in assays measuring ZnT8A to the CR variant showed highly significant concordance with a Kendall W ranking agreement coefficient of 0.8538 (χ2 = 614.7, df = 48, P < 0.0001), whereas in control samples, the Kendall W was 0.2582 (χ2 = 278.3, df = 98, P < 0.0001). Laboratory-assigned designates for positivity were reported for 11 CR assays with a median sensitivity of 55% (IQR 48%–58%) and a median specificity of 98% (IQR 95%–98%). Interassay agreement for samples designated as ZnT8A positive was good, with an average pairwise percent agreement of 88.7% (AC1 = 0.77) in type 1 diabetes and 95.2% (AC1 = 0.94) in control samples.
Additional RBAs measured ZnT8A to the human ZnT8 COOH-terminal domain variant W325 (ROC-AUC = 0.775, AS95 = 61.2%), the NH2-terminal domain (ROC-AUC = 0.527, AS95 = 4%), and to the murine ZnT8 COOH-terminal domain R325 (ROC-AUC = 0.41, AS95 = 18.4%).
ZnT8A 2009 WORKSHOP
In 2009, results were reported for 24 assays measuring simultaneously autoantibodies to both polymorphic variants of the ZnT8 COOH-terminal domain, encoding arginine and tryptophan at residue 325 (CR-CW assays, see online Supplemental Table 3). Twenty-one assays measured autoantibodies only to the polymorphic variant containing an arginine at residue 325 (CR assays, see online Supplemental Table 4), and 18 assays measured autoantibodies only to the polymorphic variant containing a tryptophan at residue 325 (CW assays, see online Supplemental Table 5). Results for the 3 types of assay are summarized in Table 1.
PERFORMANCE, SENSITIVITY, AND SPECIFICITY OF CR-CW, CR, AND CW ASSAYS
We applied ROC analysis to assess the performance of ZnT8A assays (Table 1; also see online Supplemental Tables 3–5). CR-CW assays showed greater median ROC-AUCs (0.845, IQR 0.796–0.878) than CR assays (0.742, IQR 0.686–0.828, P = 0.03) or CW assays (0.636, IQR 0.606–0.745, P = 0.0004) (Fig. 1, B–D). ROC-AUC differences between individual assays and their statistical significance are shown in the online Supplemental Tables 6–8.
Median laboratory-assigned sensitivity was highest for CR-CW assays (63%, IQR 46%–68%) compared to CR assays (48%, IQR 32%–60%) or CW assays (34%, 28%–47%) (P = 0.0077) (Table 1). Consistent with this observation, the median AS95 of CR-CW assays (70%, IQR 60%–72%) was also greater than that of CR assays (51%, IQR 44%–63%, P = 0.03) or CW assays (42%, IQR 33%–52%, P = 0.001) (Table 1). For all types of ZnT8A assays, large interlaboratory variation was observed when ROC-AUC and AS95 results were considered (Fig. 2, B–D).
CONCORDANCE OF AUTOANTIBODY LEVELS IN CR-CW, CR, AND CW ASSAYS
Results expressed as local arbitrary units or indexes were reported for 51 assays. Concordance between laboratories for ranking of type 1 diabetes samples according to local units was highly significant for all ZnT8A assays (Kendall's W ranking agreement coefficient: CR-CW assays, W = 0.8633, Fig. 3A; CR assays, W = 0.7957, see online Supplemental Fig. 1A; CW assays, W = 0.724, see online Supplemental Fig. 2A). For 62 assays for which quantitative results were available, antibody levels were calculated as common ZnT8A arbitrary units. After conversion to common arbitrary units, the interassay concordance for ZnT8A titers assigned to each type 1 diabetes sample was less good (OCCC: CR-CW assays = 0.584, Fig. 3B; CR assays = 0.670, see online Supplemental Fig. 1B; CW assays = 0.656, see online Supplemental Fig. 2B). Concordance of both ranking and common arbitrary units between laboratories improved substantially when comparisons were restricted to assays within the upper 25th percentile of ROC-AUC (Table 1).
CONCORDANCE OF DESIGNATES FOR CR-CW, CR, AND CW ASSAYS
Laboratory-assigned designates for positivity/negativity were reported for 20 CR-CW assays, 18 CR assays, and 15 CW assays. Overall, there was high concordance of designates. Average pairwise percent agreement in type 1 diabetes samples was 80.7% for CR-CW assays (AC1 = 0.64; κ = 0.63), 80.3% for CR assays (AC1 = 0.53; κ = 0.52), and 83.0% for CW assays (AC1 = 0.68; κ = 0.63) (Table 1). In control samples, average pairwise percent agreement was 90.2% for CR-CW assays (AC1 = 0.90; κ = −0.018), 92.6% for CR assays (AC1 = 0.92; κ = −0.008), and 92.6% for CW assays (AC1 = 0.92; κ = −0.011) (Table 1). The observed difference between the agreement coefficients in control samples, i.e., negative κ coefficient in presence of high AC1, is a known paradox (25) that can signal the occurrence of a large overall number of discrepancies between different assays. Indeed, while 2 of the control samples were defined as ZnT8A positive by 5 and 3 CR-CW assays, and 18 (19%) samples were defined as positive by 2 assays, another 47 (50%) samples were each defined positive by only 1 assay, and just 28 (29.8%) control samples were defined as ZnT8A negative by all assays (Fig. 4). The strong association observed between ZnT8A levels and positive designates suggests that control samples defined positive by just 1 or 2 laboratories are likely to represent assay “noise” at low ZnT8A levels. In contrast, in type 1 diabetes samples, 11 (22%) were defined as ZnT8A positive by all 19 CR-CW assays, another 9 (18%) and 12 (24%) were defined as positive by >75% and >50% of assays, respectively, and 6 (12%) samples were defined as ZnT8A negative by all assays (Fig. 5A). When the analysis of designates was restricted to the CR-CW, CR, and CW assays with the highest and statistically not different ROC-AUCs, the average pairwise percent agreement, the agreement coefficient AC1, and the κ coefficient all increased in the type 1 diabetes samples. This was also true in control samples, with the exception of the κ coefficient, which showed a clear increase only for CR-CW assays (Table 1).
CR-CW ASSAY PERFORMANCE IN DETECTION OF EPITOPE-RESTRICTED ZNT8A
We compared designates for positivity/negativity to assess the performance of CR-CW assays in detecting ZnT8A restricted to specific ZnT8 polymorphic variants at the level of residue 325. In the DASP cohort, 8 type 1 diabetes samples were defined as ZnT8A positive by >70% of CR assays but <25% of CW assays, and 2 samples were defined as ZnT8A positive by all CW assays but <25% of CR assays, suggesting the presence of autoantibodies with restricted or dominant binding to ZnT8 aa325-dependent epitopes. All of these likely epitope-restricted sera were also called positive by >50% of CR-CW assays (Fig. 5).
The main findings of the first international ZnT8A DASP workshop confirm that ZnT8A constitute an important additional autoantibody marker of type 1 diabetes. ZnT8A showed a clear ability to discriminate health from disease and achieved a sensitivity and specificity similar to that of other established major type 1 diabetes–associated autoantibodies. With few exceptions, the majority of ZnT8A assays submitted to the workshop adopted the RBA format. The RBA is based on immunoprecipitation of in vitro transcribed and translated radiolabeled antigen and is the de facto gold standard format for the assessment of type 1 diabetes autoantibodies (16).
The measurement of ZnT8A brings its own specific challenges, however, owing to the multispanning transmembrane nature of this protein antigen. Indeed, in the original publication on ZnT8A discovery (3), it was described how recombinant antigens corresponding to separate NH2- and COOH-terminal ZnT8 domains outperformed the full-length protein in immunoassays, suggesting that full-length ZnT8 is unlikely to fold properly in the simple cell-free systems adopted for antigen expression by RBA.
In addition to the difficulty of expressing in vitro an antigen suitable for binding by all ZnT8A autoantibodies, a further degree of complexity is added by the presence of ZnT8 isoforms of the NH2-terminal domain resulting from differential splicing, and of polymorphic variants in the COOH-terminal domain that are targets of distinct and specific autoantibody responses (12). In the pilot 2007 workshop, the majority of ZnT8A RBAs used as antigen a ZnT8 COOH-terminal domain encoding for arginine at residue 325, the most frequent polymorphic variant worldwide. Assays reported in 2009 used a greater variety of antigens that, in addition to arginine, encoded the tryptophan amino acid at residue 325, the second most common polymorphism. These 2 ZnT8 polymorphic variants were used as antigens separately in CR and CW assays or in combination in the so-called CR-CW assays. CR-CW assays were based on immunoprecipitation of either postexpression mixtures of the 2 antigens or coexpressed chimeric COOH-terminal variants joined in tandem. Consistent with prior observations (13, 26), the ZnT8A workshop results highlighted the superior sensitivity of CR-CW assays measuring antibodies simultaneously to both arginine and tryptophan polymorphic variants compared to assays using either alone, a performance that was achieved without sacrificing specificity. A potential bias might have been introduced in both DASP workshops, because the case samples were mostly from white patients, whereas the controls included a sizeable proportion of black patients. The frequency of islet autoantibodies in black patients with type 1 diabetes is reportedly lower than in whites, and this might therefore have affected both sensitivity and specificity analyses. However, preliminary observations (J.C. Hutton and J.M. Wenzlau, unpublished findings) showed that the prevalence of ZnT8A in black patients with type 1 diabetes is only slightly lower than that found in whites, suggesting that ZnT8A measured using CR-CW constructs should be a useful and sensitive marker in other ethnic groups as well.
Overall, the observed patterns of reactivities in type 1 diabetes samples confirmed the capacity of CR-CW assays to measure antibodies simultaneously to both polymorphic variants of ZnT8 irrespective of their epitope restriction at the level of amino acid 325. However, a lower frequency of positive designates was observed for a few sera in CR-CW assays compared to assays that used the arginine 325 polymorphic variant alone as antigen. For instance, the IDS105 and IDS004 sera were found to be positive in 58% of CR-CW assays vs 78% of CR assays and in 62% of CR-CW assays vs 86% of CR assays. Together with the relatively low titer of these sera, this observation suggests that recognition of low-titer and epitope-restricted antibodies might be less efficient in CR-CW assays and critically dependent on good overall assay performance.
There was a remarkable interassay concordance of ranking according to titer and of positive designation in type 1 diabetes samples among CR-CW assays, especially those with the highest sensitivity and specificity. In contrast, in nondiabetic controls, overall concordance of designates was good, but interassay agreement was instead aleatory for the few control subjects designated positive, suggesting that at low ZnT8A titers, random fluctuations in assays might lead to false-positive calls.
Contrary to our expectations, there was a wider spread of titers between assays, in both type 1 diabetes and control sera, following conversion of reported results to common units, even in assays with similar performance. The reason for this remains to be explored, but the observed quantitative and qualitative interassay discrepancies are likely to be related to differences in ZnT8A assay protocols between laboratories. It is worth mentioning that even subtle details, such as the concentration of nonionic detergents (A. Williams, unpublished observations), as well as amino acid differences between the various chimeric or truncated recombinant ZnT8 proteins adopted as antigen in different laboratories, can profoundly affect antibody binding to ZnT8.
Despite lack of improvement in concordance following calculation of results in common units in the DASP 2009 workshop, we still believe that the introduction of a common international standard for ZnT8A will help to bring about greater agreement of results between laboratories. Unfortunately, the international WHO standard serum for antibodies to GAD65 and IA-2 (27, 28), which could have provided a ready source, is negative for ZnT8A antibodies (A. Williams and P. Achenbach, unpublished observations). Selection of a suitable ZnT8A reference serum may be further complicated by the need for large volumes of human sera in which different ZnT8A epitope reactivities should be present at high titer and in reasonably homogeneous proportions.
In keeping with a long tradition originally established by the Immunology of Diabetes Society Workshop, the DASP will continue to conduct a systematic comparison of ZnT8A assays to identify best practice and thereby further improve overall assay performance. These approaches, already successfully applied to other major type 1 diabetes autoantibodies, will surely prove beneficial in helping the scientific community reach a consensus on what assay formats and protocols can best answer the challenges of ZnT8A antibody measurement.
We are most grateful to the patients and clinicians who have donated the blood samples that enable the DASP workshops to take place.
↵8 Nonstandard abbreviations:
- Diabetes Antibody Standardization Program;
- zinc transporter 8;
- autoantibody to ZnT8;
- glutamic acid decarboxylase;
- islet antigen 2;
- area under the curve;
- Gwet coefficient of interrater agreement reliability;
- adjusted sensitivity at 95% specificity;
- overall concordance correlation coefficient;
- radio binding assay;
- interquartile range.
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the Disclosures of Potential Conflict of Interest form. Potential conflicts of interest:
Employment or Leadership: None declared.
Consultant or Advisory Role: None declared.
Stock Ownership: None declared.
Honoraria: None declared.
Research Funding: DASP is funded at CDC by PL105-33, 106-310, 106-554, and 107-360 administered by the NIH. J.C. Hutton, UC Denver DERC (NIH P30 DK57516), NIH R01 DK052068 and the Juvenile Diabetes Research Foundation Autoimmunity Center Consortium grant 4-2007-1056; P. Achenbach, Juvenile Diabetes Research Foundation (11-2005-1117); V. Lampasona, European Union (DIAPREPP Project, FP7-HEALTH-202013).
Expert Testimony: None declared.
Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript.
- Received for publication June 16, 2011.
- Accepted for publication September 27, 2011.
- © 2011 The American Association for Clinical Chemistry