Next generation sequencing (NGS)13 technology, also known as massively parallel sequencing (MPS), is being incorporated rapidly to clinical laboratory testing. Current applications include detection of germline variants in inherited diseases, somatic variants in cancers, subpopulations of circulating cell-free DNA, and single viral or microbial genomes in infections or metamicrobial genomes in normal or altered human flora. Each application is unique and has its advantages and disadvantages. The complexity of the NGS processes (specimen preparation and testing and analysis of large amounts of data) leads to challenges in validation, QC, and data interpretation beyond what clinical laboratories have previously encountered. In this Q&A, experts in each field (inherited diseases, cancer, infectious diseases, and pregnancy testing) respond to questions relevant to adapting this technology for clinical testing, and their responses represent their own viewpoints and discipline-specific experiences. The experts and their areas of interest are: Sherri Bale and Jill Hagenkord (inherited diseases); Richard Press, Carol Beadling, and Shashikant Kulkarni (oncology); Lynn Bry (infectious diseases); and Glenn Palomaki (noninvasive prenatal genetic testing of maternal plasma). The experiences and opinions they share demonstrate the diversity of the challenges and provide examples of how they are being addressed.
Validation: Analytical validation measures the performance characteristics of a clinical test for accuracy and reproducibility (precision). In genomic sequencing, analytical sensitivity describes the ability of the assay to detect genetic variants when present, and specificity defines the capacity to correctly identify a “normal” (wild-type) sequence. Although genomic sequencing technology is advancing, regions of the genome remain problematic.
1: What has been the most difficult aspect of validating an assay using MPS? How has the laboratory been able to meet these requirements?
Sherri Bale: The major problem in validating an assay using MPS is that it is an iterative process that involves optimizing 3 components at once—the sequencing platform, the specific test/panel of genes, and the bioinformatics pipeline. Each time one part of the system is “tweaked,” (for example changing the starting quantity of DNA in the target enrichment step, or adding/changing even a single primer in a pool of hundreds, or adjusting a parameter in the informatics pipeline), the entire end-to-end assay must be revalidated. Because these tests take days to weeks to complete, and are not trivial from the cost side, validation of a NextGen panel is time-consuming and expensive.
Jill Hagenkord: One of the most difficult aspects of validation has been finding samples with larger indels (10–100 bp) to confidently determine sensitivity and specificity for those variants. I've used DNA from well-characterized genomes (e.g., Catalog ID NA19240, NA12878) from the National Human Genome Research Institute (NHGRI) Sample Repository for Human Genetic Research, Human Variation Subcollection, maintained at the Coriell Institute. These genomes contain thousands of published variants, representing a range of variant types in both coding and noncoding regions of the genome. Genomes for related individuals are also available to allow Mendelian concordance calculations and phasing. I have also used samples from the National Institute for Biological Standards and Control (NIBSC). The validated variants in these genomes tend to be those that are easiest to detect and call by NGS, so we augment the validation with samples with known positives from a range of genetic disorders.
Richard Press and Carol Beadling: In our laboratory, which has been restricted to NGS of cancer-associated mutation hotspots, the most difficult assay validation challenge has been identifying “truth,” i.e., a gold standard set of samples and/or sequence data containing a broad heterogeneity of known sequence variants. Systematically adjusting various wet bench and data analysis pipeline parameters to optimize the generation of “correct” sequence calls from each iteration of successive analyses is straightforward, but, to identify which iteration provides the optimal analytical sensitivity/specificity, knowing the true call for each base in the target sequence is essential. Typically, the laboratory community has defined “truth” as the consensus (i.e., overwhelming majority) analytical result derived from a variety of different laboratories and methods. Given that, for NGS, the ultimate sequence call may vary depending on the sample type (formalin-fixed, paraffin-embedded [FFPE] vs fresh), intended sequencing targets (different targeted hotspots for each tumor type), library preparation method (PCR amplicon-based vs hybrid capture), and sequencing platform, defining a consensus of “true sequence” has become a formidable challenge.
Because of our large archive of cancer samples with confirmed mutations, we were able to determine the sensitivity and reproducibility of NGS-based detection for these common variants (and only these variants) in various tumor types. In addition, we were able to confirm the specificity of any new variants identified by NGS on an alternate platform (primarily Sanger sequencing). However, given the limited number of mutation hotspots previously tested, there was (and remains) no feasible way to rule out NGS-derived false-negative calls at other positions. Similarly, as we have expanded our targeted panels to include more genes, the potential for false-negative calls at previously untargeted positions remains a theoretical concern. One less-than-optimal solution is to determine the sensitivity, specificity, and reproducibility for distinct “classes” of mutations. For example, we pool the validation data for all single-nucleotide changes, all insertions (of a maximal size), and all deletions (of a maximal size), under the assumption that the sequencing chemistry and analysis pipeline will be equally efficacious at detecting all such mutations within each of these classes.
Shashikant Kulkarni: Deep sequencing with MPS is able to detect low-level (<5% allele frequency) variants. Gold standard orthogonal methods like Sanger sequencing are not able to verify these low-level variants.
Availability of positive control samples for validation was the most difficult problem to circumvent. We approached many peer laboratories for sample exchange, and finding positive controls for various genes was difficult.
When our laboratory started offering MPS testing in 2011, reference cell lines with a gold standard variant data set that laboratories offering MPS could use to establish the accuracy of their assay didn't exist. Since then, the GetRM (Genetic Reference Material Program) at the CDC and NIST have released whole genome reference data sets on several hapmap cell lines that could be used for assay validation. Recently, commercial entities have begun to offer well-characterized material prepared from cell lines in various sample formats (FFPE blocks, cell suspensions). These are also available in a range of variant mixtures to validate limits of detection.
Lynn Bry: Outside of viral genotyping assays, we don't have US Food and Drug Administration–approved platforms, kits, or computational tools for pathogen genome sequencing or metagenomic studies. Furthermore, we may not have reference genomes or curated content developed to a level to support CLIA-level assays.
Quality parameters/proficiency testing: Clinical laboratories monitor the performance characteristics of their assays. Genomic sequencing assays may require different or additional parameters to ensure the quality of the assays.
2: What quality parameters does your laboratory routinely monitor—for each patient or for each run? How are you meeting the requirement for proficiency testing or its alternative assessment?
Sherri Bale: For our targeted NextGen panels, we monitor sequencing quality, coverage depth, and the number of amplicons which require Sanger fill-in. For our whole exome sequencing (WES) tests, we monitor depth of coverage (mean coverage as well as percent of exome covered at 1×, 10×) heterozygous/homozygous ratios, and copy number variant (CNV) detection.
Proficiency testing (or alternative assessment) for panels is done by the QA (quality assurance)/QC department selecting samples from previous in-house clinical tests that represent a wide variety of genes and types of variants, and then sending them through the laboratory blinded to the fact that they are proficiency samples. For WES, we participate in sample swaps with a group of 5 WES labs, through the College of American Pathologists (CAP), and we assess consistency of single-nucleotide variation (SNV) identification.
Jill Hagenkord: For the patient quality parameters, we use DNA fingerprinting, gender check, expected genome behavior (such as heterozygous/homozygous ratio and transition/transversion ratio), contamination, and GC bias. For variant quality scores, we use allelic read percentages, and coverage. For run parameters, we use capture and alignment metrics.
DNA fingerprinting and gender checks can confirm the sample was not switched during processing. An overall contamination rate is calculated by assessing reference reads at the locations of all homozygous variant calls. An excess (beyond what would be expected from random error) in reference alleles indicates the presence of a second genome biasing the discordant calls. This methodology gives a good estimate of the rate of contamination for small amounts of contamination (<5%), but the measure is less accurate for higher levels of contamination. Increased heterozygous/homozygous ratios also indicate higher levels of contamination.
The human genome contains an expected range of the different types of genomic variants, although the expected range can vary by ethnicity, which can function as QC metrics. Typically, individuals of African descent have an increased variant diversity compared to those of Caucasian descent. Different expected ranges are needed for African and non-African genomes with regard to SNV/indel counts, heterozygous/homozygous ratios, and novelty rates. A consistent and specific value that can be used to measure call accuracy is the ratio of transition SNV to transversion SNV calls, which in human genomes should only vary over a very small range. A noticeable shift in that ratio indicates a systematic calling bias. Very low numbers of SNPs are an indication of a lack of sensitivity.
Tracking GC bias can provide information on the quality of the sample preparation and the evenness of coverage. Coverage can be measured along 2 dimensions: breadth, or how much of the genome is called, and depth, or how many overlapping reads make up the calls at each location. For measuring breadth, one can use percent selected bases. For measuring depth, one can use the overall count of mapped bases and the percentage of the targeted regions where the read depth is >20.
Richard Press and Carol Beadling: Initially, FFPE-derived (or fresh) DNA samples are quantified by fluorometer, as the A260/280 spectrophotometric measurements can overestimate DNA concentrations by 10-fold or more. We use PCR amplicon-based targeted sequencing panels, and the sequencing libraries are quantified by qPCR (quantitative real-time PCR) to verify optimal quantities of adaptor-modified templates before sequencing. After sequencing, we monitor average sequence read depth per amplicon, percentage of on-target reads, average read length, and read quality for each sample. Based on statistical estimates, 450 AQ20 reads are sufficient to detect a 5% variant allele. We acquire approximately 1500 average reads per amplicon (more for fresh leukemia samples), so that >95% of amplicons have >500 reads. The amplicon libraries typically generate >95% on-target reads. In addition, for every sample, we manually examine the sequenced amplicon size distribution for outliers. For proficiency testing, we have performed external sample exchanges. We also use internally blinded samples that have been tested on other (or the same) platforms, as well as commercially available (Coriell HapMap) reference samples.
Shashikant Kulkarni: Quality metrics are monitored for each patient and compared to reference averages: total reads, percent mapped to the genome, percent of mapped reads that are on target, number of total on-target reads, percent of on-target reads that are unique, mean mapping quality, percent of positions that are unique at 50×, 400×, and 100× (depth of coverage), and average unique coverage.
Lynn Bry: For infectious agents we look at standard quality parameters (Phred scores, etc.) for the sequence quality and coverage, and add known controls to each run to assess the performance of technical and bioinformatic processes within and across runs. We also evaluate the state of curated content being used in analyses––i.e., using reference sequences to generate a phylogenetic tree for placement of sequences from an unknown organism as part of a microbial identification, to detecting potential resistance genes and modifiers of known resistance genes. While the reference content for viruses such as HIV and HCV (hepatitis C virus) is good from more than a decade of clinical use, the depth and quality of content for other viruses, bacteria, and eukaryotic pathogens can be quite variable.
Outside of Sanger-based viral genotyping, proficiency testing programs in the US for NGS infectious diseases don't exist. In many cases we'll take reference organisms that have been genome sequenced, or generate defined communities for 16S rRNA gene phylotyping and metagenomic analyses. The Human Microbiome Project (HMP) created some test materials, and NIST is actively working on well-curated bacterial strains.
Informatics: Informatics has been essential to manage the amount of data generated and informatics pipelines have been developed either within the laboratory or externally by software development companies.
3: What do you perceive as requirements for informatics pipelines and what is your approach? How do you evaluate the quality of these pipelines?
Sherri Bale: We have developed much of the pipeline internally, although we also have integrated some open-source or licensed software. We have tested our pipelines by analyzing proficiency testing samples and against newly available (generally commercial) software by analyzing our data using β-versions of new software as it becomes available. We never launch our pipeline with upgraded or new software until a full validation (see Question 1) has been completed.
Richard Press and Carol Beadling: We evaluate the quality of the informatics pipelines by comparing sensitivity/specificity for detection of known variants, as well as the speed, and ease of integration of the pipeline into current laboratory work flows. New versions of any analysis algorithms are validated before clinical use by (a) an “in silico” reanalysis of archival sequencer-derived primary data files and (b) a “wet lab” resequencing of samples with known genotypes. A persisting challenge with these frequent validations of new (presumably improved) analysis pipelines remains our inability to definitely determine whether novel variants identified by the new pipeline, and not previously identified by other methodologies or prior pipeline versions, are actually true positives. New putative variants can be confirmed by other methods (usually Sanger sequencing), but only as other methods are available and feasible. New analysis pipeline versions usually result in a longer list of confirmed mutations compared to prior versions–particularly for insertion—deletion variants of progressively larger sizes.
Shashikant Kulkarni: MPS uses a complex computational work flow from aligning the sequence to reference genome to use of various algorithms and tools to detect variants. Separate computational tools are used for detection of translocations, small insertion–deletions, larger insertion–deletions, and CNVs, and furthermore, tools are frequently optimized for somatic vs germline variant detection. Any change to the entire analytical pipeline, whether it be a tool version change, parameter change, filter change, etc., is subject to assay revalidation or reverification depending upon the magnitude of the change (e.g., a different variant caller may require more extensive revalidation, whereas a minor upgrade may require reverification). After appropriate revalidation or reverification, the pipeline is versioned, locked down, and deployed for routine production.
Lynn Bry: Informatics can mean many things—in the context of clinical genomic testing for infectious agents the “Pathology Informatics” aspects relate to how one integrates genomic testing within the existing work flows in a ClinMicro and/or Molecular Diagnostics lab, as the laboratory information systems (LIS) need to have points where testing will include these complex molecular analyses, record the worklisting of samples for testing, receive a reportable result, integrate the molecular or genomics findings with other phenotypic information, and handle billing. While some of these items can be handled in the LIS, no vendor clinical systems currently manage the all bioinformatics, computational, and warehousing/storage of data.
Evaluating the quality of the bioinformatics pipelines depends upon the clinical or research question. One needs a full understanding of the strengths and weaknesses of the algorithms and overall methods. The bioinformatics sections of the CAP NGS checklist provide a general starting point for developing and managing pipelines, including ones for infectious disease testing. These items cover common requirements such as pipeline documentation, validation, developing a quality management program, and having full traceability of what was done on each sample/case, etc.
Challenges of different sample types:
4: What are the challenges for different sample types—for tumor tissue (sampling issues, preserved vs fresh tissue, etc.), for infectious diseases?
Richard Press and Carol Beadling: The primary challenge with FFPE cancer tissue is the variable quality, due to DNA fragmentation, and, with bone marrow samples, occasional deamination. We use a minimum NGS read depth of 100 average reads across the target to define coverage “failure.” We also flag samples as failed when there is evidence of deamination (a high frequency of C>T/G>A variants).
Shashikant Kulkarni: A guide slide can be made from the FFPE block to accurately mark the tumor to allow for macro dissection of tumor without surrounding tissue. This ensures enrichment of tumor DNA and helps to circumvent some problems related to tumor heterogeneity.
Lynn Bry: Samples for infectious disease testing can range from nucleic acid from a pure microbial isolate to fluids or tissues that have a low microbial burden relative to host cells, to nonsterile samples, such as stool, where the biomass of the commensal community can exceed that of a pathogen by orders of magnitude. In addition, we have to consider the range of percent GC contents, genome lengths, mobile genetic elements, and repetitive regions in analyses when selecting a methodology to detect microbial signatures.
Proper handling of materials before testing—particularly when considering FFPE material—is also essential. Opportunities abound for environmental contamination per handling at the point of collection and beyond. Thus, one always needs to be wary of potential environmental signatures, particularly if using broad metagenomic approaches, or even amplification of conserved targets such as the 16S rRNA.
Reporting and interpreting variant/gene significance: Clinical laboratories typically use a 5-tier system to classify variants (pathogenic, likely pathogenic, uncertain, likely benign, and benign), yet these classifications are only useful for genes known to be associated with disease. Although many somatic variants typically tested have been demonstrated to be important in disease progression or response to therapy, genomic sequencing applications will uncover novel variants.
5: How do you evaluate whether a variant or a gene in which a potential pathogenic variant is found is relevant to the disease phenotype? If your laboratory does mutation discovery for somatic variants, how do you interpret somatic variants not previously described?
Sherri Bale: We use the American College of Medical Genetics and Genomics (ACMG) guidelines to assess pathogenicity in genes known to be important in inherited disease. We modify those guidelines somewhat for genes that are known only to be associated with risk of disease development, but not at the level of Mendelian inheritance, in which case we report variants as “risk alleles.”
Jill Hagenkord: The gene-tiering system parallels the variant classification nomenclature: causes the disease, likely causes the disease, gene of uncertain significance (GUS), and likely does not cause the disease. Like variant classification, there is a still a bit of art to gene classification. Essentially, you can never have a pathogenic or likely pathogenic variant in a GUS … you can only have variants of uncertain significance (VUSs) in GUSs. When designing for gene panels, we try not to include GUSs, although market demand for the information, although uncertain, can lead us to include “edge case” genes in a panel which will increase our VUS rate. But we've found that physicians who order panels are comfortable explaining the uncertainty to their patients.
Richard Press and Carol Beadling: We perform somatic mutation testing and use public databases (COSMIC, My Cancer Genome, Leiden Open Variation Database, dbSNP, 1000 genomes), published literature (PubMed), and predictors of mutation consequences (SIFT, PolyPhen2) to interpret variants. We also query our own internal historical database of variants and tumors. The NGS read-determined allele frequency of a variant often gives us an additional clue as to tumor relevance—in comparison to the “tumor burden” as determined by morphology or flow cytometry. For example, if we see a novel variant at 50% allele burden but the morphologic tumor burden is only 20%, we would be more inclined to call that variant as “germline” and likely not tumor associated. In addition, for leukemia cases, where we often monitor posttreatment molecular responses by serial NGS studies, a comparison of pretreatment vs posttreatment variant allele frequencies is another useful clue as to pathogenicity. A posttreatment leukemia with a persistent 50% allele frequency of a previously “unknown” variant is better reclassified as a germline variant of no tumorigenic relevance, particularly if the leukemia burden has been significantly reduced as documented by other methods. Despite these available interpretive resources, we still very often discover variants that have not been previously described, and typically report these as being of “uncertain” pathogenic relevance to tumorigenesis.
Shashikant Kulkarni: Guidelines for somatic variant classification do not exist currently. We have modified current ACMG guidelines to specifically address tumor-specific issues.
Lynn Bry: Clinically, we focus on targets to aid in pathogen identification, prediction of susceptibility or resistance to therapy, or toxin production. In the research space, sequence plus computational methods are used to evaluate microbial community structure, dynamic changes in host ecosystems relative to a perturbation or disease state, or changes in microbial gene content, transcripts, and correlations with metabolites. The research is often to define mechanisms by which microbial communities contribute to a particular disease. This information helps define where such methods may have clinical utility. However, given the early state of things, we have no standards for how to report these latter aspects clinically.
Incidental findings: The ACMG has provided a minimum list of genes in which pathogenic variants are considered “actionable” when discovered incidentally and not related to the reason for testing.
6: What is your laboratory's process when an incidental variant is discovered? Are such variants searched for? How has your laboratory implemented returning of incidental findings (IF)? Do you go beyond the ACMG's recommendations?
Sherri Bale: We follow the current ACMG recommendations and gene list for reporting incidental findings in a patient on whom we have completed WES, by seeking and reporting known and expected pathogenic variants if the patient has not opted out from receiving them. If family members were also sequenced by WES as part of the proband's analysis, the IF identified in the proband are also reported in the family member if present. However, we do not assess each family member independently to identify the presence of IF that were not already found in the proband.
Jill Hagenkord: The physician and patient should work together to decide what type and how much preventative information any particular patient may want. The ability to opt in or opt out of certain types of preventative information should be available, rather than getting all or nothing.
A common approach to searching for pathogenic variants in preventative genes is to check against the various public databases, such as ClinVar and the Human Gene Mutation Database (HGMD). If reported as pathogenic, the evidence is then carefully reviewed to confirm that it has been appropriately classified and meets the laboratory's criteria for being pathogenic. Software can scan based on preset criteria for novel pathogenic variants which are also reviewed.
Including genes/variants outside the ACMG recommendations, such as factor V Leiden, HFE (hemochromatosis),14 and common carrier variants in autosomal recessive disorders (e.g., ACOG variants in the CFTR [cystic fibrosis transmembrane conductance regulator (ATP-binding cassette sub-family C, member 7)] gene) can be highly automated to search only for a list of variants and could be offered in a preventative screen.
Glenn Palomaki: Although this question is not directly relevant to laboratories offering circulating cell free (ccf) DNA testing of maternal plasma, the concept of incidental finding as part of prenatal screening is becoming more of an issue. With minor modifications to the bioinformatics pipeline, one creates the potential to identify prenatal findings that have not been readily approachable with previous screening technologies. Providing reliable fetal sexing and information about sex aneuploidies has been introduced by 4 US-based commercial laboratories as part of routine testing. However, the clinical validity studies published have been relatively small, and few discussions regarding benefits and harms have occurred. The newest targets of screening are the larger microdeletions such as 22p11 (DiGeorge) syndrome, although no peer-reviewed publications are yet available concerning performance, especially in the general population. Most of our information about the prevalence and range of symptoms regarding 22p11 are derived from studies of clinically affected individuals or from cohorts with abnormal findings, such as fetal heart abnormalities. As opposed to Down syndrome testing, where the natural history of that disorder is well known, no population-based studies of the prevalence of 22p11 syndrome have been reported. It is, therefore, possible that an unknown proportion of individuals with this genotype may have relatively minor phenotypic findings. Thus, laboratories offering whole genome/exome testing need to consider what should be reported, laboratories also need to consider what might not yet be ready to be routinely reported.
Richard Press and Carol Beadling: Although the ACMG incidental finding recommendations specifically exclude tumor samples, several of the genes on our panels are included in the ACMG “must report” list. Nevertheless, some mutations identified in cancer samples may indeed be germline, and some very small subsets of these germline variants may be oncogenically pathogenic. Although we often have some indirect clues as to the germline (vs somatic) nature of the variants that we identify (databases, allele frequency, persistence after treatment), the direct approach to this question—genotyping a matched nontumor sample from each patient—is not routinely undertaken in our laboratory. In the rare circumstance that we do discover a pathogenic mutation that may be germline, we include recommendations for genetic counseling and germline testing in the report at the discretion of the reporting pathologist. We fundamentally disagree with the ACMG's paternalistic opinion that the laboratory should always be obligated to report any such incidental genetic findings, particularly when the specific question being asked by the referring clinician/patient is related to cancer (as in our lab), and the associated phenotype of the germline mutation is not cancer related.
Shashikant Kulkarni: Occasionally, we observe potential germline cancer findings that are followed up in the patient and/or at-risk family members by evaluating an uninvolved germline sample.
Clinical validity/utility: Clinical laboratories have the responsibility to offer tests that have been demonstrated to be clinically valid (i.e., the gene is associated with the disease). However, payers also want these tests to show utility.
7: How do clinical laboratories ensure that genomic sequencing assays are clinically valid? How can clinical laboratories help others to understand the clinical utility of a test? From your experience, do you have general advice for ensuring (or improving) reimbursement?
Sherri Bale: Clinical laboratories should be prepared to show payers the evidence of clinical utility of the tests they offer. This would include multiple published papers in reputable journals where different patients/data sets are used. Payers need to understand the complexity of the tests, including the cost to develop, validate, perform, interpret, and report, so that they respect our requests for reimbursement. Whenever possible, host a decision maker from the payer's organization at your laboratory and let them see what goes into performing these tests.
Jill Hagenkord: Clinical utility has different definitions to different stakeholders. Payers tend to define clinical utility in terms of whether or not the test result will change management. However, in inherited disease testing, the utility of a test can vary case by case. We test certain genes at certain times for different reasons, so there isn't a nice one-size-fits-all answer to whether or not a specific gene has clinical utility.
Many genetic disorders don't have a curative or ameliorating therapy, but there is still great value in obtaining the correct diagnosis, anticipating disease course, having the opportunity to test at-risk family members, and understanding risk to future pregnancies—as well as the preventative information available through an incidental findings screen. The payers may not agree that this information meets criteria for clinical utility, but it has overt personal utility to the patients and their family members. And, with molecular CPT (current procedural terminology) codes that are now analyte specific and transparent, payers are increasingly denying claims for inherited disease testing because these tests don't meet their strict definition of clinical utility.
Lack of payer reimbursement to the doctors seeing these patients in their clinics or the laboratories for running the tests could catalyze a new consumer-driven model for accessing the information—similar to what Uber or Lyft has done in the taxi industry. We could see an “Uber-ization” of personal genetic information whereby the individual can obtain the same quality of information in a better, faster, cheaper way. As the cost of sequencing continues to fall consumers could obtain their genomic information and share it with their healthcare providers as needed over time. This evolution would reduce the tension between the payer-centric vs patient-centric definitions of clinical utility.
Richard Press and Carol Beadling: There are abundant published data and consensus professional society guidelines confirming the clinical benefits of tumor-based mutation profiling for directly informing cancer therapies and/or defining diagnostic or prognostic categories that indirectly affect therapy. As the number of these specific “actionable” gene mutations (and targeted therapies directed against these mutations) is rapidly proliferating, it is no longer a question of whether mutation profiling is necessary for optimal patient care (it clearly is), but rather a question of choosing the particular methods and the number of genes to be assessed. NGS affords the opportunity to evaluate many genes simultaneously at a cost comparable to performing only a few single-gene assays, and thus provides more actionable clinical information for a lower overall cost. The NGS-mediated sequencing of these additional actionable genes (at minimal extra cost) also allows the discovery of both (a) clinically relevant rare variants that would not be feasible with single-gene tests and (b) unexpected or rare drug resistance (or sensitivity) mutations that may directly inform therapy or allow enrollment in molecularly targeted clinical trials. It is essential to cast as broad a net as possible to detect rare variants, as these can directly affect therapy, and collectively they account for a substantial proportion of the total cases. Instead of using multiple single-gene assays to collectively test only a limited number of genes, it makes fiscal (and clinical) sense to use NGS to sequence broad target regions, and to use pipeline filters to reveal only the clinically relevant “actionable” genes for that specific tumor type. An additional benefit of such a “broad net” approach to tumor mutation profiling is the utilization of the identical NGS wet chemistry method on all tumors, regardless of tissue origin (but with a tumor-specific analysis pipeline).
Given the relatively low frequency of a specific mutation X in a specific tumor type Y, it is not feasible to expect, before a payer reimbursement decision, the generation of statistically significant, randomized, prospective clinical trial outcomes-based data (with linked health economics data) for each of the thousands of individual gene/cancer scenarios. Rather, payers should allow the use of a broader diversity of data on the correlation between gene mutations and cancer pathogenesis, evolution, diagnosis, prognosis, and treatment—and not necessarily tied to a specific tumor type—for making informed reimbursement decisions.
Shashikant Kulkarni: I think proper education of payers is immensely important to ensure there is a good understanding of the clinical validity and utility of the test. This can be accomplished by providing consensus statements generated by professional societies and multiple lines of evidence such as published papers in peer-reviewed journals, especially those focusing on patient outcome studies and cost-effectiveness of NGS testing.
Lynn Bry: We need a clear clinical reason for performing the test, and to delineate what medical actions will be driven by NGS results. For infectious disease testing, the clearest example would be transitioning viral genotyping by Sanger sequencing to NGS. The clinical indication remains clear. Further, an NGS method has potential to increase the sensitivity of detection of resistance, improve throughput, and, one hopes, reduce costs per test.
NGS otherwise needs to be considered with existing methods when evaluating detection of pathogens. A payer wouldn't cover costs for a pathogen genome when the same medically useful information can be obtained with standard microbiologic or molecular methods that are currently faster and cheaper to perform. Opportunities for pathogen genome or mutiplex target sequencing involve cases where we lack good assays, such as for slow-growing Mycobacteria or multiplex detection of different kinds of pathogens in a sample.
We have seen benefit from pathogen genome sequencing for infectious control efforts, but such testing is not tied to reimbursements. Rather, the case needs to be made to senior leadership of the benefits in using genomically informed methods to aid surveillance and infection control activities.
Clinicians'/patients' acceptance of the new technology:
8: What feedback have you received from clinicians or patients about these tests? Is there too much “hype” or has genomic sequencing performed to or exceeded expectations?
Sherri Bale: We target our marketing mostly to geneticists who are well versed in the literature. I don't think they are disappointed at all with the information they are getting from the NextGen targeted gene panels or WES.
Jill Hagenkord: There may be a tendency for nongenetic experts to think that we understand more about the genome than we do. It's the responsibility of the genetics experts in the clinics and laboratories to set expectations, which is being done responsibly. Physicians who order disorder-specific gene panels or exomes understand that they might get a faster answer than iterative testing, and they might catch a diagnosis that wasn't high on their original differential diagnosis, but they will likely get more VUSs and GUSs. Uncertainty is not unique to genetic testing.
Glenn Palomaki: Early attempts considered ccf DNA testing a noninvasive prenatal diagnostic (NIPD) test. This acronym has morphed into NIPT (testing) and NIPS (screening), causing more confusion as current serum/ultrasound screening also meets the definitions of NIPT/NIPS. Recommendations by professional organizations reinforced the test for current use only in women who were also being offered diagnostic testing due to being at high risk of a detectable condition. However, such testing was evaluated against invasive testing and karyotyping rather than against current screening technologies (combined or integrated testing). Some practitioners in the field were thus surprised when the occasional false-positive result appeared, rather than being impressed with the large proportion of women with high-risk pregnancies who were correctly provided with a greatly reduced risk estimate and avoided invasive procedures. Were the testing to have been applied in the general population instead, the higher performance would have been embraced as the real breakthrough it is. The most obvious barrier for general population screening is the high cost of such testing.
Richard Press and Carol Beadling: Although there is substantial hype for this new technology, the feedback we have received has been positive. As a result of our tumor mutation profiling, many cancer patients have either received effective clinical care–based therapies that they otherwise would not have received or have become eligible to enroll in clinical trials of new targeted therapies. Notably, many of these actionable mutations were in genes that would not have been routinely tested by single-gene assays. An additional (and popular) utility for this testing has been the serial monitoring (by NGS) of “molecular responses” after traditional (or targeted) therapy, particularly in myeloid leukemia patients. Given the high read depth afforded by NGS, sensitive and quantitative minimal residual disease detection can be accomplished, which then better informs treatment choices.
Shashikant Kulkarni: Our clinicians, especially clinical oncologists, are pleased that they finally have hope of finding potential targeted therapies or other changes in management of their cancer to offer for treatment of refractory patients. There seems to still be an unrealistic expectation of what this testing can provide amongst patients and clinical colleagues. As laboratory professionals, it is our utmost duty to convey its realistic potential. We need to clearly describe the technological platforms, methods of data analysis, established breakthroughs, and potential future clinical and research applications of this innovative and exciting technique. Finally, gathering evidence demonstrating the value and cost-savings advantages of sequencing is necessary.
Lynn Bry: More than 10 years ago, HIV genotyping dramatically improved our ability to direct HAART (highly active antiretroviral therapy). Though done by Sanger-based methods, it remains the best example of how a genomic approach dramatically benefitted patient care.
Otherwise, I remember universal WOWs! the first time we reviewed pathogen genomes, evaluated under a research protocol, with our colleagues in Infection Control. It shed new light on the direct causes of microbial resistance and also got us thinking how to effectively leverage this information in surveillance activities within and across institutions.
In the era of NGS, there's definitely a gee-whiz factor to using it for diagnosis. With the hype, we must ensure dialog continues on how things could be used as the field develops.
↵13 Nonstandard abbreviations:
- next generation sequencing;
- massively parallel sequencing;
- National Human Genome Research Institute;
- National Institute for Biological Standards and Control;
- formalin-fixed, paraffin-embedded;
- whole exome sequencing;
- copy number variant;
- College of American Pathologists;
- single-nucleotide variation;
- Human Microbiome Project;
- laboratory information system;
- American College of Medical Genetics and Genomics;
- gene of uncertain significance;
- variant of uncertain significance;
- incidental findings;
- Human Gene Mutation Database;
- circulating cell free;
- noninvasive prenatal diagnostic;
- NIP testing;
- NIP screening.
↵14 Human genes:
- cystic fibrosis transmembrane conductance regulator (ATP-binding cassette sub-family C, member 7).
Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 3 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; and (c) final approval of the published article.
Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest:
Employment or Leadership: E. Lyon, University of Utah and Association for Molecular Pathology; F. Cockerill, Mayo Medical Laboratories, Quest Diagnostics (10/1/14), and guest editor, Clinical Chemistry, AACC; J. Hagenkord, 23andMe.
Consultant or Advisory Role: E. Lyon, Complete Genomics and Health Advantages; F. Cockerill, Roche Diagnostics; S. Kulkarni, Swift Biosciences and Bina Technologies.
Stock Ownership: S. Bale, BioReference Laboratories; J. Hagenkord, Invitae and 23andMe.
Honoraria: S. Kulkarni, Novartis, Affymetrix, and Agilent; R. Press, Life Technologies.
Research Funding: E. Lyon, institutional funding from the National Human Genome Research Institute; L. Bry, NIH; G. Palomaki, institutional funding from Natera, Inc.
Expert Testimony: None declared.
Patents: None declared.
Other Remuneration: F. Cockerill, royalties from Roche Diagnostics.
- Received for publication August 14, 2014.
- Accepted for publication September 9, 2014.
- © 2014 American Association for Clinical Chemistry