I Will Continue to Work With the Acmg Commette
Am J Hum Genet. 2020 Nov 5; 107(5): 932–941.
Variant Classification Concordance using the ACMG-AMP Variant Interpretation Guidelines across Nine Genomic Implementation Research Studies
Laura M. Amendola
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
Kathleen Muenzen
2Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA 98195, USA
Leslie G. Biesecker
3Medical Genomics and Metabolic Genetics Branch, National Human Genome Research Institute, NIH, Bethesda, MD 20892, USA
Kevin M. Bowling
4Hudson Alpha Institute for Biotechnology, Huntsville, AL 35806, USA
Greg M. Cooper
4Hudson Alpha Institute for Biotechnology, Huntsville, AL 35806, USA
Michael O. Dorschner
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
Catherine Driscoll
3Medical Genomics and Metabolic Genetics Branch, National Human Genome Research Institute, NIH, Bethesda, MD 20892, USA
Ann Katherine M. Foreman
5Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
Katie Golden-Grant
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
John M. Greally
6Albert Einstein College of Medicine, Bronx, NY 10461, USA
Lucia Hindorff
7Division of Genomic Medicine, National Human Genome Research Institute, NIH, Bethesda, MD 20892, USA
Dona Kanavy
5Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
Vaidehi Jobanputra
8New York Genome Center, New York, NY 10013, USA
9Columbia University Medical Center, New York, NY 10032, USA
Jennifer J. Johnston
3Medical Genomics and Metabolic Genetics Branch, National Human Genome Research Institute, NIH, Bethesda, MD 20892, USA
Eimear E. Kenny
10Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
Shannon McNulty
5Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
Priyanka Murali
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
Jeffrey Ou
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
Bradford C. Powell
5Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
Heidi L. Rehm
11Massachusetts General Hospital and the Broad Institute of MIT and Harvard, Boston, MA 02142, USA
Bradley Rolf
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
Tamara S. Roman
5Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
Jessica Van Ziffle
12Department of Pathology, Institute for Human Genetics, University of California, San Francisco, San Francisco, CA 94143, USA
Saurav Guha
8New York Genome Center, New York, NY 10013, USA
Avinash Abhyankar
8New York Genome Center, New York, NY 10013, USA
David Crosslin
2Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA 98195, USA
Eric Venner
13Human Genome Sequencing Center, Department of Molecular and Human Genetics, Baylor College of Medicine, Houston, TX 77030, USA
Bo Yuan
14Department of Molecular and Human Genetics, Baylor College of Medicine, Baylor Genetics, Houston, TX 77030, USA
Hana Zouk
15Department of Pathology, Massachusetts General Hospital, Harvard Medical School and Laboratory for Molecular Medicine, Partners HealthCare Personalized Medicine, Boston, MA 02139, USA
Gail P. Jarvik
1Department of Medicine, Division of Medical Genetics, University of Washington Medical Center, Seattle, WA 98195, USA
Received 2020 Aug 3; Accepted 2020 Sep 29.
- Supplementary Materials
-
Table S1. Variant Set Details, Classifications across Laboratories, and Concordance
GUID: AA661EBD-A68F-4768-A49F-497316A27E02
Document S2. Article plus Supplemental Information
GUID: 048DD504-A5F7-4195-9673-1F3AD638D8FF
- Data Availability Statement
-
The published article includes all datasets generated or analyzed during the study.
Summary
Harmonization of variant pathogenicity classification across laboratories is important for advancing clinical genomics. The two CLIA-accredited Electronic Medical Record and Genomics Network sequencing centers and the six CLIA-accredited laboratories and one research laboratory performing genome or exome sequencing in the Clinical Sequencing Evidence-Generating Research Consortium collaborated to explore current sources of discordance in classification. Eight laboratories each submitted 20 classified variants in the ACMG secondary finding v.2.0 genes. After removing duplicates, each of the 158 variants was annotated and independently classified by two additional laboratories using the ACMG-AMP guidelines. Overall concordance across three laboratories was assessed and discordant variants were reviewed via teleconference and email. The submitted variant set included 28 P/LP variants, 96 VUS, and 34 LB/B variants, mostly in cancer (40%) and cardiac (27%) risk genes. Eighty-six (54%) variants reached complete five-category (i.e., P, LP, VUS, LB, B) concordance, and 17 (11%) had a discordance that could affect clinical recommendations (P/LP versus VUS/LB/B). 21% and 63% of variants submitted as P and LP, respectively, were discordant with VUS. Of the 54 originally discordant variants that underwent further review, 32 reached agreement, for a post-review concordance rate of 84% (118/140 variants). This project provides an updated estimate of variant concordance, identifies considerations for LP classified variants, and highlights ongoing sources of discordance. Continued and increased sharing of variant classifications and evidence across laboratories, and the ongoing work of ClinGen to provide general as well as gene- and disease-specific guidance, will lead to continued increases in concordance.
Keywords: germline variant classification, ACMG-AMP recommendations, genomic implementation
Introduction
Accurate variant pathogenicity classification facilitates delivery of genomic medicine at a high standard of practice. Disagreements on variant pathogenicity classifications exist across laboratories, as well as between laboratories and clinical providers.1 This creates challenges for risk assessment, medical management recommendations, and cascade screening. Though disagreement on the clinical significance of germline sequence variants can occasionally occur for valid reasons, studies of differences in variant interpretation have shown that consensus-building activities and data sharing can improve variant classification concordance and highlight areas for which additional guidance will improve consistency.
In 2015, the American College of Medical Genetics and Genomics (ACMG) and Association of Molecular Pathology (AMP) published guidelines for the interpretation of germline sequence variation.2 These guidelines have been widely adopted by clinical laboratories both in the US and internationally, with some variation in their implementation.3 Academic groups, commercial laboratories, and the National Institutes of Health-funded Clinical Genome Resource (ClinGen) have offered tools and guidance to increase consistent use of the ACMG-AMP evidence codes in both general and specific gene-disease contexts4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 (see ClinGen in Web Resources). More recent estimates of variant classification concordance show a trend toward a decrease in clinically relevant disagreements,19 , 20 , 21 ; and large-scale efforts to resolve variant classification differences in the ClinVar public archive have been undertaken.22 Nonetheless, disagreements that may affect patient care continue.23 , 24
Shortly after the publication of the ACMG-AMP guidance on germline variant interpretation, nine laboratories in the Clinical Sequencing Exploratory Research (CSER)-phase one Consortium evaluated the implementation of this guidance for a set of 99 variants25 in any genes thought to be associated with Mendelian disease. The overall concordance rate before consensus building activities in this initial work was only 34% across laboratories. However, that result underestimated the general concordance for genomic variant interpretation because the variant set was enriched for difficult-to-classify variants and included genes for which some participating laboratories did not have extensive interpretive experience. Nonetheless, this project identified areas of the ACMG-AMP guidelines that would benefit from clarification and increased specification. Recommendations included the development of disease-specific allele frequency cut-offs and quantitative thresholds for and against segregation, and developing lists of "well-established" functional assays and genes where loss of function (LoF) is a known mechanism of disease.
In 2013, the ACMG recommended the return of "known pathogenic" and "expected pathogenic" (predicted LoF) variants in 56 genes associated with medically actionable conditions as incidental (secondary) findings to individuals with sequenced exomes or genomes.26 This list was updated in 2017 to include 59 genes,27 and additional changes to the list of gene-disease pairs are expected soon. It is also anticipated that the ACMG will endorse the return of likely pathogenic (LP) and pathogenic (P) variants, replacing the obsolete descriptors of "known pathogenic" and "expected pathogenic."2 The practice of obtaining consent for and reporting secondary findings has been incorporated into clinical and laboratory processes, though variation in the types of returned findings and consent processes exist.28 The frequency of identifying actionable secondary findings in individuals undergoing genomic sequencing has been estimated to be between 2% and 4%.29 , 30 , 31 As access to genomic sequencing expands, an increasing number of patients and research participants will have secondary findings identified and returned, expanding the benefits and challenges of identifying these actionable variants for patient populations and health systems.
In this work, the two CLIA-certified Electronic Medical Records and Genomics (eMERGE) Network sequencing centers and the six CLIA-certified clinical laboratories and one research laboratory performing genomic sequencing in the Clinical Sequencing Evidence-generating Research (CSER) Consortium collaborated to provide an updated concordance rate for germline sequence variants interpreted in a genomic implementation study setting. Importantly, variants were not selected based on a high degree of difficulty to classify. We focused on variants in the 59 genes on the ACMG secondary finding v.2.0 list27 for two reasons. First, given that all clinical genomics laboratories interpret variants in these genes, doing so removes different laboratory gene level expertise as a source of discordance. Second, in the case of secondary findings, the individual with an identified variant is less likely to display the associated phenotype than in a diagnostic setting, and thoughtful interpretation is essential to avoid over- or under-diagnosis. Importantly, variants of uncertain significance (VUS) are generally not returned as secondary findings, so the distinction between LP and VUS is critical. Concordance rates were calculated, and reasons for discordance were summarized with the goal of identifying further opportunities for harmonization in germline sequencing variant interpretation and classification across laboratories.
Material and Methods
Variant Set Creation and Distribution
Eight laboratories submitted 20 classified variants in the ACMG secondary finding v.2.0 genes identified in research participants from the CSER Consortium and eMERGE Network. The ninth participating laboratory did not submit classified variants due to bandwidth issues when the variant set was created. To ensure the variants were not over selected for complexity of classification, laboratories were asked to select the variant in an ACMG secondary finding v.2.0 gene at the top of a list of prioritized variants for further review and/or reporting in their most recently annotated exomes or genomes until they reached a total of 20 variants. Variants in prioritized variant lists across laboratories were not ranked by complexity or likelihood of reporting. For each variant, laboratories shared the chromosomal location, reference and alternate allele, transcript, amino acid change, and the race/ethnicity of the participant in which the variant was identified. Each variant entry also included the classification (P, LP, VUS, likely benign [LB], or benign [B]), and ACMG-AMP evidence codes and weight applied by the submitting laboratory for the disease paired with that gene on the ACMG secondary finding v.2.0 list. Laboratories were asked to interpret and classify variants as they would in their current workflow, including incorporating clinical judgement and available gene-specific and evidence code guidance.
Variants were uploaded into a custom REDCap32 database. The complete set of variants was distributed randomly across all laboratories such that each variant was annotated and classified by an additional two laboratories, who were blinded to the classification and evidence codes applied by the submitting lab. The two additional laboratories were asked to classify each variant for the specific ACMG secondary finding v.2.0 gene-disease pair. This resulted in three independent classifications with accompanying evidence codes and weights for each variant.
Overall Concordance Evaluation
The overall complete five-category concordance rate (i.e., all three laboratories classified the variant in the same exact category) among laboratories for each variant was assessed. Given that typically the same clinical recommendations are made for variants classified as P and LP or B and LB, clinically meaningful discordance (P/LP versus VUS versus B/LB) was also explored. The complete five-category concordance rate for variants found in self-identified white/European American participants versus all other self-identified groups, including Hispanic participants, was evaluated using a chi-square test with a statistically significant threshold of p < 0.05. We tested for differences in the rate of complete five-category concordance by the submitted pathogenicity classification, hypothesizing that more confident classifications (P and B) would have better concordance than LP and LB or VUS. We also explored the rate of complete five-category concordance by variant type (missense versus stop-gain/frameshift versus splice). Additionally, we tested the hypothesis that concordance rate varied by cancer, cardiac, or lipid disease categories using chi-square tests, each with four degrees of freedom.
Discordant Variant Discussions
Contact representatives from each of the nine laboratories were identified and included individuals with backgrounds and expertise in genetic counseling, clinical informatics, and clinical molecular genetics. Variants with three discordant classifications (i.e., three laboratories submitted three different classifications) were presented and discussed via five teleconference calls between November 2019 and January 2020. Teleconferences were scheduled so that a representative from each of the nine participating laboratories could attend. Three of these discussions took place during approximately 20 min allotted to this project on the CSER consortium Sequencing and Diagnostic Yield working group monthly call, and two took place during ad hoc 60-min calls. The contact representative from one of the three laboratories that classified each discordant variant presented the variant details and evidence codes and weights applied by all three laboratories, followed by discussion. Reasons for discordance and agreement were noted by a genetic counselor during variant discussions.
For variants where one laboratory disagreed with the other two, the contact representative from the discordant laboratory was sent their submitted classification, evidence codes, and weights and conducted a preliminary review to verify that this information was correct. For variants that remained discordant, the discordant laboratory contact representative was then emailed all classifications and evidence codes and weights from the other two laboratories and asked if their laboratory group wished to modify their call and to provide the reasoning for their decision. Laboratories were also asked to note the perceived source of the discordance. Variants that remained P/LP versus VUS discordant after email review by the discordant laboratory were discussed on a final teleconference call in April 2020 attended by at least one representative from each of the nine laboratories. As with the teleconference calls for variants with three discordant classifications, the contact representative from the laboratory that classified the discordant variant presented the variant details and evidence codes and weights applied by all three laboratories, and each presentation was followed by discussion.
Variants with initial discordance limited to LB versus B classifications were not explored further based on time constraints and the lack of clinical significance of this level of disagreement. After discordant variant discussions were complete, we explored whether resolved variants moved toward VUS or toward P/LP or LB/B.
Results
Submitted Variant Set
Eight laboratories submitted 20 variants each, resulting in a set of 158 unique variant classifications after duplicates were removed. More than half (96/158, 61%) of the submitted variants were classified by the submitting laboratory as VUS, 18% were classified as P (n = 20) or LP (n = 8), and 21% were classified as LB (n = 31) or B (n = 3). Most submitted variants were in cancer risk (APC [MIM: 611731], BMPR1A [MIM: 601299], BRCA1 [MIM: 113705], BRCA2 [MIM: 600185], MEN1 [MIM: 613733], MLH1 [MIM:120436], MSH2 [MIM: 609309], MSH6 [MIM: 600678], MUTYH [MIM: 604933], PMS2 [MIM: 600259], PTEN [MIM: 601728], RET [MIM: 164761], SDHAF2 [MIM: 613019], SDHB [MIM: 185470], SDHD [MIM: 602690], TP53 [MIM: 191170], VHL [MIM: 608537]; 40%) and cardiac risk (DSC2 [MIM: 125645], DSG2 [MIM: 125671], DSP [MIM: 125647], KCNH2 [MIM: 152427], KCNQ1 [MIM: 607542], LMNA [MIM: 150330], MYBPC3 [MIM: 600958], MYH11 [MIM:160745], MYH7 [MIM: 160760], RYR2 [MIM: 180902], SCN5A [MIM: 600163], TNNI3 [MIM: 191044]; 27%) genes; variants in lipid condition (APOB [MIM: 107730], LDLR [MIM: 606945], PCSK9 [MIM: 607786]; 10%) and vascular condition (COL3A1 [MIM:120180], FBN1 [MIM: 134797], SMAD3 [MIM: 603109]; 7%) associated genes, as well as other genes (ATP7B [MIM: 606882] [9], CACNA1S [MIM: 114208] [2], GLA [MIM: 300644] [1], NF2 [MIM: 607379] [1], OTC [MIM: 300461] [2], RYR1 [MIM: 180901] [9], and TSC2 [MIM:191092] [5]) were also submitted. The majority of submitted variants (145/158, 92%) were in genes associated with conditions inherited in an autosomal-dominant pattern. Eleven variants (7%) and 3 variants (2%) were in genes associated with conditions inherited in an autosomal-recessive and X-linked pattern, respectively. Most submitted variants were missense (84%), followed by stop gain or frameshift (10%) and splice (5%). One variant was non-coding and 1 was synonymous.
The set included variants identified across participant-reported racial and ethnic groups including white/European American (42%), Black/African American (30%), Hispanic/Latinx (16%), Asian (5%), American Indian/Native American/Alaska Native (2%), and Middle Eastern/North African/Mediterranean (1%) participants. Four percent of variants were identified in participants with unknown or unreported race/ethnicity.
Original Concordance
Overall complete five-category concordance across the three laboratories classifying each variant was found for 86 of 158 variants (54%). Concordance did not necessarily reflect the application of the same ACMG-AMP codes across laboratories for a given variant. When grouping P and LP classifications, and B and LB classifications, the clinically meaningful concordance rate was 111/158 (70%). Seventeen variant classifications (11%) had a discordance that would affect patient recommendations (P/LP versus VUS) and 30 variant classifications (19%) were discordant between LB or B and VUS (Table S1). No variant had a discordance across the classifications of P/LP and LB/B.
The rate of complete five-category concordance did not differ based on participant-reported race/ethnicity (32/66 in white/European American versus 54/92 in other groups, chi-square test p = 0.20). The number of variants concordant and discordant with the initial laboratory classifications is presented in Table 1. The rate of complete five-category concordance differed by initial laboratory classification (chi-square test, p < 0.00001); it was highest for variants submitted as VUS (69/96, 72%), followed by B and P (12/22, 55%), and LP and LB (5/40, 13%). The complete five-category concordance rates were 58% for predicted missense variants (77/133), 53% for stop gain or frameshift variants (8/15), and 13% for splice variants (1/8).
Table 1
Number of Variants Concordant and Discordant with Initial Laboratory Classifications
| Initial Laboratory Classification | |||||
|---|---|---|---|---|---|
| P | LP | VUS | LB | B | |
| Classification by Other Two Laboratories | |||||
| | |||||
| P | 10 | 2 | 3 | 0 | 0 |
| LP | 5 | 1 | 5 | 0 | 0 |
| VUS | 4 | 5 | 69 | 11a | 0 |
| LB | 0 | 0 | 14 | 4 | 1 |
| B | 0 | 0 | 5 | 17 | 2 |
| Total Variants | 19 | 8 | 96 | 32 | 3 |
| Concordance Rate | 53% | 13% | 72% | 13% | 67% |
The complete five-category concordance rate did not differ between genes grouped by cancer, cardiac, or lipid phenotype (chi-square test, p = 54). The concordance rate was 31/60 (52%) for variants in cancer genes; 8 of these 31 variants were submitted as P, 18 were submitted as VUS, and 5 were submitted as LB or B. The concordance rate was 26/42 (62%) for cardiac genes; 3 of these 26 were submitted as P or LP and 23 were submitted as VUS. The concordance rate was 8/16 (50%) for variants in lipid genes, 7/11 (64%) for variants in vascular genes, and 14/29 (48%) for variants in genes associated with other conditions. The number of variants in each disease-association category are presented in Table 2.
Table 2
Number of Submitted Variants by Disease-Association Category
| Associated Condition | |||||
|---|---|---|---|---|---|
| Cancer | Cardiac | Lipid | Vascular | Other | |
| Concordant (n, %) | 31 (52) | 26 (62) | 8 (50) | 7 (64) | 14 (48) |
| Discordant (n, %) | 29 (48) | 16 (38) | 8 (50) | 4 (36) | 15 (52) |
| Total (n) | 60 | 42 | 16 | 11 | 29 |
Eight variants had a discordance between P and VUS, and nine had a discordance between LP and VUS. Of the 17 variants with a P or LP versus VUS discordance, 8 (47%) were in cancer risk genes, 5 (29%) were in cardiac risk genes, and 4 (24%) were in genes associated with other conditions.
Discordant Variant Discussions and Outcomes
The approach to and outcomes of discordant variant classification review are presented in Figure 1. The set of 158 variants with initial and final concordant and discordant classifications are included in Table S1.
Discordant Variant Discussion Process and Outcomes
Of the 72 discordant classifications, 18 involved disagreement between LB and B and were not discussed further. Complete five-category concordance was reached for 8 of the 13 variants (62%) with disagreement across all three classifying laboratories that were discussed via teleconference calls. Five of the 41 variants where one laboratory disagreed with the other two reached complete five-category concordance after preliminary review to identify data entry errors. Complete five-category concordance was reached for 17 of the remaining 36 (47%) variants reviewed via email by the discordant laboratory. The proportion of variants resolved did not differ significantly between the email or conference call review format (chi-square test, p = 0.33). Complete five-category concordance was reached on a final teleconference call for 2 of the 7 variants with a remaining discordance of P or LP versus VUS after email review. One of these 2 resolved variants had been interpreted by two laboratories for the wrong (non-ACMG secondary finding v.2.0) gene-disease pair, and for 1 variant the laboratory revised their classification based on clinical information available after their original classification.
Four of 5 (80%) splice variants, 5 of 7 (71%) stop gain or frameshift variants, and 25 of 45 (56%) missense variants that were originally discordant and discussed via email or conference call reached complete five-category concordance. All 5 variants with a discordance between P versus LP that reached five-category concordance were classified as P. Approximately half (5/9, 56%) of variants with a P or LP versus VUS discordance that reached five-category concordance were classified as VUS. Variants discordant between LB or B versus VUS that reached five-category concordance were most often classified as B or LB (13/16, 81%). The classifications of discordant variants that reached complete five-category concordance after discordance discussions are presented by range of initial discordance in Table 3.
Table 3
Final Classification of Discordant Variants that Reached Complete Five-Category Concordance by Initial Range of Discordance
| Range of Discordance | Number of Variants | Final Classification |
|---|---|---|
| P/LP | 5 | 5 P; 0 LP |
| P/VUS | 5 | 2 P; 0 LP; 3 VUS |
| LP/VUS | 4 | 2 LP; 2 VUS |
| LB/VUS | 11 | 3 VUS; 8 LB |
| B/VUS | 5 | 0 VUS; 3 LB; 2 B |
Recurrent reasons for discordance were identified during variant discussions and email review. The perceived source(s) of discordance were noted by the discordant laboratory for 33 of the 36 variants reviewed via email. Common reasons for discordance identified during teleconference and email review are presented in Figure 2. More than one source of discordance was noted for several variants. For example, for 5 variants both lab protocol variation and differences in the application of ACMG-AMP evidence codes contributed to discordance.
Common Reasons for Variant Classification Discordance
The most common reason for discordance was different applications of evidence codes, which was involved for 33 variants. Several evidence codes applied differently for multiple variants were discussed. For 3 variants, laboratories discussed discordant applications of the moderate-strength PM1 evidence code, related to the variant being located in a mutational hotspot and/or critical or established functional domain with a lack of benign variation. There was variation in approaches and definitions of a hotspot and what constituted benign variation in a region across laboratories. The discordant application of the strong PS4 evidence code, related to the prevalence of a variant in affected individuals versus control subjects, was discussed for 6 variants. Laboratories had different thresholds for the number of affected probands necessary to apply this evidence code at all, or as a supporting versus strong level of evidence. Laboratories also considered whether applying the PM2 evidence code, related to the absence of a variant in control populations, is redundant when the PS4 case versus control evidence code is also applied. Finally, the discordant application of the strong PS3 evidence code, related to functional studies supporting pathogenicity, was discussed for 4 variants. Laboratories expressed varying levels of familiarity with gold standard or well-established assays for different genes and discussed whether benign or pathogenic control subjects need to be included to accurately interpret the results of a functional assay. ClinGen guidance regarding the application of functional evidence codes when assessing germline variant pathogenicity was published after laboratories had submitted their variant classifications7 and was thus available for consultation during discordance discussions.
Variation in laboratory protocols for applying evidence was involved for 9 of the variants. For example, one laboratory incorporates an option suggested by ClinGen expert panels11 and obtains specific prevalence and penetrance data for the associated disease to upgrade to the stand-alone evidence code that a variant has a minor allele frequency above 5% (BA1) if the allele frequency is too high to be pathogenic for the associated disease, regardless of whether it meets the 5% cut off. Another laboratory always downgraded the strong evidence code related to functional studies supporting pathogenicity (PS3) to moderate strength if the published literature only includes in vitro studies.
For 2 variants, ClinGen guidance regarding the appropriate application of an evidence code was not used consistently by all three laboratories during their original classification. Not yet publicly available ClinGen guidance was discussed for 3 variants. Discordance for 5 variants involved errors in data entry. For 4 variants, an error in the nomenclature of the variant by the non-submitting laboratory contributed to discordance in classification, and 1 variant was flagged incorrectly as in a splice region by the annotation software of one non-submitting lab. Three variants were classified by at least one of the three laboratories for the wrong gene-disease pair. When discordance caused only by errors introduced by activities specific to this project were removed, complete five-category concordance increased to 99/158 (63%) variants, and the clinically meaningful concordance rate increased to 120/158 (76%) variants.
Discussion
This project provides an updated estimate of germline variant concordance across genomic sequencing laboratories and highlights considerations to further improve consistency in interpretation and classification of germline variation. Suggestions to support continued improvement in germline variant classification concordance are presented in Table 4.
Table 4
Suggestions to Support Continued Improvement in Germline Variant Classification and Return
| Goal | Suggestions |
|---|---|
| Increase concordance in interpretation and classification of germline variation | Additional guidance to increase consistent application of ACMG-AMP evidence codes (ex. PM1 and PS4, especially in the context of applying PM2) |
| Increase laboratory awareness and consistent use of available resources (ex. ClinGen guidance) | |
| Continue and increase sharing of gene-disease specific expertise and rationale for applying evidence across laboratories | |
| Provide clinically meaningful results to patients and their families | Ongoing work to support the interpretation and classification of uncertain genomic variation |
| Contextualize laboratory results in the context of the personal and family history of the individual (especially LP variants returned as secondary findings) |
The overall complete five-category concordance rate for germline sequence variants across the nine participating genomic implementation studies was 54%, with a clinically meaningful concordance rate of 70%. The concordance rate was not different based on the race/ethnicity group of the participant the variant was identified in, or based on the associated phenotype category. Additionally, five-category and clinically meaningful concordance increased to 63% and 76%, respectively, after removing errors introduced by activities specific to this project. It is likely that familiarity with the genes represented in the variant set, the unbiased approach to selecting variants, experience implementing the ACMG-AMP guidelines, and the development of supporting guidance and clarification over the past several years contributed to this higher rate compared to the original CSER consortium initial overall and clinically meaningful concordance estimates of 34% and 59%, respectively, from 2015.25 Importantly, there were no variants that had a discordance ranging across the P/LP and LB/B classifications in this project, compared to 5% of variants from the original CSER consortium study.
Laboratories were most likely to agree on the classification of variants submitted as VUS, where there was commonly a lack of prior observations of the variant to provide evidence supporting either a B or P classification. Though concordance was highest for this group, the large number of variants that remained of uncertain significance highlights the importance of ongoing work to support the interpretation and classification of uncertain genomic variation as disease causing or benign. The lower concordance in other groups, particularly those submitted as LP or LB, highlights continued challenges agreeing on the classification of variants where more, and sometimes conflicting, evidence that suggests or refutes pathogenicity exists. In this project, varying application of ACMG-AMP evidence codes across laboratories was the most common reason contributing to discordance. Additional guidance may be helpful to increase consistent application of the PM1 evidence code related to the variant being located in a mutational hotspot and the PS4 evidence code regarding case-control data, especially in the context of applying PM2 for variants absent from control populations. The ongoing work of ClinGen (see ClinGen's Sequence Variant Interpretation working group website in Web Resources) and related efforts will provide valuable guidance for evidence codes, such as those mentioned above, that continue to be inconsistently applied.
It is important for laboratories to be aware of and use available resources. As stated above, ClinGen has published both gene-specific and evidence code-specific guidance, which was not used consistently or applied the same way across laboratories for several variants in this project. For example, ClinGen has developed guidance for the BA1 evidence code11 regarding variants with a minor allele frequency above 5%. This code was applied variably among laboratories based on different incorporations of the guidance into laboratory-specific protocols, which contributed to discordance. It is also valuable for laboratories to continue sharing classifications and supporting evidence. The concordance rate in this project increased when the gene-disease-specific expertise and rationale for applying evidence from a given laboratory was shared and incorporated by others. Other research has also highlighted variant type, disease association, time since classification, low-penetrance variants, and the use of internal laboratory data33 , 34 as factors related to discordance. Several of these variables were not applicable in this project, but they are nonetheless important for laboratories and clinicians to consider when evaluating the clinical significance of a variant.
The ACMG currently recommends the return of "known pathogenic" and "expected pathogenic" (predicted LoF) variants in secondary finding genes. However, it is common for both P and LP variants in these genes to be returned, and it is anticipated that ACMG will extend their recommendation to include the LP variant classification category.31 , 35 In this project, the complete five-category concordance rate for variants classified as P by the submitting laboratory was higher (53%) than that for variants classified as LP by the submitting laboratory (13%), though the number of variants submitted as LP was small. Additionally, variants classified as P by the submitting laboratory were less often discordant with a VUS classification than variants classified as LP by the submitting laboratory (21% for P submitted variants versus 63% for LP submitted variants). An analysis of ClinVar data has suggested that the vast majority of variants classified as LP will ultimately be deemed P;36 however, lower concordance is still a concern for the LP classification category. Providers should contextualize laboratory results considering the personal and family history of the individual when considering risk assessment and medical recommendations. The likelihood of a LP variant in a secondary finding gene being clinically diagnostic is lower than for a P variant, and lower than in a diagnostic setting.
Establishing variant classification concordance across laboratories is not a central aim of the CSER consortium given that clinical projects are funded to use different approaches and platforms to evaluate genetic variation across a diverse range of participant phenotypes including hereditary cancer, suspected genetic neurologic and cardiac conditions, and developmental delay with or without dysmorphic features. However, harmonization of variant classification has been part of previous National Institutes of Health genomic research initiatives37 and is being incorporated into the All of Us Research Program38 with the classification and return of P and LP variants in the ACMG secondary finding v.2.0 genes. This work provides an estimate of the discordance that may need to be addressed across laboratories and highlights factors likely to contribute to classification variability.
This work has several limitations. The nine involved laboratories may not be generalizable to other clinical testing settings. Additionally, the variant set may not represent all evidence codes and types of variants, so it is possible that other important causes for discordance were not captured in this project. However, this project gives a pragmatic overview of identifying and resolving discordance for the types of variants encountered in a typical workflow for a laboratory performing genomic sequencing research, highlighting the continued need for shared data and tools.
It is important to contextualize this work in the broader framework of medical practice. Our field has taken unprecedented steps toward voluntary sharing of variant classifications, using the publicly accessible ClinVar database, in order to facilitate improvements in professional practice.39 Continued submission of variant classifications and evidence to ClinVar is essential to promote transparency and consistency in variant interpretation, and the development of tools to facilitate this process are of great value to laboratories. The concordance of our work is higher than that achieved in other fields where it has been measured40 , 41 and we have ongoing and robust efforts to build consensus in laboratory classification through programs like ClinGen.42 Thus, we hope that the work across our field can continue to be an exemplar of collaboration and professional consensus needed to achieve the highest quality of medical practice possible.
A shared understanding of the strength and interpretation of evidence supporting germline sequence variant pathogenicity is necessary to support the broad, reliable practice of genomic medicine. We present an updated estimate of germline variant classification concordance across genomic sequencing laboratories in the eMERGE Network and CSER Consortium. Progress toward concordant germline classification in this context has been made since the original 2015 publication of the ACMG-AMP guidance. Additional work is necessary to further improve agreement, particularly for evidence codes discussed above which continue to be applied and interpreted differently among laboratories.
Data and Code Availability
The published article includes all datasets generated or analyzed during the study.
Consortia
The authors acknowledge the following members of the CSER Sequencing and Diagnostic Yield working group: Monica Basehore, Jonathan Berg, Gabrielle Bertier, Joanna Chao, Wendy Chung, Susan Hiatt, Lauren Hicks, Anna Hurst, Sara Kalla, Melpi Kasapi, Alexander Katz, Bruce Korf, Mark Kvale, Pui-Yan Kwok, Grace LaMoure, James Lawlor, Deanna Maida, Donna Muzny, Jacqueline Ogdis, Anish Ray, Shannon Rego, Neil Risch, Myra Roche, Nuriye Sahin-Hodoglugil, Joseph Shieh, Michelle Thompson, and Karen Weck-Taylor.
Declaration of Interests
L.G.B. is an uncompensated member of the Illumina Corp medical ethics advisory board and receives in-kind research support from Arqule, Inc., now wholly owned by Merck, Inc.
Acknowledgments
The CSER consortium is funded by the National Human Genome Research Institute (NHGRI) with co-funding from the National Institute on Minority Health and Health Disparities (NIMHD) and the National Cancer Institute (NCI). This work was funded by the following grants: U01HG007292, U01HG007301, U01HG009599, U01HG009610, U01HG006487, U01HG006485, U24HG007307, U01HG008676, U41HG006834, Z1AHG200359, and Z1AHG200387. Sequencing for the ClinSeq A2 cohort was performed at the NIH Intramural Sequencing Center.
Notes
Published: October 26, 2020
Footnotes
Web Resources
Supplemental Information
Table S1. Variant Set Details, Classifications across Laboratories, and Concordance:
Document S2. Article plus Supplemental Information:
References
1. Bland A., Harrington E.A., Dunn K., Pariani M., Platt J.C.K., Grove M.E., Caleshu C. Clinically impactful differences in variant interpretation between clinicians and testing laboratories: a single-center experience. Genet. Med. 2018;20:369–373. [PubMed] [Google Scholar]
2. Richards S., Aziz N., Bale S., Bick D., Das S., Gastier-Foster J., Grody W.W., Hegde M., Lyon E., Spector E., ACMG Laboratory Quality Assurance Committee Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet. Med. 2015;17:405–424. [PMC free article] [PubMed] [Google Scholar]
3. Niehaus A., Azzariti D.R., Harrison S.M., DiStefano M.T., Hemphill S.E., Senol-Cosar O., Rehm H.L. A survey assessing adoption of the ACMG-AMP guidelines for interpreting sequence variants and identification of areas for continued improvement. Genet. Med. 2019;21:1699–1701. [PMC free article] [PubMed] [Google Scholar]
4. Nykamp K., Anderson M., Powers M., Garcia J., Herrera B., Ho Y.Y., Kobayashi Y., Patil N., Thusberg J., Westbrook M., Topper S., Invitae Clinical Genomics Group Sherloc: a comprehensive refinement of the ACMG-AMP variant classification criteria. Genet. Med. 2017;19:1105–1117. [PMC free article] [PubMed] [Google Scholar]
5. Walsh R., Mazzarotto F., Whiffin N., Buchan R., Midwinter W., Wilk A., Li N., Felkin L., Ingold N., Govind R. Quantitative approaches to variant classification increase the yield and precision of genetic testing in Mendelian diseases: the case of hypertrophic cardiomyopathy. Genome Med. 2019;11:5. [PMC free article] [PubMed] [Google Scholar]
6. Jarvik G.P., Browning B.L. Consideration of Cosegregation in the Pathogenicity Classification of Genomic Variants. Am. J. Hum. Genet. 2016;98:1077–1081. [PMC free article] [PubMed] [Google Scholar]
7. Brnich S.E., Abou Tayoun A.N., Couch F.J., Cutting G.R., Greenblatt M.S., Heinen C.D., Kanavy D.M., Luo X., McNulty S.M., Starita L.M., Clinical Genome Resource Sequence Variant Interpretation Working Group Recommendations for application of the functional evidence PS3/BS3 criterion using the ACMG/AMP sequence variant interpretation framework. Genome Med. 2019;12:3. [PMC free article] [PubMed] [Google Scholar]
8. Luo X., Feurstein S., Mohan S., Porter C.C., Jackson S.A., Keel S., Chicka M., Brown A.L., Kesserwan C., Agarwal A. ClinGen Myeloid Malignancy Variant Curation Expert Panel recommendations for germline RUNX1 variants. Blood Adv. 2019;3:2962–2979. [PMC free article] [PubMed] [Google Scholar]
9. Lee K., Krempely K., Roberts M.E., Anderson M.J., Carneiro F., Chao E., Dixon K., Figueiredo J., Ghosh R., Huntsman D. Specifications of the ACMG/AMP variant curation guidelines for the analysis of germline CDH1 sequence variants. Hum. Mutat. 2018;39:1553–1568. [PMC free article] [PubMed] [Google Scholar]
10. Oza A.M., DiStefano M.T., Hemphill S.E., Cushman B.J., Grant A.R., Siegert R.K., Shen J., Chapin A., Boczek N.J., Schimmenti L.A., ClinGen Hearing Loss Clinical Domain Working Group Expert specification of the ACMG/AMP variant interpretation guidelines for genetic hearing loss. Hum. Mutat. 2018;39:1593–1613. [PMC free article] [PubMed] [Google Scholar]
11. Ghosh R., Harrison S.M., Rehm H.L., Plon S.E., Biesecker L.G., ClinGen Sequence Variant Interpretation Working Group Updated recommendation for the benign stand-alone ACMG/AMP criterion. Hum. Mutat. 2018;39:1525–1530. [PMC free article] [PubMed] [Google Scholar]
12. Mester J.L., Ghosh R., Pesaran T., Huether R., Karam R., Hruska K.S., Costa H.A., Lachlan K., Ngeow J., Barnholtz-Sloan J. Gene-specific criteria for PTEN variant curation: Recommendations from the ClinGen PTEN Expert Panel. Hum. Mutat. 2018;39:1581–1592. [PMC free article] [PubMed] [Google Scholar]
13. Abou Tayoun A.N., Pesaran T., DiStefano M.T., Oza A., Rehm H.L., Biesecker L.G., Harrison S.M., ClinGen Sequence Variant Interpretation Working Group (ClinGen SVI) Recommendations for interpreting the loss of function PVS1 ACMG/AMP variant criterion. Hum. Mutat. 2018;39:1517–1524. [PMC free article] [PubMed] [Google Scholar]
14. Zastrow D.B., Baudet H., Shen W., Thomas A., Si Y., Weaver M.A., Lager A.M., Liu J., Mangels R., Dwight S.S., ClinGen Inborn Errors of Metabolism Working Group Unique aspects of sequence variant interpretation for inborn errors of metabolism (IEM): The ClinGen IEM Working Group and the Phenylalanine Hydroxylase Gene. Hum. Mutat. 2018;39:1569–1580. [PMC free article] [PubMed] [Google Scholar]
15. Biesecker L.G., Harrison S.M., ClinGen Sequence Variant Interpretation Working Group The ACMG/AMP reputable source criteria for the interpretation of sequence variants. Genet. Med. 2018;20:1687–1688. [PMC free article] [PubMed] [Google Scholar]
16. Gelb B.D., Cavé H., Dillon M.W., Gripp K.W., Lee J.A., Mason-Suares H., Rauen K.A., Williams B., Zenker M., Vincent L.M., ClinGen RASopathy Working Group ClinGen's RASopathy Expert Panel consensus methods for variant interpretation. Genet. Med. 2018;20:1334–1345. [PMC free article] [PubMed] [Google Scholar]
17. Kelly M.A., Caleshu C., Morales A., Buchan J., Wolf Z., Harrison S.M., Cook S., Dillon M.W., Garcia J., Haverfield E. Adaptation and validation of the ACMG/AMP variant classification framework for MYH7-associated inherited cardiomyopathies: recommendations by ClinGen's Inherited Cardiomyopathy Expert Panel. Genet. Med. 2018;20:351–359. [PMC free article] [PubMed] [Google Scholar]
18. Tavtigian S.V., Greenblatt M.S., Harrison S.M., Nussbaum R.L., Prabhu S.A., Boucher K.M., Biesecker L.G., ClinGen Sequence Variant Interpretation Working Group (ClinGen SVI) Modeling the ACMG/AMP variant classification guidelines as a Bayesian classification framework. Genet. Med. 2018;20:1054–1060. [PMC free article] [PubMed] [Google Scholar]
19. SoRelle J.A., Pascual J.M., Gotway G., Park J.Y. Assessment of Interlaboratory Variation in the Interpretation of Genomic Test Results in Patients With Epilepsy. JAMA Netw. Open. 2020;3:e203812. [PMC free article] [PubMed] [Google Scholar]
20. Lincoln S.E., Yang S., Cline M.S., Kobayashi Y., Zhang C., Topper S., Haussler D., Paten B., Nussbaum R.L. Consistency of BRCA1 and BRCA2 Variant Classifications Among Clinical Diagnostic Laboratories. JCO Precis. Oncol. 2017;1 PO.16.00020. [PMC free article] [PubMed] [Google Scholar]
21. Harrison S.M., Dolinsky J.S., Knight Johnson A.E., Pesaran T., Azzariti D.R., Bale S., Chao E.C., Das S., Vincent L., Rehm H.L. Clinical laboratories collaborate to resolve differences in variant interpretations submitted to ClinVar. Genet. Med. 2017;19:1096–1104. [PMC free article] [PubMed] [Google Scholar]
22. Harrison S.M., Dolinksy J.S., Chen W., Collins C.D., Das S., Deignan J.L., Garber K.B., Garcia J., Jarinova O., Knight Johnson A.E., ClinGen Sequence Variant Inter-Laboratory Discrepancy Resolution Working Group Scaling resolution of variant classification differences in ClinVar between 41 clinical laboratories through an outlier approach. Hum. Mutat. 2018;39:1641–1649. [PMC free article] [PubMed] [Google Scholar]
23. Furqan A., Arscott P., Girolami F., Cirino A.L., Michels M., Day S.M., Olivotto I., Ho C.Y., Ashley E., Green E.M., Caleshu C., SHaRe Consortium Care in Specialized Centers and Data Sharing Increase Agreement in Hypertrophic Cardiomyopathy Genetic Test Interpretation. Circ Cardiovasc Genet. 2017;10:10. [PubMed] [Google Scholar]
24. Wain K.E., Azzariti D.R., Goldstein J.L., Johnson A.K., Krautscheid P., Lepore B., O'Daniel J.M., Ritter D., Savatt J.M., Riggs E.R., Martin C.L. Variant interpretation is a component of clinical practice among genetic counselors in multiple specialties. Genet. Med. 2020;22:785–792. [PMC free article] [PubMed] [Google Scholar]
25. Amendola L.M., Jarvik G.P., Leo M.C., McLaughlin H.M., Akkari Y., Amaral M.D., Berg J.S., Biswas S., Bowling K.M., Conlin L.K. Performance of ACMG-AMP Variant-Interpretation Guidelines among Nine Laboratories in the Clinical Sequencing Exploratory Research Consortium. Am. J. Hum. Genet. 2016;98:1067–1076. [PMC free article] [PubMed] [Google Scholar]
26. Green R.C., Berg J.S., Grody W.W., Kalia S.S., Korf B.R., Martin C.L., McGuire A.L., Nussbaum R.L., O'Daniel J.M., Ormond K.E., American College of Medical Genetics and Genomics ACMG recommendations for reporting of incidental findings in clinical exome and genome sequencing. Genet. Med. 2013;15:565–574. [PMC free article] [PubMed] [Google Scholar]
27. Kalia S.S., Adelman K., Bale S.J., Chung W.K., Eng C., Evans J.P., Herman G.E., Hufnagel S.B., Klein T.E., Korf B.R. Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update (ACMG SF v2.0): a policy statement of the American College of Medical Genetics and Genomics. Genet. Med. 2017;19:249–255. [PubMed] [Google Scholar]
28. Ackerman S.L., Koenig B.A. Understanding variations in secondary findings reporting practices across U.S. genome sequencing laboratories. AJOB Empir. Bioeth. 2018;9:48–57. [PubMed] [Google Scholar]
29. Amendola L.M., Dorschner M.O., Robertson P.D., Salama J.S., Hart R., Shirts B.H., Murray M.L., Tokita M.J., Gallego C.J., Kim D.S. Actionable exomic incidental findings in 6503 participants: challenges of variant classification. Genome Res. 2015;25:305–315. [PMC free article] [PubMed] [Google Scholar]
30. Olfson E., Cottrell C.E., Davidson N.O., Gurnett C.A., Heusel J.W., Stitziel N.O., Chen L.S., Hartz S., Nagarajan R., Saccone N.L., Bierut L.J. Identification of Medically Actionable Secondary Findings in the 1000 Genomes. PLoS ONE. 2015;10:e0135193. [PMC free article] [PubMed] [Google Scholar]
31. eMERGE Clinical Annotation Working Group Frequency of genomic secondary findings among 21,915 eMERGE network participants. Genet. Med. 2020;22:1470–1477. [PMC free article] [PubMed] [Google Scholar]
32. Harris P.A., Taylor R., Minor B.L., Elliott V., Fernandez M., O'Neal L., McLeod L., Delacqua G., Delacqua F., Kirby J., Duda S.N., REDCap Consortium The REDCap consortium: Building an international community of software platform partners. J. Biomed. Inform. 2019;95:103208. [PMC free article] [PubMed] [Google Scholar]
33. Garber K.B., Vincent L.M., Alexander J.J., Bean L.J.H., Bale S., Hegde M. Reassessment of Genomic Sequence Variation to Harmonize Interpretation for Personalized Medicine. Am. J. Hum. Genet. 2016;99:1140–1149. [PMC free article] [PubMed] [Google Scholar]
34. Yang S., Lincoln S.E., Kobayashi Y., Nykamp K., Nussbaum R.L., Topper S. Sources of discordance among germ-line variant classifications in ClinVar. Genet. Med. 2017;19:1118–1126. [PMC free article] [PubMed] [Google Scholar]
35. Schwartz M.L.B., McCormick C.Z., Lazzeri A.L., Lindbuchler D.M., Hallquist M.L.G., Manickam K., Buchanan A.H., Rahm A.K., Giovanni M.A., Frisbie L. A Model for Genome-First Care: Returning Secondary Genomic Findings to Participants and Their Healthcare Providers in a Large Research Cohort. Am. J. Hum. Genet. 2018;103:328–337. [PMC free article] [PubMed] [Google Scholar]
36. Harrison S.M., Rehm H.L. Is 'likely pathogenic' really 90% likely? Reclassification data in ClinVar. Genome Med. 2019;11:72. [PMC free article] [PubMed] [Google Scholar]
37. eMERGE Consortium Harmonizing Clinical Sequencing and Interpretation for the eMERGE III Network. Am. J. Hum. Genet. 2019;105:588–605. [PMC free article] [PubMed] [Google Scholar]
38. Murray J. The "All of Us" Research Program. N. Engl. J. Med. 2019;381:1884. [PubMed] [Google Scholar]
39. Rehm H.L. A new era in the interpretation of human genomic variation. Genet. Med. 2017;19:1092–1095. [PMC free article] [PubMed] [Google Scholar]
40. Herzog R., Elgort D.R., Flanders A.E., Moley P.J. Variability in diagnostic error rates of 10 MRI centers performing lumbar spine MRI examinations on the same patient within a 3-week period. Spine J. 2017;17:554–561. [PubMed] [Google Scholar]
41. Elmore J.G., Longton G.M., Carney P.A., Geller B.M., Onega T., Tosteson A.N., Nelson H.D., Pepe M.S., Allison K.H., Schnitt S.J. Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA. 2015;313:1122–1132. [PMC free article] [PubMed] [Google Scholar]
42. Rivera-Muñoz E.A., Milko L.V., Harrison S.M., Azzariti D.R., Kurtz C.L., Lee K., Mester J.L., Weaver M.A., Currey E., Craigen W. ClinGen Variant Curation Expert Panel experiences and standardized processes for disease and gene-level specification of the ACMG/AMP guidelines for sequence variant interpretation. Hum. Mutat. 2018;39:1614–1622. [PMC free article] [PubMed] [Google Scholar]
Articles from American Journal of Human Genetics are provided here courtesy of American Society of Human Genetics
Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7675005/
0 Response to "I Will Continue to Work With the Acmg Commette"
Post a Comment