Citation David M. Euhus, Kristin C. Smith, Linda Robinson, Amy Stucky, Olufunmilayo I. Olopade, Shelly Cummings, Judy E.
Garber, Anu Chittenden, Gordon B. Mills, Paula Rieger, Laura Esserman, Beth Crawford, Kevin S. Hughes, Connie A. Roche, Patricia A.
The BRCAPRO model is now included in the R package BayesMendel for carrier. I don't think anyone will be able to fit code for a full application into a tiny.
Ganz, Joyce Seldon, Carol J. Fabian, Jennifer Klemp, Gail Tomlinson; Pretest Prediction of BRCA1 or BRCA2 Mutation by Risk Counselors and the Computer Model BRCAPRO, JNCI: Journal of the National Cancer Institute, Volume 94, Issue 11, 5 June 2002, Pages 844–851, Download citation file:.
We measured the performance of eight cancer risk counselors and of a computer model, BRCAPRO, at identifying families likely to carry a BRCA gene mutation. Methods: Eight cancer risk counselors and the computer model BRCAPRO estimated BRCA gene mutation probabilities for 148 pedigrees selected from an initial sample of 272 pedigrees. The final sample was limited to pedigrees with a proband affected by breast or ovarian cancer and BRCA1 and BRCA2 gene sequencing results unequivocally reported as negative or positive for a deleterious mutation. Sensitivity, specificity, negative predictive value, positive predictive value, and areas under receiver operator characteristics (ROC) curves were calculated for each risk counselor and for BRCAPRO. All statistical tests were two sided. Results: Using a greater-than-10% BRCA gene mutation probability threshold, the median sensitivity for identifying mutation carriers was 94% (range = 81% to 98%) for the eight risk counselors and 92% (range = 91% to 92%) for BRCAPRO. Median specificity at this threshold was 16% (range = 6% to 34%) for the risk counselors and 32% (range = 30% to 34%) for BRCAPRO ( P =.04).
Median area under the ROC curves was 0.671 for the risk counselors (range = 0.620 to 0.717) and 0.712 (range = 0.706 to 0.720) for BRCAPRO ( P =.04). There was a slight, but not statistically significant, improvement in all counselor performance measures when BRCAPRO-assigned gene mutation probability information was included with the pedigrees.
Conclusions: Sensitivity for identifying BRCA gene mutation carriers is similar for experienced risk counselors and the computer model BRCAPRO. Because the computer model consistently demonstrated superior specificity, overall discrimination between BRCA gene mutation carriers and BRCA gene mutation noncarriers was slightly better for BRCAPRO. Women who inherit a BRCA1 or BRCA2 gene with a deleterious mutation face a 35%–85% lifetime risk of developing breast cancer (–) and a 16%–57% lifetime risk of developing ovarian cancer (,–). Surgical interventions may reduce these risks, with oophorectomy reducing the risk of breast and ovarian cancer by 47% and 98% , respectively, and bilateral prophylactic mastectomy reducing the risk of breast cancer by more than 90% (,). Cancer susceptibility related to mutated BRCA genes is transmitted in an autosomal dominant fashion so, on average, only half the individuals in a given generation will be at increased risk for the associated cancers. Genetic testing can identify the individuals most likely to benefit from these risk-reducing interventions and spare unaffected relatives the risks associated with them.
Cancer susceptibility gene testing is not appropriate for everyone. BRCA gene mutation testing, in particular, is costly; is sometimes complicated by social, legal, or insurance issues; and can yield results with uncertain clinical significance. For these reasons, pretest counseling performed by professionals knowledgeable about the interpretation and limitations of genetic tests is required for anyone considering BRCA gene testing. One important component of pretest counseling is estimation of the probability that a family carries a BRCA1 or BRCA2 gene mutation. In a recent policy statement, the American Society of Clinical Oncology has suggested that consideration of BRCA gene mutation testing should be limited to individuals whose probability of carrying a mutation exceeds 10%. There are currently no validated methods for estimating BRCA gene mutation probabilities. Traditionally, cancer risk counselors familiar with Mendelian genetics and cancer patterns in families with an inherited predisposition (i.e., early age at onset and multiple or bilateral cancers) collect family history information and subjectively estimate an individual's pretest gene mutation probability.
As data sets describing the cancer family histories of BRCA gene mutation carriers have accumulated, investigators have attempted to develop mathematical models for objectively estimating pretest mutation probabilities on the basis of one or more attributes of families with an inherited breast and/or ovarian cancer predisposition (–). A recently described computer model that incorporates information for all family members (affected and unaffected with breast or ovarian cancer) is BRCAPRO. This model uses Bayes' theorem to calculate the probability of an individual carrying a BRCA gene mutation, given a specific family history (,). The model is based on age-specific and cumulative breast and ovarian cancer incidence rates for BRCA gene mutation carriers (,) as compared with the same rates for BRCA gene mutation noncarriers.
There are factors in the model calculations to account for Ashkenazi Jewish heritage and male breast cancer. The best method for estimating pretest mutation probability will ideally have a high sensitivity (i.e., will not miss many mutation carriers), a high specificity (i.e., will not recommend testing for nearly everyone), and a high negative predictive value (i.e., a low probability can provide a measure of reassurance that there really is no mutation). It is currently not known how well cancer risk counselors and the BRCAPRO model measure up to this standard. In addition, it is not known whether BRCA gene mutation probability information generated by BRCAPRO improves or confounds the subjective assessment of cancer risk counselors. In this study, we measured the performance of cancer risk counselors for estimating pretest BRCA gene mutation probability and evaluated the influence of BRCAPRO model probability information on this performance. M ethods General Approach Eight experienced cancer risk counselors assigned BRCA gene mutation probabilities to a series of 148 pedigrees from families who had obtained BRCA gene mutation testing through several different university-based clinical cancer genetics programs. Mutation probability assignments were compared with results of complete BRCA1 and BRCA2 gene sequencing.
No identifying information from the families was included with the pedigrees. This research was reviewed and approved by the Institutional Review Board at The University of Texas Southwestern Medical Center at Dallas.
To measure the influence of BRCAPRO-assigned mutation probability information on the performance of the cancer risk counselors, the counselors were also supplied with a second set of pedigrees on which the BRCAPRO information was printed. The pedigrees that included BRCAPRO information were admixed with pedigrees that did not include this information, and the risk counselors were not told that they would be seeing each pedigree twice. Risk Counselors All eight cancer risk counselors were from university-based cancer genetics clinics that employ interdisciplinary teams for identifying and managing people at high risk for cancer. Each of these clinics provides patient education, risk assessment, pre- and post-test counseling, and intervention planning and execution to reduce risk. Each of the eight risk counselors indicated that 95% or more of their practice was devoted to clinical cancer genetics, with six of the eight indicating that 90% or more of their practice was devoted specifically to breast-ovarian cancer susceptibility counseling. Three of the clinics counsel more than 30 breast or ovarian cancer families each month, three counsel 11–30 families each month, and two counsel 6–10 families each month.
All eight risk counselors have a master's degree, and four are certified by the American Board of Genetic Counselors. Ascertainment Pedigrees and BRCA gene mutation testing results were submitted for 272 families by the eight cancer genetics clinics. To avoid the uncertainty that is introduced when no mutation is detected using a limited genetic test (e.g., the Ashkenazi three-mutation panel), we did not accept pedigrees for any families that had not undergone complete BRCA1 and BRCA2 gene sequencing, regardless of whether a mutation had been identified (n = 64). To make it easier for the risk counselors to assign mutation probability estimates, we also rejected pedigrees from families in which the proband was not affected by either breast or ovarian cancer (n = 44). Pedigrees from families with mutations of uncertain clinical significance were also rejected (n = 8). In addition, six families that were ascertained through a mutation screening research project rather than through a clinical counseling setting were rejected, as were two pedigrees that were exact duplicates of pedigrees submitted by another institution.
Three of the cancer genetics clinics submitted pedigrees for all families they had tested to date (n = 64), two clinics submitted a recent consecutive series of tested families (n = 62), and three clinics submitted a convenience sample of pedigrees that were not necessarily representative of all tested families from those clinics (n = 22). The proportion of BRCA gene mutation-positive families in the third group was judged by the submitting risk counselors to be higher than that of all families that had been tested in those clinics. Inclusion of the convenience sample did not bias ascertainment, however, as the proportion of pedigrees with a BRCA gene mutation was 41% (95% confidence interval CI = 33% to 50%) for the 126 consecutive families compared with 43% (95% CI = 35% to 51%) for the entire series of 148 pedigrees. Women with nondeleterious polymorphisms identified by complete gene sequencing were included in the “no mutation” group. The final sample consisted of 148 pedigrees from women affected with either breast or ovarian cancer who had been ascertained through a cancer genetics clinic and had undergone complete BRCA1 and BRCA2 gene sequencing by Myriad Genetics, Inc.
(Salt Lake City, UT). Among the 148 families whose pedigrees were included in the sample, complete BRCA1 and BRCA2 gene sequencing had shown that 42 families carried deleterious BRCA1 mutations, 21 carried deleterious BRCA2 mutations, and 85 had no deleterious BRCA gene mutations. Fifty-two different mutations were recorded among the 63 BRCA mutation-positive families (the spectrum of mutations is available on request to the author or online as supplemental data. The only mutations that occurred in more than one family (pedigree) were in BRCA1: 185delAG (six), 5382insC (four), and 1832del5 (two); in BRCA2: IVS 17–1 GC (two) and 5950delCT (two). Calculation of BRCA Gene Mutation Probabilities by the Computer Model BRCAPRO BRCAPRO uses Bayes' theorem to calculate the probability that an individual carries a mutation in the BRCA1 or BRCA2 gene on the basis of his or her family history of breast and/or ovarian cancer.
An intermediate step in the calculation of this probability is the calculation of a likelihood ratio for each individual in the family. These likelihood ratios are based on the probability that a specific cancer history (i.e., breast and/or ovarian cancer or no cancer at a given age) would be observed whether the individual were a mutation carrier or not. The final probability is calculated using the allelic frequency of BRCA gene mutations in the relevant population (Ashkenazi Jewish or non-Ashkenazi Jewish) and breast and ovarian cancer incidence rates for BRCA gene mutation carriers and noncarriers.
The calculation of the final probability incorporates age and cancer information for all first- and second-degree relatives on the basis of Mendelian inheritance of an autosomal dominant gene. A detailed description of this model has been published (,). The calculations described are too complex to complete by hand for more than one or two relatives in a given family. BRCAPRO is a DOS-based computer model that automates these probability calculations.
Additional information about the BRCAPRO computer model can be found at. Family history information, including Ashkenazi Jewish heritage, was entered into the BRCAPRO interface, CancerGene, to generate pedigrees for all 148 families that included information for four generations (proband, siblings, offspring, parents, aunts, uncles, and grandparents). CancerGene is a pedigree drawing utility that automatically generates the input text file required by the BRCAPRO model, runs the BRCAPRO model in the background, and then archives family history and mutation probability information to a relational database.
Additional information about CancerGene can be obtained at. BRCA Gene Mutation Probabilities Assigned by Risk Counselors Risk counselors from each of the eight cancer genetics clinics all came to The University of Texas Southwestern Medical Center at Dallas, where they evaluated each pedigree in a supervised environment. A supervised environment was used to ensure that counselor-generated probabilities were not contaminated by the use of BRCAPRO or other computer models. The pedigrees indicated whether a family was of Ashkenazi Jewish heritage. The risk counselors were asked to estimate the probability of BRCA gene mutation for each pedigree by using a five-point scale (1, ≤10%; 2, 11%–30%; 3, 31%–70%; 4, 71%–94%; and 5, ≥95%). These cutoffs distributed all 148 pedigrees equally into five groups based on BRCAPRO-calculated mutation probabilities.
The risk counselors evaluated the pedigrees in groups of 20. The first group of 20 pedigrees included the pedigree diagram only. Subsequent groups of pedigrees were admixed with pedigrees the counselor had previously evaluated that, the second time around, had the BRCAPRO probabilities printed on the diagram. The risk counselors were also asked to indicate whether they recognized the pedigree as coming from their own clinic; such pedigrees were excluded from the calculations for that individual risk counselor. Statistical Methods Receiver operator characteristics (ROC) curves are commonly used to determine which threshold value for a clinical test provides the best discrimination between “normal” and “abnormal.” ROC curves are constructed by plotting the sensitivity of a particular threshold value for detecting abnormal individuals (in this case, BRCA gene mutation carriers) on the y-axis against 1 minus specificity for that threshold value on the x-axis.
The area under the ROC curve is a measure of the overall discrimination that a given test can provide between individuals with the condition of interest (BRCA gene mutation, in this instance) and those without the condition. In our study, the area under the ROC curve corresponds to the probability that a family with a BRCA gene mutation will have a greater mutation probability score than a family without a BRCA gene mutation chosen at random. For instance, if the area under the ROC curve were 0.5, a mutation carrier would have a greater mutation probability score than a noncarrier only 50% of the time, a result that is no better than chance.
BRCA gene mutation probabilities (categories 1–5) assigned by the risk counselors were used as the threshold values. Sensitivity and specificity for recognizing mutation carriers at each threshold were calculated for each risk counselor, and ROC curves were plotted. A similar ROC curve, using the same five probability thresholds, was generated for mutation probabilities as calculated by the BRCAPRO model. Because pedigrees recognized as coming from the counselor's own clinic were excluded for that counselor, they were also excluded from BRCAPRO calculations that were compared with that counselor. Areas under the ROC curves were compared using the method of DeLong et al. Performance measures, such as sensitivity, specificity, positive predictive value, and negative predictive value for individual risk counselors blinded to BRCPARO mutation probability information and then provided with this information were analyzed in a pair-wise fashion and compared by using the binomial distribution. Performance measures for the entire group of eight risk counselors were expressed as medians and then compared with BRCAPRO mutation probabilities by using the Wilcoxon signed rank test.
All tests of significance were two-tailed. When means were reported, they were compared by using two-tailed t tests. Proportions were compared by using chi-square tests. R esults Characteristics of the Sample The final sample of 148 families included 15 with Ashkenazi Jewish ancestry.
Other characteristics of the study sample are listed in Table 1. The mean numbers of relatives with breast or ovarian cancer per family in the 85 families with no deleterious BRCA gene mutations were 2.5 (95% CI = 2.3 to 2.8) and 0.45 (95% CI = 0.28 to 0.62) respectively, compared with 2.7 (95% CI = 2.3 to 3.0) and 0.71 (95% CI = 0.50 to 0.93) for the 63 families with deleterious BRCA gene mutations. The difference in mean values was statistically significant for ovarian cancer cases only ( P =.05).
Although the proportion of families with breast cancer was similar between the mutation-carrying and noncarrying families (97%; 95% CI = 92% to 100% versus 95%; 95% CI = 91% to 100%; P =.96), the mean age at breast cancer diagnosis among women from mutation-carrying families (43.2 years; 95% CI = 41.4 years to 45.1 years) was lower than the age at diagnosis for women from noncarrying families (49.1 years; 95% CI = 47.4 years to 50.8 years, P. Characteristic Mutation negative Mutation positive.
P.42 BRCA1 and 21 BRCA2 families. †CI = confidence interval. ‡Two-tailed t test. §Chi-square test.
∥For women with bilateral breast cancer, only the age at diagnosis of the first breast cancer is included in the calculation. Of pedigrees 85 63 Mean No. Of relatives per pedigree (95% CI†) 14.2 (11.1 to 17.4) 13.2 (10.1 to 16.3).18‡ No. (%) of pedigrees with: Any breast cancer 81 (95.3) 61 (96.8).96§ Any ovarian cancer 26 (30.6) 31 (49.2).03§ Breast cancer only 59 (69.4) 32 (50.8).03§ Ovarian cancer only 4 (4.7) 2 (3.2).96§ Breast and ovarian cancer 22 (25.9) 29 (46.0).02§ Bilateral breast cancer 24 (28.2) 22 (34.9).49§ Breast and ovarian cancer in the same individual 7 (8.2) 14 (22.2).03§ Mean No. Of relatives with breast cancer per family (95% CI) 2.5 (2.3 to 2.8) 2.7 (2.3 to 3.0).52‡ Mean age at breast cancer diagnosis (95% CI)∥ 49.1 (47.4 to 50.8) 43.2 (41.4 to 45.1).
‡Two-tailed t test. §Chi-square test. ∥For women with bilateral breast cancer, only the age at diagnosis of the first breast cancer is included in the calculation. Of pedigrees 85 63 Mean No. Of relatives per pedigree (95% CI†) 14.2 (11.1 to 17.4) 13.2 (10.1 to 16.3).18‡ No. (%) of pedigrees with: Any breast cancer 81 (95.3) 61 (96.8).96§ Any ovarian cancer 26 (30.6) 31 (49.2).03§ Breast cancer only 59 (69.4) 32 (50.8).03§ Ovarian cancer only 4 (4.7) 2 (3.2).96§ Breast and ovarian cancer 22 (25.9) 29 (46.0).02§ Bilateral breast cancer 24 (28.2) 22 (34.9).49§ Breast and ovarian cancer in the same individual 7 (8.2) 14 (22.2).03§ Mean No. Of relatives with breast cancer per family (95% CI) 2.5 (2.3 to 2.8) 2.7 (2.3 to 3.0).52‡ Mean age at breast cancer diagnosis (95% CI)∥ 49.1 (47.4 to 50.8) 43.2 (41.4 to 45.1).
Characteristic Mutation negative Mutation positive. P.42 BRCA1 and 21 BRCA2 families. †CI = confidence interval. ‡Two-tailed t test.
§Chi-square test. ∥For women with bilateral breast cancer, only the age at diagnosis of the first breast cancer is included in the calculation. Of pedigrees 85 63 Mean No. Of relatives per pedigree (95% CI†) 14.2 (11.1 to 17.4) 13.2 (10.1 to 16.3).18‡ No. (%) of pedigrees with: Any breast cancer 81 (95.3) 61 (96.8).96§ Any ovarian cancer 26 (30.6) 31 (49.2).03§ Breast cancer only 59 (69.4) 32 (50.8).03§ Ovarian cancer only 4 (4.7) 2 (3.2).96§ Breast and ovarian cancer 22 (25.9) 29 (46.0).02§ Bilateral breast cancer 24 (28.2) 22 (34.9).49§ Breast and ovarian cancer in the same individual 7 (8.2) 14 (22.2).03§ Mean No. Of relatives with breast cancer per family (95% CI) 2.5 (2.3 to 2.8) 2.7 (2.3 to 3.0).52‡ Mean age at breast cancer diagnosis (95% CI)∥ 49.1 (47.4 to 50.8) 43.2 (41.4 to 45.1).
Characteristic Mutation negative Mutation positive. P.42 BRCA1 and 21 BRCA2 families. †CI = confidence interval. ‡Two-tailed t test. §Chi-square test. ∥For women with bilateral breast cancer, only the age at diagnosis of the first breast cancer is included in the calculation.
Of pedigrees 85 63 Mean No. Of relatives per pedigree (95% CI†) 14.2 (11.1 to 17.4) 13.2 (10.1 to 16.3).18‡ No. (%) of pedigrees with: Any breast cancer 81 (95.3) 61 (96.8).96§ Any ovarian cancer 26 (30.6) 31 (49.2).03§ Breast cancer only 59 (69.4) 32 (50.8).03§ Ovarian cancer only 4 (4.7) 2 (3.2).96§ Breast and ovarian cancer 22 (25.9) 29 (46.0).02§ Bilateral breast cancer 24 (28.2) 22 (34.9).49§ Breast and ovarian cancer in the same individual 7 (8.2) 14 (22.2).03§ Mean No. Of relatives with breast cancer per family (95% CI) 2.5 (2.3 to 2.8) 2.7 (2.3 to 3.0).52‡ Mean age at breast cancer diagnosis (95% CI)∥ 49.1 (47.4 to 50.8) 43.2 (41.4 to 45.1). Counselor Performance measures 1 2 3 4 5 6 7 8 Median.Pedigrees recognized as coming from the counselor's own institution were excluded for that counselor and from the BRCAPRO calculations used for comparison with that counselor.
†All P values are two-tailed and were based on the binomial distribution for paired proportions, unless otherwise indicated. ‡Wilcoxon signed rank test. §Because pedigrees recognized as coming from the counselor's own institution were excluded for that counselor, BRCAPRO probabilities were generated only for those pedigrees that were included in the counselor's calculation.
∥Compared with counselor blinded to BRCAPRO probabilities. Counselor Performance measures 1 2 3 4 5 6 7 8 Median.Pedigrees recognized as coming from the counselor's own institution were excluded for that counselor and from the BRCAPRO calculations used for comparison with that counselor.
†All P values are two-tailed and were based on the binomial distribution for paired proportions, unless otherwise indicated. ‡Wilcoxon signed rank test.
§Because pedigrees recognized as coming from the counselor's own institution were excluded for that counselor, BRCAPRO probabilities were generated only for those pedigrees that were included in the counselor's calculation. ∥Compared with counselor blinded to BRCAPRO probabilities. Counselor Performance measures 1 2 3 4 5 6 7 8 Median.Pedigrees recognized as coming from the counselor's own institution were excluded for that counselor and from the BRCAPRO calculations used for comparison with that counselor. †All P values are two-tailed and were based on the binomial distribution for paired proportions, unless otherwise indicated.
‡Wilcoxon signed rank test. §Because pedigrees recognized as coming from the counselor's own institution were excluded for that counselor, BRCAPRO probabilities were generated only for those pedigrees that were included in the counselor's calculation. ∥Compared with counselor blinded to BRCAPRO probabilities. Counselor Performance measures 1 2 3 4 5 6 7 8 Median.Pedigrees recognized as coming from the counselor's own institution were excluded for that counselor and from the BRCAPRO calculations used for comparison with that counselor. †All P values are two-tailed and were based on the binomial distribution for paired proportions, unless otherwise indicated. ‡Wilcoxon signed rank test.
§Because pedigrees recognized as coming from the counselor's own institution were excluded for that counselor, BRCAPRO probabilities were generated only for those pedigrees that were included in the counselor's calculation. ∥Compared with counselor blinded to BRCAPRO probabilities.
Receiver operator characteristics (ROC) curves for the computer model, BRCAPRO ( thick line), and for eight cancer risk counselors ( thin lines) in a study of pretest prediction of BRCA1 and BRCA2 gene mutations. The sensitivity ( y-axis) for identifying a BRCA gene mutation-carrying family is plotted against 1 minus specificity ( x-axis) for identifying a mutation-carrying family for each of the five possible mutation probability assignments. The area under the ROC curve is a measure of the discrimination between gene mutation-carrying and noncarrying families. Receiver operator characteristics (ROC) curves for the computer model, BRCAPRO ( thick line), and for eight cancer risk counselors ( thin lines) in a study of pretest prediction of BRCA1 and BRCA2 gene mutations. The sensitivity ( y-axis) for identifying a BRCA gene mutation-carrying family is plotted against 1 minus specificity ( x-axis) for identifying a mutation-carrying family for each of the five possible mutation probability assignments. The area under the ROC curve is a measure of the discrimination between gene mutation-carrying and noncarrying families.
Summary of performance measures for eight cancer risk counselors ( circles) and median values for the computer model BRCAPRO ( horizontal lines). NPV = negative predictive value; PPV = positive predictive value; ROC curve area = receiver operator characteristics curve area. Sensitivity, specificity, and NPV were all calculated based on a greater-than-10% probability that a pedigree represented a BRCA gene mutation-carrying family. PPV was calculated based on a 95%-or-greater probability that a pedigree represented a BRCA gene mutation-carrying family. One counselor ( open circle) did not score any pedigrees as having a mutation probability of 95% or greater, so no PPV could be calculated for that counselor. Sensitivity, specificity, NPV, and PPV were all measured as a percentage ( left-hand side y-axis) and ROC curve area was measured as the area under the curve ( right-hand side y-axis).
Summary of performance measures for eight cancer risk counselors ( circles) and median values for the computer model BRCAPRO ( horizontal lines). NPV = negative predictive value; PPV = positive predictive value; ROC curve area = receiver operator characteristics curve area. Sensitivity, specificity, and NPV were all calculated based on a greater-than-10% probability that a pedigree represented a BRCA gene mutation-carrying family. PPV was calculated based on a 95%-or-greater probability that a pedigree represented a BRCA gene mutation-carrying family. One counselor ( open circle) did not score any pedigrees as having a mutation probability of 95% or greater, so no PPV could be calculated for that counselor. Sensitivity, specificity, NPV, and PPV were all measured as a percentage ( left-hand side y-axis) and ROC curve area was measured as the area under the curve ( right-hand side y-axis).