A group of authors from The University of Texas MD Anderson Cancer Center in Houston, TX, US successfully conducted a validation of their Li-Fraumeni syndrome risk prediction models using a unique clinical counselling-based patient cohort collected at their centre from 2000 to 2020. These models had been trained and validated on research protocol-based data sets. The study was carefully designed to mimic scenarios that genetic counsellors encounter in clinical settings, with 20-45% missing data.
Cancer-specific and multiple primary cancer models demonstrated excellent discrimination and good calibration when predicting deleterious germline TP53 mutations. As expected, the performance was lower than the validation results obtained using research protocol-based cohorts, most likely due to the lack of important data such as age at last contact and age at cancer diagnoses. For predictions of cancer risks, both models displayed performance that was comparable with previous validation studies on research protocol-based cohorts in most aspects. The first risk prediction validation study of its kind is published by Wenyi Wang, PhD and colleagues on 3 April 2024 in the JCO.
Li-Fraumeni syndrome is a hereditary cancer syndrome identified by deleterious germline mutations in the TP53 tumour suppressor gene. Patients with Li-Fraumeni syndrome are at significantly increased risks of many cancer types with lifetime risk of 93% in women and 73% in men and with a 50% risk of second primary malignancy. Conversations with patients regarding genetic testing and cancer screening have been challenging, partly because genetic counsellors could only provide general cancer risks associated with Li-Fraumeni syndrome.
Risk prediction models have been developed for other hereditary cancer syndromes but Li-Fraumeni syndrome remained an untouched area until recently in that regard. The authors developed two models for families with Li-Fraumeni syndrome: a competing-risk model that predicts cancer-specific risks for the first primary and a recurrent event model that extends the prediction to multiple primary cancer. These models were trained on Li-Fraumeni syndrome cohort rich in family history, and successfully validated on independent cohorts.
Research protocol-based data sets are ideal for training statistical models to estimate key epidemiological parameters of a study population. However, they do not represent data sets that are typically observed and collected in clinical settings. The term clinical counselling-based refers to the data that are encountered by genetic counsellors during counselling sessions and differs significantly from research protocol-based data because patients may not have accurate and complete family histories and some families have younger members who have not developed cancer. This leads to a higher rate of missing information such as family relationship, age of death, and age at cancer diagnoses.
Owing to these wide discrepancies in data quality, it is important to determine whether statistical models that are trained and validated on research protocol-based cohorts can perform well enough on a clinical counselling-based cohort to be clinically useful. In this study, the authors validate their risk prediction models on a clinical counselling-based cohort of 124 families whose probands underwent genetic counselling at the Clinical Cancer Genetics programme at MD Anderson Cancer Center between 2000 and 2020.
The clinical counselling-based cohort consists of 3,297 individuals across 124 families with 522 cases of single primary cancer and 125 cases of multiple primary cancers. The study team applied their software suite LFSPRO to make risk predictions and assessed performance in discrimination using AUC and in calibration using observed/expected ratio.
For prediction of deleterious TP53 mutations, they achieved an AUC of 0.78 (95% confidence interval [CI] 0.71 to 0.85) and an observed/expected ratio of 1.66 (95% CI 1.53 to 1.80). Using the LFSPRO model for multiple primary cancers to predict the onset of the second cancer, the study team obtained an AUC of 0.70 (95% CI 0.58 to 0.82). Using the LFSPRO cancer-specific model to predict the onset of different cancer types as the first primary, they achieved AUCs between 0.70 and 0.83 for sarcoma, breast cancer, or other cancers combined.
For predictions of cancer risks, both models displayed performance that was comparable with previous validation studies on research protocol-based cohorts in most aspects. Given the promising results, the study team has implemented their risk prediction models as a simple, interactive R/Shiny app for users without any programming background, to expedite clinical applications.
The next steps to bring risk prediction models like LFSPRO closer to clinics should include a prospective evaluation of one clinical counselling-based family at a time to further refine the picture of how risk prediction can transform clinical practice.
The study was supported by grants from the Cancer Prevention and Research Institute of Texas and the US National Institutes of Health.
The study was previously presented at the International Li-Fraumeni Syndrome Association Symposium (Bethesda, MD, US; 13-16 October 2022) and at the American Association for Cancer Research Annual Meeting (Orlando, FL, US; 13-19 April 2023).
Reference
Nguyen NH, Dodd-Eaton EB, Corredor JL, et al. Validating Risk Prediction Models for Multiple Primaries and Competing Cancer Outcomes in Families With Li-Fraumeni Syndrome Using Clinically Ascertained Data. JCO; Published online 3 April 2024. DOI: https://doi.org/10.1200/JCO.23.01926