Machine Learning and External Validation of the IDENTIFY Risk Calculator for Patients with Haematuria Referred to Secondary Care for Suspected Urinary Tract Cancer

• Published in: European urology focus
• Main Authors: Khadhouri, Sinan, Hramyka, Artsiom, Gallagher, Kevin, Ippoliti, Simona, Edison, Marie, Alexander, Cameron, Kulkarni, Meghana, Zimmermann, Eleanor, Nathan, Arjun, Orecchia, Luca, Piazza, Pietro, Mak, David, Pyrgidis, Nikolaos, Narayan, Prabhat, Abad Lopez, Pablo, Nawaz, Faisal, Tran, Trung-Thanh, Claps, Francesco, Gomez Rivas, Juan, Alonso, Santiago, Chibuzo, Ijeoma, Gutierrez Hidalgo, Beatriz, Teoh, Jeremy, Marcq, Gautier, Kelsey, Tom, Tijerina, Adan, Ali, Ahmed, Nic an Riogh, Aisling, Szostek, Alexandra, Dhanasekaran, Ananda, Atayi, Andrew, Ashpak, Ashna, Teixeira, Bernardo, Maria Scornajenghi, Carlo, Marramaque, Carolina, Ho Chee Kong, Christopher, Crewe, Claire, Griffiths, Daniel, Amporore, Daniele, Sarkar, Debashis, Bheenick, Divya, Hogan, Donnacha, Orakwe, Dubem, Checcucci, Enrico, Lozano Palacio, Fernando, Prata, Francesco, Hatzichristodoulou, Georgios, Karagiannidis, Georgios, Russo, Giorgio, Abozied, Hesham, Colvin, Holly, Ali, Iftikhar, Fakhradiyev, Ildar, Sokolakis, Ioannis, Tsikopoulos, Ioannis, Chong, James, Bondad, Jasper, Luis Bauza Quetglas, Jose, Antonio Herranz Yague, Jose, Colombo Stenstrom, Juan, de Mello, Kathryn Bandeira, Brodie, Katie, Tzelves, Lazaros, Lazaros, Lazarou, Rico, Luis, Lilis, Lydia, Ullmann, Maha, Srour, Mahmoud, Mustafa, Mohamed, Eyad Takahji, Mohammad, Almusimie, Mohammed, Shakeel Inder, Mohammud, Elgamal, Mostafa, Ali, Muhammad-Kabir, Binnawara, Muhannud, Ernesto Morales Palacios, Nelson, Santoni, Nicola, Maheshkumar, Pattabiraman, Sountoulides, Petros, Moreno, Pilar, Banthia, Ravi, Mohammad, Sabir, Abu, Sadiq, Weber, Stephan, Klatte, Tobias, Trung Thanh, Tran, Yeoh, W.S., Feuer, Zach, Elahi, Zain, Gall, Zara
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 20.06.2024
• Subjects: Bladder cancer; Cancer risk; Haematuria; Prediction; Predictive model; Renal cancer; Risk calculator; Upper tract urothelial cancer; Urinary tract cancer; Validation; Validation; Renal cancer; Upper tract urothelial cancer; Risk calculator; Predictive model; Prediction; Haematuria; Bladder cancer; Urinary tract cancer; Cancer risk
• ISSN: 2405-4569; 2405-4569
• DOI: 10.1016/j.euf.2024.06.004

The IDENTIFY risk calculator is valid for urologists to risk stratify patients referred to secondary care with haematuria. Very-low-risk patients may avoid investigations. High-risk patients have more aggressive cancers and should be prioritised for investigation. The IDENTIFY study developed a model to predict urinary tract cancer using patient characteristics from a large multicentre, international cohort of patients referred with haematuria. In addition to calculating an individual’s cancer risk, it proposes thresholds to stratify them into very-low-risk (<1%), low-risk (1–<5%), intermediate-risk (5–<20%), and high-risk (≥20%) groups. To externally validate the IDENTIFY haematuria risk calculator and compare traditional regression with machine learning algorithms. Prospective data were collected on patients referred to secondary care with new haematuria. Data were collected for patient variables included in the IDENTIFY risk calculator, cancer outcome, and TNM staging. Machine learning methods were used to evaluate whether better models than those developed with traditional regression methods existed. The area under the receiver operating characteristic curve (AUC) for the detection of urinary tract cancer, calibration coefficient, calibration in the large (CITL), and Brier score were determined. There were 3582 patients in the validation cohort. The development and validation cohorts were well matched. The AUC of the IDENTIFY risk calculator on the validation cohort was 0.78. This improved to 0.80 on a subanalysis of urothelial cancer prevalent countries alone, with a calibration slope of 1.04, CITL of 0.24, and Brier score of 0.14. The best machine learning model was Random Forest, which achieved an AUC of 0.76 on the validation cohort. There were no cancers stratified to the very-low-risk group in the validation cohort. Most cancers were stratified to the intermediate- and high-risk groups, with more aggressive cancers in higher-risk groups. The IDENTIFY risk calculator performed well at predicting cancer in patients referred with haematuria on external validation. This tool can be used by urologists to better counsel patients on their cancer risks, to prioritise diagnostic resources on appropriate patients, and to avoid unnecessary invasive procedures in those with a very low risk of cancer. We previously developed a calculator that predicts patients’ risk of cancer when they have blood in their urine, based on their personal characteristics. We have validated this risk calculator, by testing it on a separate group of patients to ensure that it works as expected. Most patients found to have cancer tended to be in the higher-risk groups and had more aggressive types of cancer with a higher risk. This tool can be used by clinicians to fast-track high-risk patients based on the calculator and investigate them more thoroughly.