Effectiveness of AI for Prostate Cancer Risk Assessment

At the Society of Urologic Oncology Annual Meeting, Dr. Randie White presented a study investigating the effectiveness of artificial intelligence (AI) in reviewing pathology reports and clinical vignettes for patients with prostate cancer.

Dr. White and colleagues identified AI as a potential tool to accurately create risk assessments and recommendations for patients.

From January 2022 to January 2023, researchers conducted a retrospective review of localized patients. After completing their first biopsy, the patients received consultation and risk stratification from a urologic oncologist in accordance with the National Comprehensive Cancer Network (NCCN) guidelines.

The generative AI model ChatGPT was fed a prostate cancer risk stratification algorithm following NCCN Version 1.2023, and investigators then tested the model to ensure its accuracy. After providing the model with patients’ clinical and histological features, researchers prompted ChatGPT to “risk stratify the following patient.” They also asked the model for initial therapy and additional evaluation recommendations.

Of the 60 participating patients, the AI model provided an accurate risk stratification for 39 (65%). While the model incorrectly risk stratified 21 of the patients, it directed them to an adjacent risk group. The generative AI did categorize 14 patients into the intermediate-risk group, but it required extra prompting to label them with either favorable or unfavorable risk.

Researchers reported that the AI model prescribed the correct treatment and imaging modalities to each risk group.

Dr. White and colleagues concluded that generative AI was not effective in algorithmic risk stratification with standardized synoptic reports.

“The implications of inaccurate assessments need to be considered as more health care professionals and systems look to [incorporate] generative AI into clinical tools,” the researchers wrote. “A healthy amount of skepticism and close review [are] needed to vet any clinical tool.”

They noted that an experienced physician should still analyze patients’ information to provide accurate recommendations.