Imagine relying on a cancer diagnosis from an AI tool that’s essentially guessing based on shortcuts rather than real biology. That’s the alarming reality researchers are warning us about, as AI pathology models may be taking unreliable detours to identify cancer biomarkers. But here’s where it gets controversial: while these tools show promise, their accuracy might be built on shaky ground, potentially jeopardizing patient care. Let’s dive into why this matters and what it means for the future of AI in medicine.
Recent findings published in Nature Biomedical Engineering (https://www.nature.com/articles/s41551-026-01616-8) reveal that AI tools designed to detect molecular biomarkers from histological images often rely on correlational relationships with clinicopathological features. This means they’re not truly understanding the biology behind the biomarkers but instead using 'shortcuts' to make predictions. For instance, it’s like judging a book by its cover—useful in a pinch, but not a reliable method for deeper analysis.
'It’s akin to assessing a restaurant’s quality by the length of its queue,' explains Fayyazul Amir Afsar Minhas, PhD, Associate Professor at the University of Warwick. 'While a long line might suggest popularity, it doesn’t reveal the chef’s skill or the food’s taste.' Similarly, many AI models focus on obvious tissue features or correlations between biomarkers rather than isolating the specific signals that matter. And this is the part most people miss: when conditions change, these shortcuts often fail, leaving the models vulnerable to inaccuracies.
Nasir Rajpoot, PhD, Professor of Computational Pathology, emphasizes the need for rigorous evaluation: 'We can’t rely solely on headline accuracies. To truly impact medicine, AI predictions must be tested for bias and confounding effects.' This isn’t just a technical detail—it’s a call to rethink how we validate AI in healthcare.
Study Methods
To uncover these issues, researchers analyzed over 8,000 tissue samples from patients with breast, colorectal, lung, and endometrial cancers. They compared the performance of leading deep-learning models using permutation testing and stratification analyses. These methods revealed how models often falter when biomarkers are interdependent or mutually exclusive, leading to misleading predictions.
Key Findings
One striking example involves BRAF mutations in colorectal cancer. Instead of identifying the true BRAF signal, AI tools often detect its relationship with microsatellite instability (MSI) status, conflating the two. 'A model that can’t distinguish between MSI-high and BRAF status might score high on accuracy metrics but fails in clinical utility,' the authors note. This highlights a broader issue: AI models must be evaluated not just for overall accuracy but for their ability to disentangle correlated biomarkers with different treatment implications.
The study also warns that if test conditions change, model performance could plummet, especially in patient subgroups with altered or abnormal factors. While AI tools hold value in cancer research and treatment, they should be used cautiously. The authors advocate for stratification-based evaluation frameworks to identify biases and develop more trustworthy models.
'This isn’t a rejection of AI in pathology,' Dr. Minhas clarifies. 'It’s a wake-up call. Current models excel in controlled settings but lack genuine biological understanding. Until we have robust evaluation standards, these tools shouldn’t replace molecular testing.' Clinicians and researchers must recognize their limitations and proceed with care.
Controversy & Discussion
Here’s the controversial question: Are we rushing AI into healthcare without fully understanding its limitations? While AI shows immense potential, this study suggests we’re not yet ready to fully trust it with life-or-death decisions. What do you think? Is AI in pathology a game-changer or a risky gamble? Share your thoughts in the comments—let’s spark a debate!
DISCLOSURE: Dr. Branson is affiliated with GSK. For full author disclosures, visit Nature Biomedical Engineering (https://www.nature.com/articles/s41551-026-01616-8). The content in this post has not been reviewed by the American Society of Clinical Oncology, Inc. (ASCO®) and does not necessarily reflect ASCO®’s views.
