אודות פרק זה
In this episode of Data in Biotech, host Ross Katz sits down with Sadegh Salehi, Director of Research and Principal Scientist at Overjet, to explore what rigorous model evaluation actually looks like when the stakes are clinical.
Overjet builds FDA-cleared vision models that detect and quantify dental disease across billions of X-ray images from thousands of practices - a data problem with a staggering number of dimensions. Thirty-two teeth per adult patient, each with different morphology. Multiple image types capturing different anatomy. Fifteen to twenty sensor manufacturers producing perceptually distinct images, each with different contrast, resolution, and noise characteristics. And disease severity distributions ranging from barely visible early-stage decay to obvious pathology.
Sadegh walks through what it takes to evaluate models responsibly across all of those dimensions and discusses why aggregate metrics like F1 score can mask catastrophic failures on specific subgroups, how models find and exploit shortcuts in training data, and why the same flawed sampling that creates gaps in your training set also creates them in your test set.
He also traces Overjet's architectural evolution from over twenty narrow task-specific models to a single foundation model they call Unity, explains how treatment plan procedure codes provide a noisy but real production feedback signal, and describes how Overjet became one of the first companies to secure the FDA's Predetermined Change Control Plan (a framework that allows model updates without filing a new clearance each time.)
What you’ll learn in this episode:
>> Why aggregate evaluation metrics are insufficient for high-stakes medical AI
>> How models exploit shortcuts in training data: if all images from a rare sensor in the training set happen to be healthy, the model doesn't learn to read that sensor, it learns that the sensor means healthy, bypassing the visual task entirely and producing systematic false negatives in production
>> How Overjet evolved from over twenty narrow, sensor-specific and indication-specific models into a single foundation model called Unity, using noisy labels generated by the small models as the training signal for a much larger backbone, then building independent prediction heads for each clinical indication on top of it
>> Why the decision to keep prediction heads architecturally independent from one another was driven as much by FDA regulatory strategy as by modeling considerations
>> How Overjet uses dental treatment plan procedure codes as a production monitoring signal
Meet our guest:
Sadegh Salehi is Director of Research and Principal Scientist at Overjet, where he leads the team responsible for building, evaluating, and deploying FDA-cleared vision models for dental disease detection and quantification.
About the host:
Ross Katz is Principal and Data Science Lead at CorrDyn. Ross specializes in building intelligent data systems that empower biotech and healthcare organizations to extract insights and drive innovation.
- Connect with Ross Katz on LinkedIn: https://www.linkedin.com/in/b-ross-katz/
Connect with us:
- Follow the podcast for more insightful discussions on the latest in biotech and data science.
- Subscribe and leave a review if you enjoyed this episode!
Sponsored by…
This episode is brought to you by CorrDyn, the leader in data-driven solutions for biotech and healthcare. Discover how CorrDyn is helping organizations turn data into breakthroughs at CorrDyn. https://www.linkedin.com/company/corrdyn/
