このエピソードについて
For most of life sciences history, validation has been a snapshot — freeze the configuration, prove it behaves as designed, trust the system until you change it. Nuno Valério has spent his career inside that paradigm, and he's now one of the clearest public voices on what has to change for it to survive the AI era. As Head of Innovation, R&D Quality at Merck, Nuno is building AI governance frameworks for pharma R&D in real time — and he joins Matt Neal for a wide-ranging conversation about validating probabilistic systems, the trap of blanket "human-in-the-loop" thinking, and what genuine trust looks like when the model itself keeps changing.
Key TopicsAI as a liberator — when you stay the driver. Nuno's framing: AI is an enabler and a modulator, but the moment you let it produce your voice instead of you being behind the voice, it becomes hollow. The F1 analogy: what was great about watching Senna and Prost wasn't the cars, it was the art of the driver — the late brake that was almost too late, but not quite. AI is a powerful car. The driver still matters.
Set the model to challenge you. A practical antidote to the validation-loop trained into LLMs: prompt the model to push back, ask clarifying questions, and always offer a different angle. Sometimes the angle is irrelevant; sometimes it reshapes the whole question. It's how you keep the tool from collapsing into agreeable blandness.
The expertise paradox. AI is hugely powerful when you know your subject deeply — you can dig with a backhoe instead of a shovel. When you don't, it sounds great and can be completely wrong, and you won't know to push back. Matt's framing: when you really know something, you notice how not great it is on first blush.
AI as the first alien intelligence. Not alien in the extraterrestrial sense — alien in the sense of an intelligence that originated outside the patterns of natural selection that produced us. We've never met one before. The implication: we shouldn't assume it will behave like the only kind of intelligence we already know.
Trust architecture — validating the workflow, not the model. The old validation paradigm — take a snapshot of a deterministic system, freeze the configuration, trust it holds — doesn't survive probabilistic models whose outputs change with each input. Nuno's framework validates the whole ecosystem around the model: the tool, the human reviewing the output, the infrastructure, the guardrails, and the drift monitoring that flags when the model wanders. The goal isn't perfection — it's predictability you can sign under.
The human-in-the-loop trap. Putting a human everywhere isn't governance — it's burnout. Picture the reviewer at 5pm with 300 outputs to validate and a partner waiting at home. The first 257 were perfect, so he clicks through 258, 259, 260. "Human-in-the-loop" needs to mean human-on-the-loop where it matters — triggered by drift, risk thresholds, or signals the model is operating outside its trusted envelope.
Risk-based proportionality. A model that summarizes a meeting doesn't carry the same risk as one producing a safety report for a submission. The validation effort should reflect the consequence of failure. Quality has been doing risk-based work for decades — sampling, focusing where it counts, accepting you can't be everywhere. AI doesn't change that principle; it raises the stakes for applying it well.
The customization trap. Nuno's pushback on Matt's optimism about Veeva and Salesforce implementations: pharma companies routinely insist they're special, customize the standard configuration to match how they already work, and then can't absorb new features. AI capabilities increasingly only work — or only work well — on standard configurations. The cost of "specialness" is now showing up in the roadmap. And requirements gathered from people doing it the old way produce new systems that look exactly like the old systems.
Data quality, compounded over decades. Pharma's data is messy because there was never an incentive to fix it. Decades of operations stack up. Synergies across silos and regions matter only if the underlying data can be connected — which is exactly why frontier AI labs see life sciences as so much opportunity. Nuno's advice: don't try to fix 50 years; cut a reasonable line and move forward from useful data.
The GIP provocation. Matt's controversial proposal: the industry is missing a standard. GMP and GCP cover their domains. The little "x" in GxP gets stretched until everything is high-risk — and the result is fear-based bottlenecks. He proposes Good Information Practice — a discipline grounded in modern systems that trace every click, every change, every reason. If a spreadsheet column took three months to add, the real risk isn't governance; it's the columns you stopped adding. Nuno's response: he's wary of more letters, but agrees the binary GxP / non-GxP switch is broken, and proportionality has to be applied inside GxP too.
Sandboxes and pre-competitive collaboration. Nuno's call for shared, experimental spaces where industry, regulators, and vendors define what "good" looks like together — modeled on aviation safety. Pre-competitive information isn't IP. We can all get better at what everyone has to do, without giving up what makes anyone different. He sees the beginnings of that maturity in the sector, and signs from regulators that make him hopeful.
The dawn of AI maturity. Quality is a culture used to knowing what it's talking about — built from decades of guidelines, mistakes, and corpus. AI shifts every professional out of that seat. The only honest path forward, in Nuno's framing, is to think out loud, share the work, and accept that nobody has it all figured out yet.
Notable Quotes"If you use AI just to produce your voice instead of you being behind that voice, it becomes hollow."
"What I was seeing was not the cars. What I was seeing was the art of the person at the wheel."
"AI might be the first alien intelligence — not in the sense of being from outside Earth, but in the sense of being originated outside of our patterns."
"Trust, to me, is predictability that you can sign under."
"He actually was very thorough. He checked everything. This one is certainly fine as well. Click, click, click."
"Every pharma company thinks they are very special. And then they pay the price of that specialty."
References, People & Resources"The age of AI shifts everyone — every professional — from their seat."
Guest & Company
- Nuno Valério on LinkedIn — Head of Innovation, R&D Quality, Merck
- Merck (KGaA) — Darmstadt, Germany
Events & Public Work
- Clinical Trial Innovation Summit 2026 — Basel, 24 June 2026 (Nuno speaks on designing AI governance from both sides of the wall)
Platforms & Tools Discussed
- Veeva and Salesforce — referenced on standard vs. customized configurations
- Anthropic Claude, OpenAI ChatGPT, and the broader LLM landscape
Concepts Referenced
- Trust architecture (provenance, drift monitoring, human-on-the-loop, predictability)
- Deterministic vs. probabilistic validation
- Risk-based proportionality in GxP
- Good Information Practice (GIP) — Matt's proposed framing
- Pre-competitive collaboration and regulatory sandboxes
Everyone relaxes when you say "human in the loop." Nobody pictures the reviewer at 5pm, 257 clean outputs deep, clicking approve on the 258th because the first 257 were fine. The loop isn't a safeguard; it's an architecture problem. Validation used to prove a system does what you specified; with probabilistic systems the spec can't save you, so the real question stops being *does it work* and becomes *under what conditions can I sign under it.* That's the shift I care about. That's what I mean by trust architecture; not a new framework I'm selling, more a way of framing what our industry already half-know.
- Check out additional materials from Nuno:
TA # 3 - The "On Innovation" interview with Roberto Zicari at ODBMS
Operations Utopia - Where Regops, Innovation, Technology, and Execution Meet.
Disclaimer: This podcast reflects only the opinion of the podcaster and guests and does not reflect those of their organizations, system vendors, or service provider
Original show theme "Little Sammy" by Matt Neal
