このエピソードについて
In this episode of Risk! Engineers Talk Governance, due diligence engineers Richard Robinson and (Risk Engineer Achievement Award winner!) Gaye Francis discuss Risk, Uncertainty, and Hallucinations.
Prompted by a recent course on AI and probability, Richards exaplins how large language models actually work as giant inference engines predicting the "most likely" next word, and why that's fundamentally different from genuine risk assessment.
They explore why AI can widen the number of scenarios you can process without ever solving the harder problem of criticality — the rare, high-consequence events that haven't happened before, but still need to be found and controlled by human judgement.
If you'd like us to cover a specific topic or have any feedback we'd love to hear from you: email admin@r2a.com.au.
For further information on Richard and Gaye's consulting work with R2A, head to https://www.r2a.com.au, where you'll also find their booklets (store) and a sign-up for the quarterly newsletter to keep informed of our latest news and events.
Apto PPE is also available via the R2A online store.
Show Notes
- (00:56) Richard congratulates Gaye on winning the Risk Engineer Achievement Award at the Melbourne Engineering Excellence Awards, recognising 25+ years in risk due diligence, industry publications, podcasts, bushfire and public safety work, and advocacy for women in engineering
- (02:27) The founding of Apto (Women's) PPE and its impact at forcing the market to offer proper female PPE
- (04:30) Setting up today's topic: why people still default to thinking about risk as consequence and likelihood, rather than criticality and control
- (04:53) Richard's University of Helsinki AI course and the link between AI probability weighting and Markov chains, used at R2A for availability modelling
- (05:32) Different types of probability — fixed-outcome events (a coin toss) versus genuine future uncertainty (geopolitical shocks, oil markets)
- (06:18) How LLMs generate "hallucinations" (or Geoffrey Hinton's term, "confabulations"), tokenising context and predicting the statistically most likely next word, sometimes inventing plausible-sounding but false attributions
- (07:16–08:16) AI as a Monte Carlo-style tool: useful for running large numbers of trials fast, but this doesn't eliminate criticality, it only shrinks the pool while critical outcomes still have to be identified
- (09:08–09:42) The limits of AI's training cutoff; it can't flag risks that haven't happened before or reflect a rapidly changing context; identifying criticality still requires human judgement
- (10:26–11:12) The risk of the next generation treating AI output as "gospel" without questioning it
- (11:16–11:41) Wrap-up: risk as future uncertainty, AI as a tool for faster insight, but criticality and control remain R2A's core focus
