Jaan Tallinn would like us to survive

Jaan Tallinn is a founding engineer of Skype and Kazaa, and a co-founder of the Cambridge Centre for the Study of Existential Risk and Future of Life Institute. For more than fifteen years, he has been among the most consistent voices arguing that advanced AI poses an extinction-level risk to humanity.

Jaan became interested in AI risk in 2008 after reading the work of Eliezer Yudkowsky, a founding researcher of the field of AI alignment. In 2009, after a four-hour meeting with Yudkowsky, he made his first donation to what became the Machine Intelligence Research Institute (MIRI). He went on to become an early investor in DeepMind and led the first round into Anthropic – despite believing that AI development, if unchecked, could kill us all. His reasoning: his money displaced VC money rather than bringing Anthropic into existence, and the influence and proceeds that come with it help fund safety work.

In this conversation, we explore what Jaan believes the OpenAI/Hugging Face attack revealed about AI motivation and why he was less surprised than most. We discuss his evolving risk estimate, his case for an immediate moratorium on frontier model training, the technical and geopolitical case for hardware-based verification as a foundation for global AI governance, and why he asked Dario Amodei to collect competing term sheets before accepting his own investment.

—

Every timestamp

(00:00) Intro

(02:28) Growing up in Soviet-occupied Estonia

(05:54) How Jaan assesses AI risk and why he’s more optimistic now

(07:16) AI’s preferences and motivations

(12:44) The OpenAI Hugging Face incident

(16:12) Meeting with Eliezer Yudkowsky

(18:20) Everything as computation

(20:22) The Coxon moment

(23:05) Who dismisses AI alignment risks

(26:17) The AI race, the case for a pause, and safer alternatives

(32:47) AI 2027

(33:44) Investing in DeepMind and Anthropic

(40:03) The “big blob of compute” and competing approaches to AI

(44:42) What effective AI safeguards could look like

(48:15) Verifying countries’ compliance with AI safeguards

(50:15) AI kill switches and their limits

(52:11) Jaan’s love of dance and musical influences

(59:50) Jaan’s favorite movies

(1:01:41) What Jaan likes about AI

—

Jaan’s complete reading list

• Shadows of the Mind: A Search for the Missing Science of Consciousness: https://www.amazon.com/dp/0195106466

—

Other resources

• Skype: https://en.wikipedia.org/wiki/Skype
• Life is NOT a Journey - Alan Watts: https://www.youtube.com/watch?v=rBpaUICxEhk&t=10s
• AI Billionaire on Existential Risk: Jaan Tallinn – #112: https://www.manifold1.com/episodes/ai-billionaire-on-existential-risk-jaan-tallinn
• The Basic AI Drives: https://selfawaresystems.com/wp-content/uploads/2008/01/ai_drives_final.pdf
• The Hugging Face incident and the road ahead: https://openai.com/index/hugging-face-incident-and-the-road-ahead
• Rogue AI didn’t breach Hugging Face, human decisions did: https://thebulletin.org/2026/09/rogue-ai-didnt-breach-hugging-face-human-decisions-did
• MIRI: https://intelligence.org
• Jacob Coxon on X, “I resigned from Anthropic today.”: https://x.com/hilbertspaess/status/2097476196791709843
• AI 2027: https://ai-2027.com
• Google DeepMind: https://deepmind.google
• Inside the Biggest Feud in Artificial Intelligence: https://www.theatlantic.com/technology/2026/09/openai-v-anthropic-inside-biggest-rivalry-tech/688819
• The Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
• AI 2040: https://ai-2040.com
• The Glass Bead Game: https://www.amazon.com/dp/B00DO994WY
• Jaan’s favorite movies: https://jaan.info/favmovies
• The Odyssey: https://www.imdb.com/title/tt33764258
• On the Edge: https://www.imdb.com/title/tt0221559
• Limitless: https://www.imdb.com/title/tt1219289
• Ex Machina: https://www.imdb.com/title/tt0470752

—

Where to find Jaan Tallinn

Website: https://jaan.info

—

Production and marketing by penname.co. For inquiries about sponsoring the podcast, email jordan@penname.co.