TENTANG EPISODE INI
If you are helping build something that might kill everyone, maybe stop doing that.
Ideally, when you do stop, tell everyone else to also stop building it.
That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone.
Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself.
Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear.
This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions.
Table of Contents
Outline:
(01:20) Scott Alexander's Core Claims
(05:06) My High Level Responses To Scott's Core Claims
(08:08) Briefly on Alignment
(10:11) Probabilities and How They Work
(13:25) Building a Thing That Might Kill Everyone is Deontologically Evil
(20:38) People Quitting Major Labs While Raising Alarms Obviously Works
(24:39) On AI Forecasting
(27:31) Advancing Capabilities Burns Timeline
(29:34) But China
(33:23) Ethical People Can Disagree With You
(34:45) There Are More Than Two Outcomes
(38:01) Efforts Are Complementary, Not Mutually Exclusive
(40:23) If Anyone Builds It, Everyone Dies
(41:55) Vaniver is Not Convinced
(46:19) There Is No Consensus That We Should Exclusively Focus on Plan A
(56:01) It's Not the Incentives, It's You
(58:08) Anthropic Is Not Optimally Pursuing These Plans
(01:04:00) Against Against Sura
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2026
Source:
https://www.lesswrong.com/posts/ziMP83D2JYWRQ5jtu/scott-alexander-s-against-sura-seems-deeply-confused
---
Narrated by TYPE III AUDIO.
Ideally, when you do stop, tell everyone else to also stop building it.
That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone.
Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself.
Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear.
This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions.
Table of Contents
- [...]
Outline:
(01:20) Scott Alexander's Core Claims
(05:06) My High Level Responses To Scott's Core Claims
(08:08) Briefly on Alignment
(10:11) Probabilities and How They Work
(13:25) Building a Thing That Might Kill Everyone is Deontologically Evil
(20:38) People Quitting Major Labs While Raising Alarms Obviously Works
(24:39) On AI Forecasting
(27:31) Advancing Capabilities Burns Timeline
(29:34) But China
(33:23) Ethical People Can Disagree With You
(34:45) There Are More Than Two Outcomes
(38:01) Efforts Are Complementary, Not Mutually Exclusive
(40:23) If Anyone Builds It, Everyone Dies
(41:55) Vaniver is Not Convinced
(46:19) There Is No Consensus That We Should Exclusively Focus on Plan A
(56:01) It's Not the Incentives, It's You
(58:08) Anthropic Is Not Optimally Pursuing These Plans
(01:04:00) Against Against Sura
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2026
Source:
https://www.lesswrong.com/posts/ziMP83D2JYWRQ5jtu/scott-alexander-s-against-sura-seems-deeply-confused
---
Narrated by TYPE III AUDIO.
Inggris
Amerika Serikat
TRANSKRIP 🔗
Are you the producer of this podcast?
Add a podcast transcript
Need Audio-to-Text?
Transcribe with Listen411 in Just 60 Seconds
CARI EPISODE TERDAHULU
Cari episode sebelumnya dari LessWrong (Curated & Popular).
EPISODE LAIN DALAM PODCAST INI
How do you get a friend to leave an abusive relationship? If you talk to people who work at women's shelters, they will tell you, but there is something resigned about the way they do it: they know that it's too difficult an ask. Those who can do it reliably are often people who are unluc…
Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. Heavily inspired by a conversation with Keshav Srinivasan and Richard Ngo during MATS 10.0. The week ahead of you is packed. This is…
Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect i…
We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought: “Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization press…
This is a link post. METR recently estimated that Anthropic is now making AI progress about twice as fast as its historical rate (they would not be surprised by anything from 1.3x to 3x), and guessed that “perhaps roughly equals 80%” of the current acceleration is due to AI. If they’re right, that …
Sanggahan: Podcast dan karya seni yang disematkan pada halaman ini berasal dari LessWrong, yang merupakan milik pemiliknya dan tidak berafiliasi atau didukung oleh Listen Notes, Inc.
Sunting
Terima kasih telah membantu agar basis data podcast tetap terbaru