معلومات عن هذه الحلقة
If you are helping build something that might kill everyone, maybe stop doing that.
Ideally, when you do stop, tell everyone else to also stop building it.
That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone.
Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself.
Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear.
This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions.
Table of Contents
Outline:
(01:20) Scott Alexander's Core Claims
(05:06) My High Level Responses To Scott's Core Claims
(08:08) Briefly on Alignment
(10:11) Probabilities and How They Work
(13:25) Building a Thing That Might Kill Everyone is Deontologically Evil
(20:38) People Quitting Major Labs While Raising Alarms Obviously Works
(24:39) On AI Forecasting
(27:31) Advancing Capabilities Burns Timeline
(29:34) But China
(33:23) Ethical People Can Disagree With You
(34:45) There Are More Than Two Outcomes
(38:01) Efforts Are Complementary, Not Mutually Exclusive
(40:23) If Anyone Builds It, Everyone Dies
(41:55) Vaniver is Not Convinced
(46:19) There Is No Consensus That We Should Exclusively Focus on Plan A
(56:01) It's Not the Incentives, It's You
(58:08) Anthropic Is Not Optimally Pursuing These Plans
(01:04:00) Against Against Sura
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2026
Source:
https://www.lesswrong.com/posts/ziMP83D2JYWRQ5jtu/scott-alexander-s-against-sura-seems-deeply-confused
---
Narrated by TYPE III AUDIO.
Ideally, when you do stop, tell everyone else to also stop building it.
That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone.
Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself.
Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear.
This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions.
Table of Contents
- [...]
Outline:
(01:20) Scott Alexander's Core Claims
(05:06) My High Level Responses To Scott's Core Claims
(08:08) Briefly on Alignment
(10:11) Probabilities and How They Work
(13:25) Building a Thing That Might Kill Everyone is Deontologically Evil
(20:38) People Quitting Major Labs While Raising Alarms Obviously Works
(24:39) On AI Forecasting
(27:31) Advancing Capabilities Burns Timeline
(29:34) But China
(33:23) Ethical People Can Disagree With You
(34:45) There Are More Than Two Outcomes
(38:01) Efforts Are Complementary, Not Mutually Exclusive
(40:23) If Anyone Builds It, Everyone Dies
(41:55) Vaniver is Not Convinced
(46:19) There Is No Consensus That We Should Exclusively Focus on Plan A
(56:01) It's Not the Incentives, It's You
(58:08) Anthropic Is Not Optimally Pursuing These Plans
(01:04:00) Against Against Sura
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2026
Source:
https://www.lesswrong.com/posts/ziMP83D2JYWRQ5jtu/scott-alexander-s-against-sura-seems-deeply-confused
---
Narrated by TYPE III AUDIO.
الإنجليزية
الولايات المتحدة
النص 🔗
Are you the producer of this podcast?
Add a podcast transcript
Need Audio-to-Text?
Transcribe with Listen411 in Just 60 Seconds
البحث في الحلقات الماضية
البحث في الحلقات السابقة من LessWrong (Curated & Popular).
حلقات أخرى في هذا البودكاست
I'm part of a team of 26 researchers that recently released a 212-page paper proposing a new plan for pausing AI, the "hardwired pause." The paper is called "The Feasibility of a Hardwired Pause of Frontier AI Training." Our analysis suggests that, under this hardwired paus…
METR recently released a report estimating that Anthropic is getting an overall 1.7x acceleration in AI capability progress as a result of AI accelerating their work. As many have noted (see e.g. Ajeya's post), this 1.7x acceleration implies that research is being sped up by much more than 1.7…
I’m pretty interested in conveying the costs of letting AI development proceed unchecked to the general public. In relation to this, I feel like I run into a lot of disagreements with people around me. Misc very quickly written points on which my own impression has seemed very different from others…
Image by Claude Opus 5.5 using Python (NumPy and Pillow) This is written in my free time and personal capacity. Two Frames In AI safety, there are a couple of tensions at the heart of the field, which are interrelated. Talent: we are talent-constrained and must recruit. But jobs and fellowships tur…
“Claude Mythos Preview is, on essentially every dimension we can measure, the best-aligned model that we have released to date by a significant margin.” – Claude Mythos Preview system card “Mythos 5 consistently continues attacking under many edits to the compaction, even in settings where it corr…
إخلاء المسؤولية: البودكاست والأعمال الفنية المضمنة في هذه الصفحة مأخوذة من LessWrong، وهي مملوكة لمالكها وليست تابعة لشركة Listen Notes, Inc أو معتمدة منها.
تعديل
شكراً على مساعدتنا في الحفاظ على قاعدة بيانات البودكاست مُحدَّثة.
