关于这一集
Thanks to (in alphabetical order) Joshua Batson, Roger Grosse, Jeremy Hadfield, Jared Kaplan, Jan Leike, Jack Lindsey, Monte MacDiarmid, Francesco Mosconi, Chris Olah, Ethan Perez, Sara Price, Ansh Radhakrishnan, Fabien Roger, Buck Shlegeris, Drake Thomas, and Kate Woolverton for useful discussions, comments, and feedback.
Though there are certainly some issues, I think most current large language models are pretty well aligned. Despite its alignment faking, my favorite is probably Claude 3 Opus, and if you asked me to pick between the CEV of Claude 3 Opus and that of a median human, I think it'd be a pretty close call. So, overall, I'm quite positive on the alignment of current models! And yet, I remain very worried about alignment in the future. This is my attempt to explain why that is.
What makes alignment hard?
I really like this graph from Christopher Olah for illustrating different levels of alignment difficulty:
If the only thing that we have to do to solve alignment is train away easily detectable behavioral issues—that is, issues like reward hacking or agentic misalignment where there is a straightforward behavioral alignment issue that we can detect and evaluate—then we are very much [...]
---
Outline:
(01:04) What makes alignment hard?
(02:36) Outer alignment
(04:07) Inner alignment
(06:16) Misalignment from pre-training
(07:18) Misaligned personas
(11:05) Misalignment from long-horizon RL
(13:01) What should we be doing?
---
First published:
November 27th, 2025
Source:
https://www.lesswrong.com/posts/epjuxGnSPof3GnMSL/alignment-remains-a-hard-unsolved-problem
---
Narrated by TYPE III AUDIO.
---
Though there are certainly some issues, I think most current large language models are pretty well aligned. Despite its alignment faking, my favorite is probably Claude 3 Opus, and if you asked me to pick between the CEV of Claude 3 Opus and that of a median human, I think it'd be a pretty close call. So, overall, I'm quite positive on the alignment of current models! And yet, I remain very worried about alignment in the future. This is my attempt to explain why that is.
What makes alignment hard?
I really like this graph from Christopher Olah for illustrating different levels of alignment difficulty:
If the only thing that we have to do to solve alignment is train away easily detectable behavioral issues—that is, issues like reward hacking or agentic misalignment where there is a straightforward behavioral alignment issue that we can detect and evaluate—then we are very much [...]
---
Outline:
(01:04) What makes alignment hard?
(02:36) Outer alignment
(04:07) Inner alignment
(06:16) Misalignment from pre-training
(07:18) Misaligned personas
(11:05) Misalignment from long-horizon RL
(13:01) What should we be doing?
---
First published:
November 27th, 2025
Source:
https://www.lesswrong.com/posts/epjuxGnSPof3GnMSL/alignment-remains-a-hard-unsolved-problem
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
英语
美国
在这一集里
文字稿 🔗
Are you the producer of this podcast?
Add a podcast transcript
Need Audio-to-Text?
Transcribe with Listen411 in Just 60 Seconds
本播客中的其他单集
Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community. Next month, I am moving to Beijing and joining Tsinghua University's College of AI as an assistant professor working on frontier AI safety, especially loss of control. I h...
As our understanding of mechanistic interpretability and steering techniques increase, for both LLMs and uploaded connectomes, there is a corresponding "hellish capabilities" increase. In layman's terms, we get better at torturing the digital minds we create. Our brief history with...
This is a link post. Announcing www.huggingface.wtf, a site that explains: "Hugging Face. Wtf?" I wrote the most accessible-but-complete explanation I could. Along the way, I made a nice AI Quotes database that I think will be useful for people who already understand the Hugging Face in...
Ice cream
There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this a...
Why LM Psychology? One of the great ironies of modern language models is that we create them, we know exactly what computations constitute them, and yet we do not really understand them. More precisely, we do not understand them on a level of abstraction that lets us reliably predict how their b...
免责声明: 本页面嵌入的播客和封面图片来自 LessWrong，归其所有者所有，与 Listen Notes, Inc. 无关联，也未获得 Listen Notes, Inc. 背书。
