MENGENAI EPISOD INI
What would it take to get assurance that we have actually explained a neuron? Certainly, it would be a good sign if neuron activations looked like they had a consistent pattern, and a bad sign if they looked like a random jumble of unrelated concepts.
Nick Cammarata's recent post makes an important and true observation: in the past, people have been too quick to write off neurons as polysemantic, but if you look harder, there is actually often a simple pattern behind the activating examples. This historically caused people to overestimate how much polysemanticity there is. This is a really good point and we should update on it. But it's nowhere near a sufficient argument that we understand most of what neurons are doing, and these results do not show that interpretability is solved, or even that it is vastly easier than previously anticipated.
What does it take to explain a neuron?
It's unclear whether there even is a simple pattern that explains every neuron. But even if there is a simple pattern that matches the activating examples of a neuron, that still doesn’t mean that the explanation is correct. Suppose we look at the top activating examples of [...]
---
Outline:
(00:59) What does it take to explain a neuron?
(03:15) Simulation scores tell us whether we explained a neuron
(05:41) Beyond understanding neurons
The original text contained 7 footnotes which were omitted from this narration.
---
First published:
October 17th, 2026
Source:
https://www.lesswrong.com/posts/Dkh7pXKjoNWkkxQDz/neural-networks-are-not-already-clean
---
Narrated by TYPE III AUDIO.
Nick Cammarata's recent post makes an important and true observation: in the past, people have been too quick to write off neurons as polysemantic, but if you look harder, there is actually often a simple pattern behind the activating examples. This historically caused people to overestimate how much polysemanticity there is. This is a really good point and we should update on it. But it's nowhere near a sufficient argument that we understand most of what neurons are doing, and these results do not show that interpretability is solved, or even that it is vastly easier than previously anticipated.
What does it take to explain a neuron?
It's unclear whether there even is a simple pattern that explains every neuron. But even if there is a simple pattern that matches the activating examples of a neuron, that still doesn’t mean that the explanation is correct. Suppose we look at the top activating examples of [...]
---
Outline:
(00:59) What does it take to explain a neuron?
(03:15) Simulation scores tell us whether we explained a neuron
(05:41) Beyond understanding neurons
The original text contained 7 footnotes which were omitted from this narration.
---
First published:
October 17th, 2026
Source:
https://www.lesswrong.com/posts/Dkh7pXKjoNWkkxQDz/neural-networks-are-not-already-clean
---
Narrated by TYPE III AUDIO.
Bahasa Inggeris
Amerika Syarikat
TRANSKRIP 🔗
Are you the producer of this podcast?
Add a podcast transcript
Need Audio-to-Text?
Transcribe with Listen411 in Just 60 Seconds
CARI EPISOD LEPAS
Cari episod lepas bagi LessWrong (Curated & Popular).
EPISOD LAIN DALAM PODCAST INI
I’m writing these lines in Alamogordo, New Mexico. I’m about as close as a civilian can get to the site of the Trinity test, the first nuclear bomb detonation in history. I fled here from the San Francisco Bay Area; after two weeks of being back I just couldn’t take it anymore. The fear-greed co ...
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another cat ...
For a while, we felt we had a pretty good grasp over LLM behaviour because we had to exert significant effort to elicit misaligned behaviour. Then, fairly suddenly, we moved to undereliciting misaligned behaviour. Let us recount:
For a while, there was a bit where safety research seemed to ov ...
The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering the gist of the dialogue rather than the exact words used. I’ve also swapped some pronouns and other personal identifiable details. Please ...
Ice cream
There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this ...
Penafian: Podcast dan karya seni yang dibenamkan pada halaman ini adalah daripada LessWrong, yang merupakan hak milik pemiliknya dan tidak bergabung dengan atau disokong oleh Listen Notes, Inc.
SUNTING
Terima kasih kerana membantu memastikan pangkalan data podcast dalam keadaan terkini.
