"Neural networks are not already clean" by leogao
Failed to add items
Sorry, we are unable to add the item because your shopping cart is already at capacity.
Add to basket failed.
Please try again later
Add to Wish List failed.
Please try again later
Remove from Wish List failed.
Please try again later
Follow podcast failed
Unfollow podcast failed
"Neural networks are not already clean" by leogao
-
Narrated by:
-
By:
Nick Cammarata's recent post makes an important and true observation: in the past, people have been too quick to write off neurons as polysemantic, but if you look harder, there is actually often a simple pattern behind the activating examples. This historically caused people to overestimate how much polysemanticity there is. This is a really good point and we should update on it. But it's nowhere near a sufficient argument that we understand most of what neurons are doing, and these results do not show that interpretability is solved, or even that it is vastly easier than previously anticipated.
What does it take to explain a neuron?
It's unclear whether there even is a simple pattern that explains every neuron. But even if there is a simple pattern that matches the activating examples of a neuron, that still doesn’t mean that the explanation is correct. Suppose we look at the top activating examples of [...]
---
Outline:
(00:59) What does it take to explain a neuron?
(03:15) Simulation scores tell us whether we explained a neuron
(05:41) Beyond understanding neurons
The original text contained 7 footnotes which were omitted from this narration.
---
First published:
October 17th, 2026
Source:
https://www.lesswrong.com/posts/Dkh7pXKjoNWkkxQDz/neural-networks-are-not-already-clean
---
Narrated by TYPE III AUDIO.
adbl_web_anon_alc_button_suppression_t1
No reviews yet
In the spirit of reconciliation, Audible acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.