“Condensation” by abramdemski
Failed to add items
Sorry, we are unable to add the item because your shopping cart is already at capacity.
Add to basket failed.
Please try again later
Add to Wish List failed.
Please try again later
Remove from Wish List failed.
Please try again later
Follow podcast failed
Unfollow podcast failed
“Condensation” by abramdemski
-
Narrated by:
-
By:
About this listen
The goal of this post is to popularize Sam's theory and to give my own perspective on it; however, it will not be a full explanation of the math. For technical details, I suggest reading Sam's paper.
Brief Summary
Shannon's information theory focuses on the question of how to encode information when you have to encode everything. You get to design the coding scheme, but the information you'll have to encode is unknown (and you have some subjective probability distribution over what it will be). Your objective is to minimize the total expected code-length.
Algorithmic information theory similarly focuses on minimizing the total code-length, but it uses a "more objective" distribution (a universal algorithmic distribution), and a fixed coding scheme (some programming language). This allows it to talk about the minimum code-length of specific data (talking about particulars rather than average [...]
---
Outline:
(00:45) Brief Summary
(02:35) Shannons Information Theory
(07:21) Universal Codes
(11:13) Condensation
(12:52) Universal Data-Structure?
(15:30) Well-Organized Notebooks
(18:18) Random Variables
(18:54) Givens
(19:50) Underlying Space
(20:33) Latents
(21:21) Contributions
(21:39) Top
(22:24) Bottoms
(22:55) Score
(24:29) Perfect Condensation
(25:52) Interpretability Solved?
(26:38) Condensation isnt as tight an abstraction as information theory.
(27:40) Condensation isnt a very good model of cognition.
(29:46) Much work to be done!
The original text contained 15 footnotes which were omitted from this narration.
---
First published:
November 9th, 2025
Source:
https://www.lesswrong.com/posts/BstHXPgQyfeNnLjjp/condensation
---
Narrated by TYPE III AUDIO.
No reviews yet
In the spirit of reconciliation, Audible acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.