tubeshowcase●1,669,243 videos♡ My collection 0
← Science

Science37:25

How to compress text to 1 bit per character | Compression is Intelligence Part 3

3Blue1Brown ↗

From the creator

Arithmetic codes and how they elucidate this trilogy's title Channel career fair: https://3b1b.com/talent New store items! https://3b1b.com/store Early views for supporters: https://3b1b.com/support Other resources you may enjoy. On Asymmetric numeral systems: https://arxiv.org/abs/0902.0271 https://demonstrations.wolfram.com/DataCompressionUsingAsymmetricNumeralSystems/ https://arxiv.org/pdf/2201.01741v2 https://arxiv.org/pdf/2510.13867 On Arithmetic Codes: https://dl.acm.org/doi/10.1145/214762.214771 Ilya Sutskever, An Observation on Generalization https://www.youtube.com/watch?v=AKMuA_TVz3A&t=24s A bit on prequential coding: https://yidingjiang.github.io/blog/post/curriculum/ A few other good papers/posts on the relationship between compression, prediction, and ML: https://deepmind.google/research/publications/39768/ https://www.hutter1.net/ai/uaibook.htm https://link.springer.com/chapter/10.1007/3-540-62858-4_69 https://www.sciencedirect.com/science/article/abs/pii/0005109878900055 Timestamps: 0:00 - The absurdity of Shannon’s estimate 6:27 - From text to real numbers 16:57 - Is this Shannon’s limit? 20:26 - Reframing pretraining 22:41 - Arithmetic codes in action 27:29 - Reflections on “Compression is Intelligence” 36:30 - Three ways to support Secret Enscreen Vlog: https://youtu.be/hoX3IiKXT-M ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim Music by Vincent Rubine

Published 2026-11-17 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.

Keep exploring Science