02:16How to compress text to 1 bit per character | Compression is Intelligence Part 3
From the creator
Arithmetic codes and how they elucidate this trilogy's title Channel career fair: https://3b1b.com/talent New store items! https://3b1b.com/store Early views for supporters: https://3b1b.com/support Other resources you may enjoy. On Asymmetric numeral systems: https://arxiv.org/abs/0902.0271 https://demonstrations.wolfram.com/DataCompressionUsingAsymmetricNumeralSystems/ https://arxiv.org/pdf/2201.01741v2 https://arxiv.org/pdf/2510.13867 On Arithmetic Codes: https://dl.acm.org/doi/10.1145/214762.214771 Ilya Sutskever, An Observation on Generalization https://www.youtube.com/watch?v=AKMuA_TVz3A&t=24s A bit on prequential coding: https://yidingjiang.github.io/blog/post/curriculum/ A few other good papers/posts on the relationship between compression, prediction, and ML: https://deepmind.google/research/publications/39768/ https://www.hutter1.net/ai/uaibook.htm https://link.springer.com/chapter/10.1007/3-540-62858-4_69 https://www.sciencedirect.com/science/article/abs/pii/0005109878900055 Timestamps: 0:00 - The absurdity of Shannon’s estimate 6:27 - From text to real numbers 16:57 - Is this Shannon’s limit? 20:26 - Reframing pretraining 22:41 - Arithmetic codes in action 27:29 - Reflections on “Compression is Intelligence” 36:30 - Three ways to support Secret Enscreen Vlog: https://youtu.be/hoX3IiKXT-M ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim Music by Vincent Rubine
Published 2026-11-17 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.
Keep exploring Science
02:16
07:55
06:42
01:3813 Mind-Blowing Science Experiments You Can Do At Home! 🤯 #ScienceForKids #STEM #MagicTricks
Vaish The Academy
01:4015 Mind-Blowing Science Tricks You Can Do At Home! 🤯 #ScienceExperiments #STEM #ScienceTricks
Vaish The Academy
01:52
07:19DIY Water Purification Model | Easy Science Experiment & STEM Project for School
Creative fest