Code Your Own Llama 4 LLM from Scratch – Full Course

From the creator

This course is a guide to understanding and implementing Llama 4. @vukrosic will teach you how to code Llama 4 from scratch. Code and presentations: https://github.com/vukrosic/courses Code DeepSeek V3 From Scratch: https://youtu.be/5avSMc79V-w ⭐️ Contents ⭐️ - 0:00:00 Introduction to the course - 0:00:15 Llama 4 Overview and Ranking - 0:00:26 Course Prerequisites - 0:00:43 Course Approach for Beginners - 0:01:27 Why Code Llama from Scratch? - 0:02:20 Understanding LLMs and Text Generation - 0:03:11 How LLMs Predict the Next Word - 0:04:13 Probability Distribution of Next Words - 0:05:11 The Role of Data in Prediction - 0:05:51 Probability Distribution and Word Prediction - 0:08:01 Sampling Techniques - 0:08:22 Greedy Sampling - 0:09:09 Random Sampling - 0:09:52 Top K Sampling - 0:11:02 Temperature Sampling for Controlling Randomness - 0:12:56 What are Tokens? - 0:13:52 Tokenization Example: "Hello world" - 0:14:30 How LLMs Learn Semantic Meaning - 0:15:23 Token Relationships and Context - 0:17:17 The Concept of Embeddings - 0:21:37 Tokenization Challenges - 0:22:15 Large Vocabulary Size - 0:23:28 Handling Misspellings and New Words - 0:28:42 Introducing Subword Tokens - 0:30:16 Byte Pair Encoding (BPE) Overview - 0:34:11 Understanding Vector Embeddings - 0:36:59 Visualizing Embeddings - 0:40:50 The Embedding Layer - 0:45:31 Token Indexing and Swapping Embeddings - 0:48:10 Coding Your Own Tokenizer - 0:49:41 Implementing Byte Pair Encoding -

Published 2025-04-24 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.