Scaling Up (Part 1)

From the creator

In part one of this three part series on sharding and parallelism we’ll explore how to scale your Flax NNX models using JAX's powerful distributed computing capabilities, specifically its SPMD paradigm. If you're coming from PyTorch and have started using JAX and Flax NNX, you know that modern models often outgrow single accelerators. Let’s discuss JAX's approach to parallelism and how NNX integrates with it seamlessly. This episode will cover the "why" and "what" of distributed training, introducing the fundamental concepts of parallelism and the core JAX primitives needed to implement them. Resources: Learn more → https://goo.gle/learning-jax Subscribe to Google for Developers → https://goo.gle/developers Speaker: Robert Crowe

Published 2025-12-04 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.