Running Deepseek-R1 671B without a GPU

From the creator

We ran a giant AI model, the Deepseek-R1 671B FP16 model, on an AMD EPYC 9965 server to see if the CPU server could handle what many GPU servers cannot. What is more, we ran the large model alongside traditional virtualization workloads to see how that impacted performance. We even have a how-to so you can get started running AI in your virtualization cluster using Open WebUI and Ollama. STH Main Site Article: https://www.servethehome.com/running-the-deepseek-r1-671b-model-at-fp16-fidelity-alongside-amd-virtualized-workloads/ Substack: https://axautikgroupllc.substack.com/ STH Top 5 Weekly Newsletter: https://eepurl.com/dryM09 ---------------------------------------------------------------------- Become a STH YT Member and Support Us ---------------------------------------------------------------------- Join STH YouTube membership to support the channel: https://www.youtube.com/channel/UCv6J_jJa8GJqFwQNgNrMuww/join Professional Users Substack: https://axautikgroupllc.substack.com/ ---------------------------------------------------------------------- Where to Find STH ---------------------------------------------------------------------- STH Forums: https://forums.servethehome.com Follow on Twitter: https://twitter.com/ServeTheHome Follow on LinkedIn: https://www.linkedin.com/company/servethehome-com/ Follow on Facebook: https://www.facebook.com/ServeTheHome/ Follow on Instagram: https://www.instagram.com/servethehome/ ----------------------------------------------------

Published 2025-04-03 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.