DiffusionGemma Local Test | 4x Faster but How Accurate? | Text Generation & Coding with llama.cpp

Name: DiffusionGemma Local Test | 4x Faster but How Accurate? | Text Generation & Coding with llama.cpp
Uploaded: 2026-06-16T19:00:06.000Z
Duration: 10 min 34 s
Channel: Venelin Valkov
Description: Practical evaluation of DiffusionGemma, a 26B MoE diffusion model that promises 4x faster inference than autoregressive LLMs. Video tests the model locally with llama.cpp on speed, memory usage, and accuracy.

Venelin Valkov17 June 2026Watch on YouTube

Part of series

Ep. 7 · Coding Cpp Llama

View the series

What you'll learn

DiffusionGemma generates 256 tokens in one step instead of sequentially, putting parallel hardware to better use.
The model family was released on June 10, 2026 under Apache 2 and matches the Gemma 4 mixture of experts size.
On an Apple M5 Pro with 48 GB of unified memory, the model takes roughly 26 GB for a 4,096-token context.
The creator does not recommend the model for practical use yet and calls some test answers outright weak.

Frequently asked questions

How does DiffusionGemma differ from classical language models?

DiffusionGemma uses diffusion instead of autoregressive generation, producing 256 tokens at once rather than one at a time. This lets the model look both forward and backward across the output and put parallel hardware to better use.

Is DiffusionGemma as accurate as the classical Gemma 4 models?

Independent benchmarks show DiffusionGemma trailing the classical Gemma 4 models. On GPQA Diamond, DiffusionGemma 26B A4B reaches 73.2 percent, where Gemma 4 26B A4B lands on 84.3 percent, and Google itself recommends the standard models for maximum quality.

Can I already run DiffusionGemma locally with llama.cpp?

Only with a specialised pull request from the Unsloth authors, because the model does not yet run on the master branch of llama.cpp. It also needs substantial memory, roughly 26 GB on an M5 Pro for a 4,096-token context.

Do the creators recommend using DiffusionGemma in production?

No. The DiffusionGemma authors present it as a research model and advise against using it to replace the autoregressive Gemma 4 models. The reviewer in the video also advises against the model for practical use at this stage.

Topics

large language models diffusion models model evaluation local inference text generation

Sources

What is known about this topic outside the broadcast, and where it says so.

Description from the channel

DiffusionGemma is a 26B MoE diffusion model that promises 4x faster inference compared to the "conventional" autoregressive LLMs. We'll run it locally with llama.cpp, check the memory usage, inference speed and the overall accuracy on various tests. Blog post: https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/ GGUF model: https://huggingface.co/unsloth/diffusiongemma-26B-A4B-it-GGUF llama.cpp PR: https://github.com/ggml-org/llama.cpp/pull/24423 AI Academy: https://mlexpert.io/ Work with me: https://mlexpert.io/consulting LinkedIn: https://www.linkedin.com/in/venelin-valkov/ Follow me on X: https://twitter.com/venelin_valkov Discord: https://discord.gg/UaNPxVD6tv Subscribe: http://bit.ly/venelin-subscribe GitHub repository: https://github.com/curiousily/AI-Bootcamp 👍 Don't Forget to Like, Comment, and Subscribe for More Tutorials! Join this channel to get access to the perks and support my work: https://www.youtube.com/channel/UCoW_WzQNJVAjxo4osNAxd_g/join