Meta's Muse Glimmer: An Agentic AI Model

Meta open-weighted a 30B agentic model that runs on one 24 GB consumer GPU, an RTX 4090 or a 32 GB Mac, at its full 131,072 token context. Four architectural decisions got it there. In this post we'll build up from what an agentic model actually has to do. Then we'll go through Glimmer's attention layout, its channel-scoped output, its XML tool calls, and the block diffusion drafter that makes it fast. We'll finish with what runs on your hardware, and a lab you can point at your own Glimmer server.

On 10 August 2026 Meta released Muse Glimmer, a 30B open-weight agentic model under Apache 2.0 that runs on one 24 GB consumer GPU. Four architectural decisions got it there, starting with an attention layout that turns a 104 GB KV cache into 1.7 GB. This post builds them up from first principles, traces one request through the whole system, and ends with a runnable lab you can point at your own Glimmer server.

Introduction

On 10 August 2026 Meta open-weighted Muse Glimmer, a 30 billion parameter agentic model under Apache 2.0. Open-weighted means the trained parameters are published for anyone to download and run, which is not the same as publishing the training data or the code. Those 30 billion parameters are normally stored at 16 bits each, which comes to 58 GB, more than any consumer graphics card holds. Meta also ships the model quantized, meaning each parameter is squeezed into 4 bits instead of 16. That version is 16.8 GB, and it fits on a single 24 GB GPU such as an RTX 4090, or on a Mac with 32 GB of unified memory.

Many 30B models fit on a 24 GB GPU. Glimmer is unusual because it still fits after hours of tool use, the workload that normally blows a local model's memory budget. Every tool result is appended to the context, so the cost climbs for as long as the session runs. Fitting that into 24 GB took four architectural decisions, and this post takes them in order.

Everything here is checked against the model card, the GGUF repository, Meta's research blog, the vLLM recipe and the DFlash paper. Where something is our reading of the architecture rather than a claim Meta makes, we say so.

What Muse Glimmer is

You will see two names, and they are not the same model. Muse Spark is Meta's larger model, and Glimmer was distilled from it. Distilled means Glimmer was trained to copy Spark, so Spark is the teacher and Glimmer is the student. Spark's weights have never been released, so you cannot run it. Muse Glimmer is the 30B student, and it is what this post is about. It inherits what Spark learned and does not exceed it. Comparing it with the big hosted models compares a student with other companies' teachers.

It is a dense model, so every parameter runs for every token. That is a deliberate choice against the mixture-of-experts designs common at this size. It costs efficiency and buys predictable memory use, which is what you want when the goal is a fixed 24 GB budget. The feed-forward block is SwiGLU. It sends the input down two paths and multiplies them together, so one path is a volume control on the other. Parts of the signal get turned up, parts get turned down, and what survives carries on to the next layer.

The problem it solves

A chat model reads a question and writes an answer. An agentic model reads a task, calls a tool, reads what came back, decides it needs another tool, and keeps going until it is done. That loop changes what the model has to be good at.

• Precise tool calls, thousands of times: A 99% success rate sounds fine until you make 300 calls in a session, at which point three are malformed and the run is broken.
• A context that keeps growing: Every tool result is appended, so the KV cache, meaning the stored keys and values from every token so far, grows with it.
• Speed that compounds: One step at 20 tokens a second feels fast. Forty steps is a coffee break, and it has to recover from its own errors along the way rather than repeat a failing call.

Each of those pushes against a fixed memory budget, and the growing context pushes hardest. Glimmer answers with four decisions, one per pressure. Share the key-value heads to cut the cost per token. Let most layers forget to stop that cost growing. Separate the channels for a reliable loop, and guess in blocks to kill the latency. A section each.

Why this is hard

A transformer stores a key and a value for every token it has seen, in every layer, or it would recompute the whole conversation for each new token. That storage is the KV cache, and its size per token per layer is

Take a conventional transformer of Glimmer's shape, where each of the 32 query heads gets its own key-value pair, at a 131,072 token context.

104 GB, before a single weight is loaded. A 30B model at 4-bit is only about 17 GB of weights, so the cache alone would be six times the model and several times the memory of any consumer GPU. Quantization, meaning storing each weight in fewer bits, shrinks the weights and does nothing for the cache.

This is why parameter count tells you so little about whether a model fits. Two models with identical parameter counts can differ by a factor of sixty in what they need to run a long session.

Decisions 1 and 2, the attention layout

Assume a team of 52 people is reading one long document. If they all memorise the whole thing, that is accurate but expensive. If they all remember just the last page, that is cheap but they lose the plot. Glimmer does neither.

• 39 local layers: Sliding window attention over the last 2,048 tokens, meaning attention restricted to a recent window so the layer's memory cost stops growing once the window is full. These carry RoPE, the rotary position embedding, with theta 500,000.
• 13 global layers: Full attention over the whole 131,072 token context, and no RoPE at all. These are the only layers whose cache keeps growing.

Combine that with grouped-query attention, where 32 query heads share just 2 key-value heads, and the arithmetic changes completely.

1.7 GB against 104. The two decisions together are worth a factor of about 61, and neither changes the parameter count by a single weight. Both had been used before, in the Llama and Mistral lineages. Interleaving them 3 to 1 rather than picking one is the new part.

Why no RoPE on the global layers

Meta states the layout without explaining it, so what follows is the standard reading. RoPE marks a token's position by rotating its query and key vectors. It is reliable over short distances and gets vague past the lengths it was trained on, which is why long-context models drift at the far end. Glimmer keeps it inside the 2,048 token windows, where distances stay short. The global layers carry no position encoding at all, so there is nothing to extrapolate and nothing to break at 131,072 tokens.

Each mechanism is used only where it is good. Local layers handle precise nearby structure with RoPE, global layers handle loose long-range recall without it, instead of one mechanism being stretched over both jobs.

There is a real cost. Anything from 50,000 tokens back reaches the current token through 13 layers instead of 52, so distant recall is weaker than in a full-attention model of the same size. That suits an agent, which needs recent steps in detail and older ones in outline. It hurts if you need an exact quote from deep in a document.

Decision 3, channel-scoped output

Most reasoning models wrap thinking in think tags and treat everything else as the answer. Glimmer tags every message with a recipient instead, so treating it like a think-tag model gets you an empty string back with no clue why.

The recipient drives your loop. self is private reasoning you log and never display, a tool name is a request to run that tool, and user is the answer, the only one that ends the turn.

• Never stop on the end-of-message marker: It means end of message, not end of turn. A tool call ends with it and the turn continues once the result comes back. Only the end-of-turn and end-of-text markers are real stop tokens. Get this wrong and the model seems to ask for a tool and then go silent.
• Never strip the special tokens: The channel markers are special tokens, so a decoder running with the usual skip-special-tokens setting deletes the delimiters that separate reasoning from the answer, and all three channels collapse into one string. This is why vLLM's reasoning parser forces that setting off.

That wants_tool property is the whole loop condition. A turn with reasoning, a tool call and no final message is the normal shape of a working step.

Where reasoning strength actually gets set

Reasoning strength is Glimmer's one latency knob, with values low, medium, high and xhigh, and Meta recommends high or xhigh for agentic and coding work. The model card says to set it with a Reasoning strength: low line in your system prompt. But your system prompt is not the last thing the model reads. The server runs every request through the chat template, and that template adds its own reasoning-strength line after yours. It says high unless you change it there, and the model follows whichever line comes last. So if you set low and nothing speeds up, this is why.

ATEM, tool calls in XML

Nearly every tool-calling model emits JSON. Glimmer emits this.

That looks like a step backwards until you think about what a model has to do to produce valid JSON. What follows is the standard case for XML-shaped calls rather than a claim Meta makes.

• JSON is all-or-nothing: A call is only well formed once the final brace lands, and one dropped token invalidates the whole object. Models do drop tokens over hours of tool use.
• JSON needs escaping: Code passed as an argument becomes a thicket of backslash-n and backslash-quote, and one wrong escape breaks everything.
• ATEM degrades gently: Each parameter has its own closing tag, so a malformed value damages one argument instead of the call. Values are raw text, so a shell command goes in verbatim.

What ATEM gives up is types. Every value arrives as text, so top_k reaches you as the characters 3, not the number 3. Convert it using the tool schema before you run anything, or the number stays a string.

When one of your tools fails, send the error back to the model as text instead of throwing it. A thrown error stops the loop. An error the model can read is just another observation, so it tries something else and the run continues.

Decision 4, guessing several tokens at once

Generating one token means reading all 17 GB of weights out of the GPU memory, doing a small amount of maths with them, and discarding the read. The time goes into moving that data, not into the maths, which is what memory-bound means, and the GPU sits idle while it waits. Speculative decoding puts that idle capacity to work. A small model, the drafter, guesses the next few tokens. The big model, called the target, checks the whole batch in one pass, and you keep the guesses it agrees with. Checking sixteen guesses costs barely more than checking one, because either way the weights are read once.

Speculative decoding is lossless. The big model still decides every token, so the output is identical to what you would have got without it. You are trading idle parallel compute for latency.

Glimmer ships a small model for this job, called DFlash. It comes from a 2026 paper by a group at UC San Diego. Their equation for the average time per token shows where the speed comes from.

That says you pay for one round of drafting and one round of verifying, then divide that bill across however many tokens you kept. Tau is the acceptance length, the average number of guesses accepted per cycle.

The best known drafter before DFlash is EAGLE-3, and like every drafter before it, it writes one guess at a time. It predicts the next token, then the one after that, and so on, so sixteen guesses meant sixteen passes through it. Guessing more raises tau, but it raises the drafting bill by just as much, and the two cancel out. That is why earlier methods stall at roughly two to three times faster.

DFlash writes the whole block in one pass instead. It starts with sixteen blank positions and fills them in together, a technique called block diffusion, so drafting sixteen guesses costs about what drafting one costs. Now that extra guesses are nearly free, the drafter can afford to be bigger and think harder. DFlash uses five layers, where EAGLE-3 used one.

One more trick makes the guesses good enough to be worth checking. As the big model works, DFlash takes the half-finished calculations from five of its layers and feeds them into every layer of the drafter. That gives the drafter a head start, because it can see what the big model is already computing instead of guessing from scratch. EAGLE-3 handed the drafter that information once, at the start. DFlash gives it to every layer, and the researchers built both versions to check that this is what makes the difference.

DFlash comes from a paper by Chen, Liang and Liu at UC San Diego, ICML 2026. It is not Meta's work. Glimmer adopts the technique and ships a trained drafter for it, which is a different thing from inventing it.

The paper reports over 6 times on smaller models under ideal conditions. Meta measures 3.1 times on a 30B at 4-bit on an RTX 5090, at batch size one under llama.cpp, and that is the number that matters to you. The rest goes on sampling, turning tokens back into text, general runtime overhead, and the fact that rejections arrive in clusters rather than one at a time.

How it was trained

Meta describes three phases, at a high level.

• Pre-training: Logit distillation, meaning Glimmer is trained to match Muse Spark's full output distribution over the vocabulary rather than only the token the teacher picked. That passes on more information per example, which is why distilled models reach a given quality on less data.
• Mid-training: Extended-context, agent-heavy data with reasoning traces, which is what teaches the model to behave across a long session rather than a single exchange.
• Post-training: Supervised fine-tuning, meaning training on example pairs of input and desired output, plus on-policy distillation and reinforcement learning across reasoning, coding and agentic domains.

On-policy distillation matters most here. Ordinary distillation trains the student on the teacher's outputs, so it only ever sees situations the teacher would have created. On-policy distillation instead has the teacher grade what the student itself produced, so it learns to recover from its own mistakes. For a model doing a hundred steps in a row, that beats being right first time.

Meta has not published the training data, the token count or the compute budget. Open weights are not open data.

Under the hood, one request end to end

You ask Glimmer, running on your own machine with a documentation-search tool attached, what sliding window it uses on its local layers.

• 1. Tokenize: The chat template renders your messages into the exact token string the model was trained on, and the tokenizer splits it against a 202,048 token vocabulary. An attached image goes through the frozen perception encoder into up to 4,096 visual tokens, interleaved here.
• 2. Prefill: This is the prefill. All prompt tokens run through the 52 layers in one pass and the keys and values are written into the cache. The 39 local layers keep the last 2,048 positions, the 13 global layers keep everything. Compute-heavy, and it happens once.
• 3. Decode: The DFlash drafter proposes sixteen tokens in one pass. The target verifies all sixteen, keeps the correct prefix, corrects the first wrong token, repeats. On an RTX 5090 that is roughly 233 tokens a second instead of 75.
• 4. Channels come out: A message to self, ended with the end-of-message marker, which does not stop generation. Then a message addressed to search_docs carrying an ATEM block, ended the same way.
• 5. Tool, then answer: Your code sees a tool recipient rather than the user, converts top_k from string to integer, runs the search and appends the result. The model reads it and emits a message to the user ending with the end-of-turn marker, and your loop stops.

That is the entire agent, no framework. Everything Glimmer-specific sits in the parsing, so swapping in a JSON tool-calling model leaves the loop unchanged.

Quantization and local inference

Glimmer at full precision is 58 GB of weights, which needs a data-center GPU and defeats the point. Think of quantization as rounding. Trained at 16 bits per weight and stored at 4, it keeps roughly one significant figure instead of four. Each weight is slightly wrong, but a transformer averages over billions so the errors mostly cancel. Modern 4-bit formats also work in small blocks with a shared scale and keep the layers that matter most at higher precision, which is what the dynamic in UD-Q4_K_XL means.

The kquant figures are Meta's, and those builds live in the official GGUF repository. The UD- builds are Unsloth's and Meta makes no claim about them.

File size is not the memory you need. Add the KV cache, 1.4 GB for images, 1.6 GB for the drafter, and about a gigabyte of runtime overhead. That gap is where most out-of-memory errors come from. On 24 GB, kquant-17gb fits at full context with both optional files at about 22.5 GB. kquant-dynamic does not.

Running it yourself

Runtime support is unusually broad for a launch-day model. GGUF, the single file carrying weights, tokenizer and chat template together, is read by llama.cpp, Ollama and LM Studio, and is the only route exposing the DFlash drafter directly. Unsloth and TorchTitan handle fine-tuning, meaning continuing training on your own data. vLLM and SGLang serve at scale with paged attention and continuous batching. ExecuTorch and MLX target edge devices and Apple silicon, where unified memory means the RAM figure is also the usable VRAM figure.

• -md and -ngld: The first loads the DFlash drafter, the second puts its layers on the GPU. Set one and forget the other and the drafter runs on the CPU while the GPU waits, which can make speculative decoding slower than none at all.
• -np 4: Four parallel slots, and the context divides between them, so each gets 32,768 tokens rather than 131,072.
• --jinja: Use the chat template embedded in the GGUF. Without it the channel markers never appear and quality drops in a way that looks like a bad model.

ollama run hf.co/meta-models/Muse-Glimmer-30B-GGUF is the one-line version, and it hides the drafter and the vision projector, so you get neither speculative decoding nor image input. vLLM needs both parsers together, since they key off the same framing and the reasoning parser is what keeps special tokens alive.

Use temperature 1.0, top-p 0.95 and top-k 64, which Meta, Unsloth and the vLLM recipe all publish. Greedy decoding is a bad idea here, because a reasoning model's output length varies even at a fixed seed, so temperature 0 buys no reproducibility and costs quality.

The lab, and five experiments

The companion project muse-glimmer-lab turns each decision into something you can run and change.

Experiments 4 and 5 are the two that need no server, being arithmetic over the published architecture and the paper's latency equation. Experiment 4 pays for the whole lab, and every number comes from the model card, so glimmer/config.py is the architecture. Set kv_heads to 8 and watch the cache quadruple.

Experiment 5 predicts 5.7x where Meta measures 3.1x, and the script says so rather than hiding it. It assumes rejections are independent when they cluster, and ignores runtime overhead. Read it for the shape of the argument.

How it performs

Meta's own figures, so read them as you would any vendor's. It does publish the results where Glimmer loses.

Glimmer leads by a wide margin on tool orchestration and multi-step research, which is what it was post-trained for, and trails Qwen3.6-27B on computer use and TerminalBench. Tuned hard for one shape of agentic work does not mean better at every shape of it. SWE-Bench Verified and Pro are different benchmarks, so its 76.0 and 51.2 are not in conflict.

Against the alternatives

On the Siren AgentDojo evaluation Meta reports a 28.4% attack success rate alongside 94.2 utility. Prompt injection means text the model reads, such as a web page or a file, carrying instructions that hijack what it does next. A model that follows instructions well follows the wrong ones well, and running locally moves the trust boundary rather than removing it.

Limitations

• Long-range recall is weaker, which follows from the design: Only 13 of 52 layers see beyond 2,048 tokens, so an exact quote from the middle of a long document is a harder ask than for a full-attention model.
• Multi-step reasoning still breaks, and Meta says so: The model card admits errors in multi-step reasoning despite the agentic tuning. Long runs need checkpoints and a step limit.
• Prompt injection lands over a quarter of the time: The 28.4% figure is Meta's own. Keep a human in front of any tool that writes, sends or spends.
• Its ceiling is its teacher: No amount of prompting gets past Muse Spark. Video is frames, audio is unsupported, and the cutoff is January 2026.
• Underrepresented languages degrade: Meta claims over 100 languages and says quality may drop on the less represented ones, without saying which.

The failure mode to design against is the quiet one. A run that drifts because the model half-remembers a tool result from 40 steps ago produces a confident answer with no error anywhere in the trace. Step limits and checkpoints catch that. Watching for exceptions does not.

What to remember

Two ideas here outlast this model. The first is that architecture is a memory budget rather than a parameter count. The same 30 billion parameters need 104 GB of cache in one layout and 1.7 GB in another. The second is that diffusion has found a job it wins at. It was never better than autoregressive generation at writing text, so DFlash uses it only for guessing and leaves the deciding to the model that checks every token.

• Two attention decisions make it fit: 32 query heads share 2 key-value heads, and 39 of 52 layers see only the last 2,048 tokens. That turns a 104 GB KV cache into 1.7 GB without changing a parameter, and RoPE sits only on the local layers so nothing has to extrapolate. Do this arithmetic before you download any model.
• Output is channel-scoped and tool calls are XML: Never stop on the end-of-message marker, and never let a decoder strip special tokens. Both make the model look broken when the parsing is what broke. ATEM then hands you every argument as a string, so convert against your schema.
• DFlash is a lossless block diffusion drafter: 16 tokens per forward pass, conditioned on the target's hidden states, verified in parallel. Meta measures 3.1x on an RTX 5090, and the technique is UC San Diego's. File size is not memory, so add the cache, the vision projector and the drafter before sizing a GPU.

What to learn next

• Attention mechanics, then positional encoding: The memory argument rests on knowing what keys and values are and why they get cached. Then read why RoPE degrades past its training length, and the no-RoPE-on-global-layers choice stops looking odd.
• Quantization properly: Block-wise scales and mixed precision are the difference between picking a quant by file size and picking one by what it costs you.
• Agent loop engineering: Step limits, checkpoints, idempotent tools and error-as-text. The loop here is forty lines because everything hard about agents is in what surrounds it.
• Then the paper: Start with the tracks in this app, run experiment 4, then read the DFlash paper. It is short and unusually clear about what each choice is worth.

Sources

• Model card and research blog: The model card for architecture, sampling defaults, evaluations and stated limitations. The research blog for the training phases, the distillation source and the measured speedups.
• Official GGUF repository: meta-models/Muse-Glimmer-30B-GGUF for quant names, file sizes, the companion vision and drafter files, and the llama.cpp flags.
• Serving and fine-tuning docs: The vLLM recipe for the channel format, ATEM syntax and parser flags. The Unsloth guide for community dynamic quants and fine-tuning.
• The DFlash paper: arXiv:2602.06036, Chen, Liang and Liu, UC San Diego, ICML 2026. The drafter, equation 1 and the ablations.

When two of these disagree, the model card wins. Every number in the companion lab has a comment saying where it came from.