The GLM 1M context window is the defining feature of the current flagship GLM model, and it is not a configuration flag. Raising a usable ceiling from 200,000 to 1,000,000 tokens required changes to the attention architecture, to speculative decoding and to the serving stack — three separate pieces of engineering that only produce a usable window when combined.
This page explains each of them in plain language, with the published figures attached. The short version: a shared indexer cuts per-token computation by 2.9× at 1M context length, a redesigned multi-token-prediction layer raises speculative-decoding acceptance length by about 20%, and a set of memory and kernel changes stops KV-cache capacity from becoming the wall. All figures are as published by the model authors.
The naive problem with long context is quadratic attention cost, and sparse attention solves that in principle by having each token attend to a selected subset of previous tokens rather than all of them. But sparse attention needs an indexer — a lightweight mechanism that decides which tokens are worth attending to — and running that indexer in every layer reintroduces a cost that grows with context length. At 200,000 tokens that overhead is tolerable. At a million it is not.
The second problem is that past a few hundred thousand tokens the bottleneck stops being compute at all. It becomes KV-cache capacity, long-context kernel efficiency and CPU-side overhead — a memory and systems problem rather than a mathematics one. A model can be architecturally capable of a million-token window and still be unservable if the cache does not fit or the scheduler stalls.
IndexShare is the architectural answer. Instead of every transformer layer computing its own top-k indices, every four layers share a single lightweight indexer. It sits on the first of the four, and its top-k indices are reused by the other three — which removes the indexer dot-product and top-k selection work from three of every four layers.
The published result is a 2.9× reduction in per-token FLOPs at 1M context length, with no quality trade-off reported at long range. IndexShare was used from mid-training onward at a 128K sequence length rather than being bolted on at inference time, which matters: a sharing scheme introduced only at serving time would create a mismatch between how the model was trained and how it runs.
Multi-token prediction speeds up generation by drafting several tokens ahead and then verifying them, and its efficiency is measured by acceptance length — how many drafted tokens survive verification on average. In the previous generation the MTP layer suffered a mismatch between the KV cache used during training and the one used at inference, which capped acceptance length at 4.56 across 7 MTP steps.
The current design runs the indexer once on the first draft step and reuses the indices for later steps, sharing both indices and KV state. Adding rejection sampling to the speculative decoding path and an end-to-end total-variation loss lifts acceptance length to 5.47 — roughly 20% more tokens accepted per verification pass. The published ablation is below.
| Baseline | 4.56 |
| + IndexShare + KVShare | 5.10 |
| + Rejection Sampling | 5.29 |
| + End-to-end TV Loss | 5.47 (+20%) |
All figures on this page are results as published by the model authors. glmmodel.com reports them; it does not run these evaluations.
Three changes address the systems side. LayerSplit-based memory management gives finer-grained control over how model state is partitioned and parallelised, so cache allocation stops being all-or-nothing. Kernel work scaled to long contexts is coordinated with the cache-transfer pipeline, so compute and memory movement overlap rather than serialise. And CPU-side cache management, request scheduling and runtime path tuning remove overhead that is invisible at short context but dominant at long context.
The reported effect is that the throughput advantage widens as context grows — the longer the prompt, the larger the gap against a naive serving path. That is the property that turns a 1M context window from a specification into something you can actually run under load.
Relative throughput advantage as context length grows, as described by the model authors.
Read the GLM model benchmarks, then put a real model to work. The playground above is free and needs nothing from you; if you want a fuller AI toolkit, our partner's free tier starts here.