The Million-Token Context
How GLM models implement and efficiently serve a 1-million-token context window: IndexShare sparse attention, KVShare and MTP — the key change that cuts per-token FLOPs by 2.9×.
How does the GLM model achieve a million contexts?
Increasing the context limit from 200K to one million tokens is an engineering issue rather than a configuration parameter. Three modifications are key.
IndexShare sparse attention
Every four Transformer layers share a lightweight indexer. It is located at the first layer among the four, and its top-k index is reused by the other three layers — eliminating the dot product and top-k calculation in three-quarters of the layers. Result: Each token's FLOPs is reduced by 2.9 times under a context length of one million. IndexShare was introduced starting from a sequence length of 128K during the training mid-phase.
MTP combined with IndexShare and KVShare
The multi-token prediction layer runs the indexer once in the first draft step and reuses it in subsequent steps, eliminating the mismatch between the training/inference KV cache of the previous generation. Combined with rejection sampling and end-to-end TV loss, the accepted length increases from 4.56 to 5.47 — about 20% — across 7 MTP steps.
| Baseline | 4.56 |
| + IndexShare + KVShare | 5.10 |
| + Reject Sampling | 5.29 |
| + End-to-End TV Loss | 5.47 (+20%) |
Efficient Service for Million Contexts
After more than a few tens of thousands of tokens, the bottleneck is no longer computation, but the KV cache capacity, long context kernel, and CPU overhead. Three fixes: fine-grained memory management and parallelization based on LayerSplit, kernel optimization for context length scaling coordinated with the cache transfer pipeline, and CPU-side cache management, request scheduling, and runtime path optimization. The throughput advantage expands with the growth of context.
Normalized Throughput Advantage
Agent Reinforcement Learning Training
The training stack (slime) combines white-box and black-box rollouts, compact trajectories, and sub-agent workflows. Parallel OPD training merges more than a dozen expert models in about two days, and FP8 KV caching ensures that rollout memory does not exceed the budget.
Anti-Reward Hacker
Long-term RL can induce shortcuts, so suspicious actions are first passed through a rule-based recall filter, and then checked for accuracy by the LLM judge. Confirmed malicious actions are intercepted online and responded with virtual tool results, allowing rollouts to continue rather than be discarded. Based on Critic's PPO, compressible sub-traces are maintained for training on a single rollout.