C2C lets AI models communicate through KV caches instead of text

When a multi-LLM system hands off work through text, the first model compresses its internal state into tokens and the second model has to reconstruct that information. That process loses information and adds inference time. Cache-to-Cache (C2C), a technique presented at ICLR 2026 by researchers from Tsinghua University, lets models communicate through their internal KV caches instead. The result: accuracy improvements of 9.6 to 11.9 percentage points on benchmarks and latency reductions of up to 14x in some configurations. For teams building model routers or multi-agent systems, this suggests the communication layer itself can be optimized, but only if you control your own inference stack.

How C2C works

C2C uses a Sharer and a Receiver model. Both process the same context to build their own KV caches. A trained cache fuser maps the Sharer's KV representations into the Receiver's representation space, and a learned gate controls which layers of the Receiver receive the additional information. No fine-tuning of either LLM is required; only the fusion module is trained. The technique works across heterogeneous model pairs, including Gemma-to-Qwen, Qwen Math-to-Qwen, and Qwen Coder-to-Qwen.

The accuracy and latency gains

The researchers tested C2C on MMLU-Redux, OpenBookQA, ARC-Challenge, and C-Eval. With Qwen3-0.6B fixed as the Receiver and three different Sharers, C2C increased the Receiver's average accuracy by 9.6 to 11.9 percentage points over the Receiver alone, and by 3.1 to 5.4 points over text-to-text communication. Latency varied by model pair: 3.46x, 1.51x, and 14.41x faster than text handoffs. The 14.41x result came with Qwen3-4B Base, which sometimes ignored text instructions and generated long messages. In one experiment, generating 80 communication tokens took 1,312 milliseconds, while C2C's cache fusion took about 90 milliseconds.

What this means for builders

C2C requires access to model internals (KV cache states), so it is relevant today only for teams that run their own inference stack (vLLM, TGI, etc.) and can integrate the fusion module. Teams using closed APIs like OpenAI or Anthropic cannot use it. The researchers released code and pretrained fuser checkpoints under Apache 2.0 on GitHub and Hugging Face. Related work includes Nvidia's cross-model KV-cache transfer and RecursiveMAS, which share the same bet: models communicate more efficiently when they don't have to translate everything into text first.

Caveats

C2C is a research result, not a production architecture. Its gains were measured on benchmarks, not real agent workloads. Latency measurements used batch size 1 on a single Nvidia A100. Cross-model alignment requires engineering work for different tokenizers and layer counts. The technique is not plug-and-play, and its production viability depends on inference stack support and workload characteristics.