When a multi-LLM system hands work from one model to another through text — whether a router switching between models or agents collaborating on a task — the first model has to compress what it knows into text, and the next has to read that text and reconstruct the information it contains. That handoff can lose information and add inference time.
Cache-to-Cache (C2C), a technique proposed by researchers from Tsinghua University and other Chinese institutions, tries to address this problem by letting models exchange information through the internal representations stored in their key-value (KV) caches instead of relying only on intermediate text.
In the researchers’ experiments, C2C improved accuracy by around 3.1 to 5.4 percentage points over text-based communication and reduced latency across the model pairs tested. For enterprise teams building model routers or multi-agent systems, the result suggests that the communication layer itself can become an optimization target.
C2C also fits into a broader line of multi-model AI research toward letting models exchange internal representations directly, rather than forcing every handoff through text tokens. The work was published at ICLR 2026, and in September the researchers said they plan to release an “agent-managed KV-Cache” implementation along with a serving system.
Text is an expensive bottleneck between models
Multi-LLM systems come in different configurations. In collaborative systems, models take on different roles and exchange analyses, instructions or partial solutions. In routing systems, an orchestrator switches between models with different capabilities and costs as a conversation or workflow progresses.
These systems commonly rely on text tokens to carry information between models. The researchers argue that this creates three problems.
The first is information loss. A model represents context internally as high-dimensional numerical states. To communicate through text, it must compress part of that representation into a sequence of text tokens. The receiving model then has to interpret that sequence and build its own representation of what the first model meant, possibly losing information along the way.
Another problem is that textual information can be ambiguous. The researchers illustrate this with a Coder and Writer model modifying an HTML document. The Coder understands that <p> marks a particular structural location in the document, but its textual instruction does not communicate that information precisely enough. The Writer fails to put the requested content in the right place.
Finally, text-based communication imposes a heavy inference cost on the AI system. The first model must generate its message sequentially, token by token. Those tokens are then added to the second model’s input, which must process the longer context before it can generate a response. The longer the handoff, the more work both models have to do. In systems with long contexts or repeated agent handoffs, that communication overhead can quickly add up.
Turn the KV cache into a communication layer
C2C tries to avoid the extra text generation and the Receiver’s processing of that handoff by communicating through a representation the models already create: the KV cache.
When an LLM first processes a prompt, it runs a “prefill” stage that converts the input into internal key and value representations. These are stored in the KV cache and reused as the model generates its response. Those cached key and value states encode information the model derived from the context.
The researchers first tested whether the KV cache could serve as a communication medium. Their experiments showed that the information stored in a KV cache can be enriched without making the cache longer, and that a cache produced by one model can be transformed into the representation space of another model.
C2C builds on those findings. Both models process the same context. One acts as the “Sharer” and the other as the “Receiver.” A trained cache fuser maps the Sharer’s cache into the Receiver’s representation space, then combines it with the Receiver’s own cache. A learned gate controls which layers of the Receiver should receive the additional information.
In routing systems, this lets the Receiver incorporate information from the Sharer’s cache without requiring an intermediate text message. Both models still process the shared context to build their own KV caches. In collaborative systems, it creates another communication channel alongside text.
C2C in action
The researchers tested C2C on reasoning and knowledge benchmarks including MMLU-Redux, OpenBookQA, ARC-Challenge and C-Eval. With Qwen3-0.6B fixed as the Receiver and three different Sharers, C2C increased the Receiver’s average accuracy by around 9.6 to 11.9 percentage points over the Receiver working alone, beating text-to-text communication by around 3.1 to 5.4 points. These are benchmark results rather than tests of production agent workloads. The latency measurements were run with batch size 1 on a single Nvidia A100.
The latency improvement varied substantially by model pair. C2C was 3.46x, 1.51x and 14.41x faster than text-to-text communication across the three main Sharer configurations. The 14.41x result came with Qwen3-4B Base as the Sharer; the researchers say that model sometimes ignored the text-communication instructions and generated longer-than-expected messages, making the text baseline particularly slow.
In one experiment with text-to-text communication, the Sharer generated an average of 80 communication tokens. Generating those tokens took 1,312 milliseconds. The resulting message also increased the number of tokens the Receiver had to process. In contrast, C2C only required around 90 milliseconds of cache fusion and did not need to generate text tokens.
The technique also worked when the two models had different architectures or specializations. Tests included Gemma-to-Qwen, Qwen Math-to-Qwen and Qwen Coder-to-Qwen combinations. C2C outperformed text communication across all five heterogeneous and swapped model configurations reported in the experiment.
What it takes to deploy C2C
C2C does not require fine-tuning either participating LLM. The researchers freeze the Sharer and Receiver and train only the cache-fusion module using a standard next-token prediction objective.
However, cross-model communication requires engineering work. Different models can have different tokenizers, numbers of layers and representation sizes. C2C aligns the tokens generated by different tokenizers and maps corresponding transformer layers before fusing their caches. Training the bridge also has an upfront cost. The team has released the C2C code and configurations on GitHub under an Apache 2.0 license, along with pretrained C2C fuser checkpoints on Hugging Face.
One caveat is that because C2C needs to read, transform and replace internal KV-cache states, it requires an inference stack that exposes those internals. That makes it relevant today primarily to teams that control their own inference stack, not applications stitching together models solely through closed APIs.
LLMs may not need tokens to talk to each other
C2C fits into a broader line of research that questions whether text should remain the default interface inside multi-model AI systems.
One example is Nvidia’s research on cross-model KV-cache transfer. When a system switches models during a long-running session, the new model would normally have to process the accumulated context and build its cache again. Nvidia instead maps the existing cache into the target model’s format. On compatible model pairs, Nvidia’s researchers reported that the technique ran 2.7 to 25 times faster than re-prefilling.
While Nvidia’s technique transfers the full KV cache between the source and destination model, C2C has both models process the context, then combines their caches so the Receiver can benefit from another model’s semantic representation.
Another example in the field is RecursiveMAS, a multi-agent framework that replaces textual messages with continuous latent representations passed between agents. Its experiments reported up to 2.4x faster inference and a 75.6% reduction in token usage compared with its text-based recursive counterpart.
The mechanisms differ, but they share the same underlying bet: models may communicate more efficiently when they do not have to translate everything into text first.
For now, C2C is a research result, not a production architecture: it requires access to model internals, and its gains were measured on benchmarks, not real enterprise agent workloads.