A misconfiguration in the SGLang inference framework caused Cloudflare's GLM-5.3 deployment to return cached responses from one tenant to a different tenant. Before the issue was caught and corrected, 155 cross-tenant prompt responses were observed. The incident is a concrete example of how AI infrastructure configuration choices can create data boundaries that neither the operator nor the tenant expects or intends.
What happened
SGLang is an open-source framework for serving large language models that supports disaggregated prefill and decode. In this architecture, separate worker pools handle the prefill phase, which processes the input prompt and builds the KV cache, and the decode phase, which generates tokens. The two pools share a KV cache store. The design improves throughput by allowing prefill and decode to scale independently.
The misconfiguration in Cloudflare's GLM-5.3 deployment set the KV cache pool size smaller than the model's context window. When the pool filled, SGLang's cache eviction logic did not correctly isolate per-tenant cache entries before reuse. As a result, a new request received a cached response that belonged to a previous request from a different tenant. Cloudflare identified the issue through internal monitoring, corrected the configuration, and audited affected requests to determine the scope of exposure.
The SGLang architecture gap
SGLang's disaggregated serving mode is relatively new, and the cache isolation guarantees under capacity pressure are not prominently documented in the project's deployment guides. The framework assumes that operators will size the KV pool to accommodate the full context window plus headroom. When that assumption is violated, the cache eviction path can return stale entries rather than forcing a fresh prefill. This is a latent assumption that becomes a security property when the serving layer is multi-tenant.
GLM-5.3 is a 1-trillion-parameter multimodal model released by Zhipu AI as open weights. Cloudflare's deployment of it through Workers AI makes it accessible to developers building on Cloudflare's platform. The cross-tenant leak affected users of that public API.
What to check before deploying disaggregated serving
If you are running or planning to run SGLang in disaggregated prefill/decode mode for a multi-tenant workload, verify that the KV pool size is at least as large as the maximum context window you will serve. Set this conservatively, accounting for concurrent requests and peak load. Do not rely on the framework's cache eviction behavior as a data isolation mechanism. For workloads where tenant data must not cross boundaries, treat the KV cache as a trust boundary and test it explicitly before moving to production.
For operators of any LLM inference infrastructure, this incident illustrates a class of misconfiguration risk that is distinct from traditional application vulnerabilities. There is no CVE here. The code did what it was configured to do. The risk emerged from a capacity assumption that became a trust boundary assumption at the wrong moment. Reviewing your inference stack's caching behavior under load, and in conditions where the cache is near capacity, is a reasonable configuration audit to add to your deployment checklist.
Gigia Tsiklauri is a Security Architect and founder of Infosec.ge. Get in touch if you want to discuss AI infrastructure security or LLM deployment risk.