June 14, 2026|6 min read|UMB Advisors

Xiaomi’s MiMo V2.5 Hits 1000‑3000 TPS with DFlash & Persistent Kernel

Xiaomi’s recent announcement that its MiMo V2.5 model is now being served at 1 000‑3 000 tokens per second (tps) using a novel DFlash kernel paired with a persistent execution context marks a concrete step toward high‑throughput, locally‑run inference that does not rely on massive cloud‑scale GPU farms [2]. The claim is backed by a live demo on the MiMo blog and a Reddit thread where the team notes that the DFlash model weights are already available, with an open‑source release promised imminently [2]. This development is noteworthy not just for the raw numbers but because it showcases a software‑driven acceleration path that can make powerful language models accessible on modest hardware, reinforcing the “off the thumb” ethos of independence from large AI/cloud providers.

The core of Xiaomi’s achievement lies in the DFlash kernel, which appears to restructure how attention computations are cached and reused across successive token generations. By persisting key‑value states in a specialized memory layout and minimizing data movement between compute units, the kernel reduces the per‑token latency that typically bounds LLM serving speed. While the paper‑level details have not yet been published, the reported throughput places MiMo V2.5 in the same ballpark as specialized inference engines that have traditionally required multiple high‑end GPUs or custom ASICs. For perspective, a dual DGX Spark configuration running DeepSeek‑V4 Flash achieves roughly 350 tps aggregate when scaling to a 1 M‑token context, a figure that drops sharply when the model is forced into a single‑GPU scenario [3]. Xiaomi’s single‑system approach therefore offers a compelling alternative: comparable or superior throughput without the need for multi‑node interconnects or the $180 CX7 cable that the DGX Spark setup demands [3].

This result aligns with broader trends in efficient model design that have been gaining traction in the open‑source community. Recent discussions highlight how sparse attention, mixture‑of‑experts (MoE) architectures, latent KV compression, multi‑token prediction, and aggressive four‑bit quantization have collectively lowered the hardware barrier for running state‑of‑the‑art models locally [4]. Although Xiaomi has not disclosed whether MiMo V2.5 explicitly employs any of these techniques, the reported performance suggests that the DFlash kernel is able to exploit similar efficiencies—perhaps by compressing the KV cache or by reusing computed attention slices across tokens. In any case, the outcome reinforces the idea that algorithmic innovation can yield tangible speed gains even before the next generation of AI‑optimized silicon arrives.

From a systems perspective, the persistent kernel concept also hints at a shift away from the stateless request‑response model that dominates most LLM serving stacks. By keeping the model’s execution context alive across requests, the system avoids repeated weight loading and state reconstruction, which are major sources of overhead in traditional serving frameworks like vLLM or TensorRT‑LLM. This approach mirrors the benefits seen in WebAssembly‑based Python packaging, where publishing WASM wheels to PyPI allows libraries to be instantiated once and reused across multiple invocations without re‑compilation [20]. Just as WASM wheels reduce the friction of deploying Python tools in the browser, a persistent inference kernel reduces the friction of deploying large models on edge devices or on‑premises servers.

Of course, high throughput alone does not guarantee reliable behavior. Recent cautions about over‑reliance on large context windows remind us that pushing a model to generate many tokens quickly can exacerbate hallucinations or drift when the model’s internal state becomes noisy [6]. Xiaomi’s team has not yet published detailed quality metrics alongside the tps figures, so practitioners will need to evaluate whether the speed gains come at an acceptable cost in factual consistency or reasoning depth. The open‑source release of the DFlash model, once available, will allow the community to benchmark not only raw speed but also perplexity, latency‑variance, and robustness under varied prompting strategies.

The broader ecosystem already offers complementary tools that could amplify Xiaomi’s advance. For instance, the SQL‑to‑ER diagram runner that operates entirely in the browser demonstrates how complex workflows can be shifted to client‑side environments without uploading sensitive data [12]. Pairing such a tool with a locally served MiMo V2.5 instance could enable end‑to‑end, privacy‑preserving data‑analysis pipelines that run entirely on a user’s workstation. Similarly, the recent progress in ReactOS achieving 3D‑accelerated Half‑Life on real hardware shows that open‑source stacks are increasingly capable of handling demanding graphics workloads, suggesting that the same community drive could soon produce robust driver and runtime support for kernels like DFlash on a wider range of GPUs [22].

Looking ahead, the promised open‑source release of the DFlash model will be a critical inflection point. If the code is permissively licensed and includes clear instructions for building the persistent kernel on commodity GPUs (or even on emerging AI accelerators), we may see a wave of community‑driven forks that tailor the approach to specific domains—code generation, scientific simulation, or real‑time agent loops. The ability to serve a model at multiple thousands of tokens per second on a single GPU could also shift the economics of agent‑based workflows, where frequent tool calls and rapid reasoning loops currently incur significant latency penalties.

In summary, Xiaomi’s MiMo V2.5 demonstration is more than a headline‑grabbing throughput number; it is a tangible illustration of how software‑centric innovations—persistent kernels, specialized memory layouts, and efficient inference schemes—can democratize access to high‑performance language model serving. By situating this achievement within the current wave of efficient model techniques, contrasting it with heavyweight multi‑GPU benchmarks, and linking it to broader trends in portable, open‑source tooling, we see a clear path toward a future where powerful AI runs locally, privately, and at speeds that keep pace with the demands of interactive applications. The imminent open‑source release will be the litmus test: if the community can replicate and extend these results, the “off the thumb” vision of independent, self‑hosted AI may move from aspiration to everyday practice.

Sources

local inferenceopen modelsself-hostingAI hardwareoff the thumb

←All Insights