CPU‑Only TTS Benchmark Shows Inflect‑Nano‑v1 Leads in Speed, Supertonic‑3 Wins in Quality

A fresh head‑to‑head comparison of three open‑weight text‑to‑speech models running purely on CPU has surfaced, offering concrete numbers that matter for anyone looking to run voice capabilities locally or on modest hardware [2]. The benchmark pits Kokoro 82M, Supertonic‑3 (in 2‑step and 5‑step variants), and Inflect‑Nano‑v1 (4.6 M parameters) against each other on a modest Intel Xeon with four cores and 15.6 GB of RAM — no GPU in sight. Each configuration was timed across six text lengths, warmed up, and repeated five times, yielding 150 measured runs. Every audio sample was scored with the UTMOS 22‑strong model, turning subjective impression into an objective MOS (mean opinion score).

The headline numbers are striking. Inflect‑Nano‑v1 achieves the lowest real‑time factor (RTF) of 0.1376, meaning it synthesizes speech roughly 7.3× faster than real time on this CPU setup. Its MOS, however, reads 3.48, which the author notes is likely inflated by the model’s tendency to over‑emphasize certain phonetics. Supertonic‑3’s 5‑step configuration trades speed for fidelity: RTF climbs to 0.3164 (about 3.2× real‑time) but MOS jumps to 4.37, a clear perceptual win. The 2‑step Supertonic‑3 sits in the middle with RTF 0.1781 and a low MOS 1.53, illustrating how fewer diffusion steps degrade quality sharply. Kokoro 82M, while not quoted in the excerpt, falls somewhere between these extremes in the full table.

Why does this matter for the broader AI landscape? First, it reinforces the “off the thumb” mantra: high‑quality speech synthesis no longer demands a GPU‑heavy cloud endpoint. By showing that a sub‑5 M‑parameter model can outrun real‑time on a modest Xeon, the benchmark opens the door to self‑hosted voice agents that run entirely on edge devices, private servers, or even a laptop. This aligns perfectly with recent work on lightweight agentic harnesses — CUGA’s two dozen working examples demonstrate how to stitch together perception, reasoning, and action modules without leaning on massive foundation models [1]. If you can generate speech locally at 0.14 RTF, you can embed a TTS pipeline inside those agentic loops without incurring latency penalties or ongoing API costs.

Second, the result dovetails with the growing interest in reusable workflows for long‑running local LLMs. A common pain point when chaining language model calls is the bottleneck introduced by external services — whether for retrieval, tool use, or now, speech output. By moving TTS onto the same CPU that hosts your LLM, you eliminate network round‑trips and keep the entire pipeline within a single memory space. The reusable workflows described in the LocalLLaMA thread show how to decompose lengthy tasks into bite‑sized chunks that fit modest context windows; adding a fast, local TTS step means those chunks can now produce audible feedback without breaking the flow [7].

Third, the benchmark hints at a broader trend of hardware repurposing. Many hobbyists report their servers idling 99 % of the time, waiting for the next prompt [14]. A CPU‑only TTS model that runs at 0.14 RTF turns that idle cycles into useful work — generating narration for documentation, creating audio alerts, or voicing agent responses — without needing to power on a GPU. For those experimenting with pooled, round‑robin hardware among friends, the ability to offload TTS to any idle CPU node makes the scheme even more attractive: each participant’s machine can contribute voice synthesis while the heavier LLM inference stays on the strongest GPU [9].

Finally, the result serves as a reminder that an LLM is not always the right tool. In multimodal agentic systems, the language model handles reasoning and planning, while dedicated, lightweight models — like the TTS networks here — manage sensory output. The benchmark shows that these specialist models can be both efficient and effective when run locally, reducing the temptation to push every modality through a massive, costly transformer.

Looking ahead, the numbers invite experimentation. Developers could swap Inflect‑Nano‑v1 into a voice‑enabled CUGA agent to achieve sub‑real‑time response latency, or pair Supertonic‑3’s 5‑step output with a local LLM for high‑fidelity narration in audiobooks or accessibility tools. The MoE caching strategies discussed elsewhere — where experts are split across CPU and GPU to maximize throughput — could also be applied to TTS, treating the diffusion steps as “experts” that are scheduled based on available compute [18].

In short, this CPU‑only TTS benchmark is more than a academic curiosity; it is a tangible step toward democratizing voice capabilities, shrinking the reliance on cloud providers, and enabling truly private, low‑latency agentic applications. For anyone building the next generation of local AI systems, the data point is clear: you can get fast, decent‑quality speech today on the CPU you already have.

Sources

1. Build real agentic apps using CUGA: two dozen working examples on a lightweight harness
2. CPU-only TTS benchmark: Kokoro 82M vs Supertonic 3 vs Inflect-Nano-v1 (4.6M params), with UTMOS scoring on every sample
3. Reusable workflows for long running local llms
4. Pooled round robin hardware with friends?
5. My local server idling 99% of the time!
6. Multi Tier MoE Caching