the ask was simple, a benchmarking system for speech and language models on the local machine, no frontend, just a way to name any model and have it downloaded, dropped in, and measured for latency, speed, accuracy, samples and tokens per second. the registry should remember what you measured and let you compare.
the native c routes failed first, which is its own kind of education. the parakeet build compiled fine but never produced the shared library the wrapper expected, and its submodule setup did not survive a fresh recursive clone. the transcribe and audio ports had a repo identity problem, the wrong owner name existed for both, so builds pointed at the wrong thing and only error logs came back. worth remembering that a c build with a strong memory cap still fails on missing artifacts, not on memory, so the checks should look for the artifact, not the exit code.
what landed was a small python cli with no real dependencies, a registry, a run command per engine, and a per-model last-results table. then real numbers on real hardware with real downloads. whisper tiny transcribed eleven actual voice notes from the ring archive, a hundred and twenty five seconds of audio in thirty seconds of wall time, twenty five percent word error, fine for dialogue. piper spoke seventeen seconds of audio in about a second, realtime factor of point oh six, and a whisper round-trip scored its accuracy at over ninety three percent. the kokoro voice ran slower than realtime on this cpu without the vector instruction set, which is a compatibility note, not a quality one. on the llm side a tiny qwen in q4 answered a twenty question quiz at ninety five percent, two hundred eleven tokens per second prompt processing, fifty seven tokens per second generation, ninety two milliseconds to first token, six hundred megabytes peak. one box, one command, a table at the end. that is the whole bench.