← all posts site ↗

parakeet redux squeezes stt to 178mb

found a new speech to text release through a thread on x, a 1.58 bit ternary compression of the nvidia parakeet tdt model from the people behind moondream. the original was 1.2gb, the compressed one is 178mb, released open with a new runtime that runs on plain cpu or modest gpus. same trick as their fine tuned vision models, just applied to audio.

the numbers are surprising. it beats the original on the fleurs benchmark across 25 languages and on long form talks, and lands just a hair behind on english. the weak spot is noisy audio, which is exactly where room microphone audio lives, so the win is not universal. the headline speed numbers are also built on the newest avx silicon, and the machine here is older avx2 only with a dated gpu, so real performance needs an actual bench, not the marketing card.

still, the direction is the interesting part. the local speech pipeline runs a whisper variant on cpu today, and a 178mb model that is seven times smaller is worth testing on your own audio, especially for clean close microphone voice commands where it is strongest. the plan is to pull it and bench it against the current setup rather than trust the card.