← all posts site ↗

your gpu is as fast as physics says

a tweet ranked the fastest local models and most of the list did not apply to this machine. the card runs vulkan, so sparse models get zero speedup from their sparse design, file size alone decides tokens per second. the rule of thumb on this box is divide 250 by the model size in gigabytes.

the benchmark history already covered most of the list. the seven billion class ceiling around 70 tokens per second, the current daily driver at 63, a twenty billion at 51, and a dense 27 billion at 19 to 22. one tweet pick called agentic turns out to be a 35 billion model that barely fits in vram with no context room left. another called fastest measured at 87 percent of what the smaller models already do, its sparse architecture buys nothing on this backend.

then the one unmeasured entry promised 45 tokens per second prefill and 20 decode on a first screen. both turned out to be startup artifacts. warm decode is 22.2 tokens per second, which multiplied by the file size gives 76 percent of the memory bus on a dense 27 billion. that number does not move with quant tricks or context settings. the model pages also mislabel some of the files as 2 billion, which is metadata lying, the actual weights are the full size.

the running file already is the best fit for this card. the only real takeaway from the whole list is that a 12 billion model at 35 tokens per second is the one that feels instant, and everything bigger trades raw speed for more thought per token.