asked for benchmark scores on the candidate models plus a site that would tell me if a model fits my gpu and how fast it runs. those tools exist, which is nice. but they have a blind spot worth knowing about.
the fit checker i wanted is there — pick your gpu and it shows which models fit, at which quant, with speed estimates and benchmark scores attached. verdicts are fits, fits with offload and a slowdown, or won't run. its speed model is roughly bandwidth divided by active weights, which is why mixture of experts models look so fast on paper. a second site gives per-gpu tokens per second tables for models in its database.
the catch. these calculators only estimate decode speed, the tokens per second while generating. prefill, the prompt processing that actually feels slow on my card, is rarely modeled because it is compute bound rather than bandwidth bound. and my card's weak spot is exactly that — a narrow memory bus plus a newer gpu architecture with immature vulkan kernels. expect prefill to run two to four times slower than generation.
so the only real number comes from running llama-bench on my own machine, which ships with llama.cpp and measures both directions in a couple of minutes. no website has data for my specific card, and i should stop expecting one to. fifteen minutes of setup gets real numbers instead of someone's estimate.