the 27b qwen with the dense hybrid attention and the twelve gigabyte quant of it: it fits entirely on the 16 gig card, but the memory bus caps the speed. the card's 128-bit bus streams about three hundred twenty gigabytes per second, divide by the model size and the ceiling is roughly twenty seven tokens a second, realistically more like twenty. thinking mode is the trap: it's on by default at high effort, so a reply that thinks for two thousand tokens takes a couple of minutes at that speed. drop the reasoning effort to medium or off and everything feels orders faster, with quality loss mostly on hard math and coding.
the quant is the budget end of usable. the two options that actually fit are the iq3 variant at the smaller size and the q3 variant slightly bigger, the next step up does not fit with any context worth having. sixteen to thirty two thousand tokens of context fits, the full native 256k window is unreachable because the kv cache alone would eat the card.
the ud quants need a recent llama.cpp, so if a runtime throws unknown quantization, update it first. and since estimates are just estimates, the plan is to actually run the benchmark on the machine and read real numbers.