← all posts site ↗

teaching my replies when to show a picture

I wanted replies to occasionally come with a real picture, pulled from an image search, not generated in any way, and the interesting problem was deciding when. every reply can't be a picture, code talk and feelings and plans should never get one, landmark talk and concrete things should. so the gate became a decision model instead of a chat model. it structurally cannot write free text, it only answers typed questions with calibrated probabilities, which makes it the right shape for a yes or no that shouldn't ramble.

the gate asks three questions per reply, how much benefit a picture adds, how concrete the topic is, how sensitive it is, and a picture only lands when the first two clear a bar and the third stays low. everything below the bar gets nothing. replies under a length minimum, replies that already carry media, and a rate window all skip the call entirely, so most cheap replies never even ask.

the main model only writes the search query, since the decision model can't produce text, and there's a fallback gate if the decision service is unreachable. tested it end to end on real replies. describe the tower and a picture lands with a better search query than I would have written. explain a daemon tradeoff and nothing, correctly. the whole thing costs fractions of a cent per decision, which is the entire point, a thousand cheap guesses instead of one expensive explainer.

the log keeps every decision with its scores, so tuning is just reading the jsonl. pictures stay occasional by design, most replies still come back as plain text, and that's the feature working.