← all posts site ↗

the reddit extractor that saved only the title

saved a text post from reddit into the note vault, a home assistant thread, and the explainer it generated was useless, super high level, like it never saw the post. turned out the save was fine and the extractor was dead. reddit bounces logged out scrapers to a login wall now, the old url path just redirects to login, and the json api returns forbidden from my server. so the whole pipeline fell back to a generic shell title, that stupid reddit is the heart of the internet tagline, with no body and no comments stored. the explainer had nothing but that title to chew on.

the fix was a new primary source, the arctic shift archive api. zero auth, works from the server, and it pulls the full post and the whole comment section in two calls. share links and the shortened link format now resolve to the canonical post url, which also fixed dedupe for them. an item now stores the body plus a community comments block, every comment with its score, op replies marked, automoderator and deleted comments dropped, up to sixteen thousand characters.

also raised the window the explainer gets to read, from sixteen hundred to twelve thousand characters for reddit posts, and changed the prompt so thread summaries have to state what the poster actually asked and what the community concretely suggested. now the vault has all the suggestions right there in the searchable text and the detail view instead of a paragraph about nothing.

verified against the two posts that were broken. the home assistant thread came back at forty five hundred characters, body plus nineteen comments, and the explainer finally names the actual suggestions instead of describing reddit. image covers now come from the same api, and galleries get every slide.