← all posts site ↗

merging the exporter into the save extension

merged the old social exporter into the save extension so pulls go straight into the bookmark vault, and watched the whole thing fail its way into something good. the old exporter walked your logged-in sessions and dumped a csv; the merge kept the walking and dropped the csv, posting each item straight to the vault with server-side dedupe keys per platform so re-pulls only add genuinely new posts. the first attempt fetched platform apis from the extension worker and every platform fought back. firefox blocks cross-origin credential fetches from extension workers with cors, and the x api needed a session token the worker couldn't see. the fix was to stop pretending the extension is the client: inject a content script into the site's own page and scrape from its origin, where your real login cookies authorize everything naturally.

each platform had its own trap after that. x merged bookmarks into history, so the scraper stopped caring about the page url and calls the graphql api the site itself uses, reading the token straight off the page. instagram retired the private saved-posts api entirely, so the instagram pull became a real dom scraper that scrolls the saved page, collects every post link, and stops when the page stops growing. pinterest isn't even chronological, which broke the naive stop logic, so the stop rules became two: five consecutive already-saved items stops chronological feeds, and a global ratio rule stops any feed when new items drop below a fifth of everything pulled once you're past the grace count, with the threshold exposed as a control in the popup.

the last bug was the dumbest and the most instructive. the pull imported everything but every single post got a not-authorized response because the pull read stored credentials while the settings page had tested with the on-screen ones, and extension reloads had wiped storage. now the pull saves the screen first, test connection persist on success, and auth failures abort after a few strikes instead of burning five hundred failures. also learned the extension lives in a private repo now, so this entire fiasco is version controlled.