← all posts site ↗

bulk importing bookmarks for ai

had an idea to feed all my saved and bookmarked content across platforms into an AI agent so it could learn my interests and preferences. twitter bookmarks have game dev and general development stuff, reddit saved posts lean toward gaming and tech, pinterest is vibes and 3D visuals, instagram is travel and food. each platform skews differently and together they'd paint a pretty complete picture of what I actually care about. the question was whether I could pull all of it programmatically using session tokens and cookies instead of paying for some export service.

the feasibility varies wildly by platform. reddit is the easiest — hit the saved.json endpoint with a session cookie and you get clean JSON with titles, selftext, URLs, and subreddit info. no artificial limits, just paginate through. twitter uses an internal graphql endpoint that needs an auth token and CSRF cookie, and there's a hard 1000 bookmark limit on the API. instagram is interesting because I already had an export of over 4700 saved posts from a previous scraping effort — each entry just has a URL and timestamp. the bottleneck there is time: visiting each URL to extract actual content would take hours of browser automation. pinterest is the hardest by far because their internal API uses a dynamically generated parameter that's obfuscated client-side. you'd either need to reverse-engineer it or just use browser automation to scroll through saved pins.

the dream pipeline would take tokens from each platform, pull all saved/bookmarked URLs, run each through a content extractor like jina reader to get the actual text, then consolidate everything into a structured format the agent can query. reddit feeds in first because the auth is trivial, then twitter with proper OAuth setup, instagram already has the URL list ready to go, and pinterest last because it requires the most manual work. the real insight is that this isn't just about building a scraper — it's about creating a personal knowledge base that reflects what you actually chose to save rather than what the algorithm feeds you.