saw a video of google photos' new wardrobe feature and it does the thing i've been hand-building in my own app. it drops every piece of clothing out of your photos into a clean category grid, even a try-on that dresses an avatar. the catch is the avatar is some generic ai-smooth guy who looks nothing like the owner. google handles the segmentation for free but its try-on is the weak link, and mine keeps my actual face.
spent the day building the matching pipeline. a vision model reads each photo and returns bounding boxes per garment, then you crop and you have a tile. fun discovery: the model reports coordinates in a weird internal 1000 by 1000 grid no matter the resolution, so i had to calibrate it with a synthetic test image and scale to real pixels. rembg was pure person-segmentation, useless on pants, so i went with tight crops on dark tiles. a few hundred clean tiles later the visual layer worked.
the dedup pass was the real rabbit hole. one pair of green cargo pants kept showing up as eight separate items because lighting shifted the color from olive to khaki to brown. i tried clip embeddings plus an llm confirmation loop and overcorrected both ways: a chain bug merged like ninety different pants into one tile, then i swung back and split every black jeans sighting into its own. the fix was embarrassingly simple — assume a small wardrobe. same label means the same garment, different labels merge only when the model confirms they look alike, and anything over a year old gets dropped.
then the ai thumbnails, an image model rewriting each worn crop into a clean product shot. one model refuses to call a watch a garment, so the prompt had to list glasses and bags explicitly. finished off with recommender fails: a cardigan for hot and humid weather, shoe thumbnails showing a full-body photo you couldn't see the shoe in. both fixed with heat-aware filtering and real garment images. a lot of dumb failures, but the closet finally looks like a closet.