Latest / Big Iron Radio / Homelab Brief - Auto-Tagging Your Photos with CLIP and pgvector

Transcript

Homelab Brief - Auto-Tagging Your Photos with CLIP and pgvector · Big Iron Radio

• 0:02
  Welcome to the Homelab Brief from Big Iron.
• 0:06
  Today, auto-tagging a photo archive
• 0:09
  with Clip built to outlive whatever app you use today.
• 0:13
  Quick note, this is an AI voice drawn from a Big Iron guide.
• 0:17
  Links below.
• 0:19
  The premise is a warning.
• 0:21
  App-locked machine learning tags the ones inside image,
• 0:24
  photo prism, or diggy cam die the day you migrate away
• 0:27
  from that app.
• 0:29
  A standalone pipeline, when you own, outlives all of them.
• 0:32
  So build the pipeline, not the lock-in.
• 0:35
  The engine.
• 0:37
  Run open Clip fast and a good default or SIG LIP,
• 0:41
  which gives better zero-shot classification accuracy.
• 0:45
  You run it once per image batched to turn every photo
• 0:48
  into an embedding and a set of tags.
• 0:50
  Now, two very different outputs,
• 0:52
  and this is the key architectural idea.
• 0:55
  Write the embeddings to Postgres with PG vector, HNSW indexed,
• 1:00
  for fast semantic search,
• 1:02
  but treat that as disposable infrastructure you can always rebuild.
• 1:06
  And write the human-readable keyword tags to XMP sidecar files,
• 1:10
  using EXIF tool, for permanents portable to any tool, forever.
• 1:15
  Disposable index, permanent tags.
• 1:18
  The highest leverage step is also the most manual.
• 1:21
  Zero-shot tagging needs a curated vocabulary.
• 1:25
  Give it a real list of the categories you care about,
• 1:28
  or you get useless captions like a photo of a thing.
• 1:31
  It's one-time work, and it's the difference between a searchable archive and noise.
• 1:35
  On time, for a 200,000-image archive, CPU only runs 1 to 5 hours, depending on the model.
• 1:45
  A modest GPU does it in 7 to 40 minutes.
• 1:49
  Either way, it's a one-time pass.
• 1:51
  The full pipeline's in the show notes.
• 1:53
  That's your brief.