Latest / The Feed & The Thread / The Feed & The Thread - October 21, 2026

Transcript

The Feed & The Thread - October 21, 2026 · The Feed & The Thread

• 0:00
  Digital twins are systematically overestimating user satisfaction
• 0:03
  scores. So the AI is just guessing what the average person
• 0:06
  wants. Which means we're missing the bad days that actually break
• 0:09
  the product.
• 0:14
  Welcome to The Feed and The Thread, brought to you by
• 0:16
  Chicago Camps. The Feed & The Thread is available online at
• 0:19
  feed and thread dot com to submit your feeds, or download
• 0:21
  our app for all the feeds and threads delivered right to your
• 0:24
  pocket.
• 0:32
  We built AI to simulate users, yet the most human moments still
• 0:35
  slip through the cracks. Lucas Plabst, Jeff Sauro, and Jim
• 0:39
  Lewis at MeasuringU found in "Digital Twins Systematically
• 0:42
  Overestimated Business Software SUS Scores" that digital twins
• 0:46
  systematically overestimated satisfaction scores, inflating
• 0:49
  product ratings by a half to full letter grade. This isn't a
• 0:53
  minor statistical blip; it's enough to change a product's
• 0:56
  standing on the Sauro - Lewis curved grading scale. Relying on
• 0:59
  these synthetic users risks masking real usability problems
• 1:03
  behind false confidence. Kai Wong at UX Design.cc argues in
• 1:06
  "Let AI have the average user: Why User Research should focus
• 1:10
  on bad days" that AI is efficient at capturing average
• 1:13
  responses, but it fails to identify the bad days that drive
• 1:16
  innovation. By delegating analysis of common behaviors to
• 1:20
  AI, human researchers can focus on the rare, high-impact moments
• 1:23
  that algorithms miss. That shift ensures design solutions address
• 1:27
  the nuanced, qualitative insights that actually
• 1:30
  differentiate a product. Luke Wroblewski on LukeW's blog
• 1:33
  details in "Ask LukeW: Instant Content Indexing" a new
• 1:36
  automated pipeline that ingests audio, video, and PDFs every
• 1:40
  hour. The system uses specialized AI models to
• 1:42
  transcribe speech and describe visual elements, keeping the
• 1:46
  knowledge base current. This technical upgrade allows the AI
• 1:49
  to provide accurate answers by processing nearly twenty
• 1:52
  thousand Twitter posts in real- time via webhooks. Also out
• 1:56
  today: Smashing Magazine on playful CSS functions, UX Planet
• 1:59
  on model comparisons, and Jeffrey Zeldman on Figma
• 2:02
  shortcuts.
• 2:10
  The tension in today's discussions is clear. We're
• 2:13
  using AI to automate the mechanical parts of design, yet
• 2:16
  the human context and the cost of entry are the barriers. Over
• 2:19
  on r/UXDesign, a developer- turned-designer is asking how
• 2:22
  teams are handing off UI feedback now that devs use AI
• 2:25
  agents. The core problem is that these tools are fast at fixing
• 2:28
  code, but they struggle to map vague human feedback to the
• 2:32
  right component. It's a gap between intent and execution
• 2:35
  that simple prompts don't bridge. Also in r/UXDesign, one
• 2:38
  practitioner is pushing for a mindfulness-based approach to
• 2:41
  the industry's current climate. They argue that combining
• 2:44
  spirituality with product design can help us rediscover the joy
• 2:47
  of creation. This is a direct pushback against burnout,
• 2:50
  suggesting that how we work is as important as what we build.
• 2:54
  In the same subreddit, a designer with thirteen years of
• 2:57
  experience is asking how to catch up on AI workflows without
• 3:00
  the subscription fees. This highlights a financial barrier
• 3:03
  for professionals re-entering the workforce. The rapid
• 3:06
  evolution of tools creates a paywall that excludes those who
• 3:09
  can't afford the latest stack. Over on r/UXResearch, a design
• 3:13
  undergraduate is weighing whether to minor in psychology
• 3:16
  or cognitive science to break into research. They're planning
• 3:19
  a gap year to build a heavy portfolio. The question is
• 3:22
  whether academic rigor or practical project experience
• 3:25
  matters more when you're trying to prove you can do the job.
• 3:28
  Finally, in r/UXResearch, a candidate is sharing their
• 3:31
  experience with Google's interview panels. They note the
• 3:34
  conversations flowed well, but they're asking about the
• 3:37
  difficulty level and how role matching works if a specific
• 3:40
  manager declines them. It's a reminder that the process is
• 3:43
  often as opaque as the job itself. We're automating the
• 3:46
  code, but we're still paying a premium for the human judgment
• 3:49
  that makes it work.
• 3:58
  The Feed and The Thread is brought to you by Chicago Camps.
• 4:01
  For a weekly and monthly exploration of the patterns and
• 4:04
  signals across UX, you can also check out Patterns and Signals
• 4:07
  at the website patterns and signals dot com. Chicago Camps
• 4:10
  will have more events in 2027. You can submit to be a speaker
• 4:13
  at a future event at Chicago Camps dot org.
• 4:22
  There's a specific gap in the MeasuringU study that really
• 4:25
  stuck with me. They found that digital twins, or synthetic
• 4:28
  users, systematically overestimated satisfaction
• 4:30
  scores by a half to full letter grade. That half-letter grade
• 4:33
  is the killer detail. It isn't a minor statistical blip. On the
• 4:36
  Sauro and Lewis curved grading scale, that shift is enough to
• 4:39
  move a product from a passing grade to an excellent one.
• 4:41
  Exactly. And that creates a false sense of confidence. We're
• 4:44
  building these AI simulations to handle the volume of average
• 4:47
  user behavior, which is efficient. But if the synthetic
• 4:50
  data consistently skews positive, we're masking real
• 4:52
  usability problems behind a polished report. I see the
• 4:55
  risk, but I'm looking at it differently. If we trust the
• 4:57
  average, we stop noticing the outliers. Kai Wong makes the
• 5:00
  point that AI is great at capturing the norm, but it fails
• 5:03
  at identifying the bad days. Those rare, high-impact moments
• 5:06
  are where innovation actually comes from. So the systemic
• 5:09
  issue is that we might be delegating the wrong kind of
• 5:11
  insight to the wrong tool. We're using synthetic data to validate
• 5:14
  the status quo, rather than using human research to disrupt
• 5:17
  it. That's the trap. We're letting the algorithm tell us
• 5:20
  what's working, while the humans are busy doing the work that
• 5:23
  actually differentiates the product. The community is seeing
• 5:26
  this in handoffs too, where vague human feedback gets lost
• 5:28
  in translation to code. Freeing up the researchers is the goal.
• 5:31
  We ask humans "what went wrong?" instead of asking AI "is this
• 5:34
  good enough?" The standard for quality should be the ability to
• 5:37
  find the edge case. We need to build systems that prioritize
• 5:40
  the nuanced, qualitative insights that algorithms miss,
• 5:43
  because that's where the actual product value lives.
• 5:47
  That's The Feed and The Thread for today. We'll catch you next
• 5:50
  time!