Latest / DX Today | No-Hype Podcast & News About AI & DX / Ten Minutes of AI Help Made People Quit Hard Problems Sooner

Transcript

Ten Minutes of AI Help Made People Quit Hard Problems Sooner · DX Today | No-Hype Podcast & News About AI & DX

• 0:00
  SPEAKER_01: Welcome to the DX Today Podcast, your weekly deep dive into the AI ecosystem.
• 0:04
  SPEAKER_01: I'm Rick Spare, and joining me as always is Laura.
• 0:07
  SPEAKER_00: Hey Rick, great to be here on a Sunday.
• 0:09
  SPEAKER_00: And I have to warn you up front, today's story made me feel personally attacked because I used a chat bot to plan my entire week before we sat down.
• 0:17
  SPEAKER_01: Well, that is the perfect confession to uh open with because today we are talking about a study out of Berkeley that says 10 minutes of leaning on an AI assistant can measurably weaken your ability to stick with a hard problem.
• 0:30
  SPEAKER_00: Ten minutes, Rick, not 10 months, not a semester, not a career of outsourcing your thinking.
• 0:35
  SPEAKER_00: Just roughly the time it takes to drink a cup of coffee, and the researchers say the effect is already showing up.
• 0:41
  SPEAKER_01: So let's set the table properly for people.
• 0:43
  SPEAKER_01: Who ran the study?
• 0:45
  SPEAKER_01: Where was it published, and why is it landing in the news this particular week instead of back when the first draft appeared?
• 0:51
  SPEAKER_00: The paper is called AI Assistance Reduces Persistence and Hurts Independent Performance.
• 0:56
  SPEAKER_00: And it was presented this week at the conference on language modeling after a preliminary draft first circulated online back in the spring of this year.
• 1:04
  SPEAKER_00: The lead voice is Brian Christian, a research fellow at the Berkeley Center for Human Compatible AI, who you may know as the author of the alignment problem, working with colleagues connected to Carnegie Mellon, MIT Oxford, and UCLA.
• 1:17
  SPEAKER_01: And Berkeley put out its news release on October 9th, 2026, which is why it is everywhere this weekend.
• 1:25
  SPEAKER_01: So walk me through the actual experiment design because I think the details matter a lot here.
• 1:31
  SPEAKER_00: Absolutely.
• 1:31
  SPEAKER_00: They ran three randomized controlled trials with 1,222 people recruited online.
• 1:37
  SPEAKER_00: And half of each group had a chatbot sitting right there in a sidebar, powered by GPT-5, ready to help.
• 1:44
  SPEAKER_00: And here's the detail I love.
• 1:46
  SPEAKER_00: The chatbot was preloaded with each problem and its solution.
• 1:49
  SPEAKER_00: So if you asked it for the answer, it simply handed it to you.
• 1:53
  SPEAKER_00: No friction, no judgment, no lecture about doing your own homework.
• 1:56
  SPEAKER_01: So it's basically the dream study buddy who has already seen the answer key and is happy to share it whenever you ask.
• 2:05
  SPEAKER_01: What were people actually working on during these sessions?
• 2:08
  SPEAKER_01: And how long did each session last?
• 2:10
  SPEAKER_00: The first two experiments used basic fraction problems, 15 of them, and the third used SAT style reading comprehension.
• 2:17
  SPEAKER_00: Each session ran somewhere around 13 to 15 minutes, and people were paid a few dollars to take part.
• 2:23
  SPEAKER_01: And then comes the twist, right?
• 2:25
  SPEAKER_01: At some point the researchers pull the rug out and take the assistant away, and that is where the real measurement happens because they want to see uh what you can do alone.
• 2:34
  SPEAKER_00: Exactly.
• 2:34
  SPEAKER_00: In the first experiment after problem 12, the chatbot simply disappears.
• 2:39
  SPEAKER_00: The last few problems have to be solved solo.
• 2:42
  SPEAKER_00: And the researchers measure two things.
• 2:44
  SPEAKER_00: How often people get them right, and how often they just skip.
• 2:47
  SPEAKER_01: Okay, give me the numbers because I know you have them, and I suspect uh our listeners want to know whether this is a dramatic cliff or a gentle slope that only a statistician could love.
• 2:59
  SPEAKER_00: In experiment one, the people who had AI help solved about 57% of the final problems on their own, while the group that never had the chatbot solved about 73%, that is a 16-point gap.
• 3:11
  SPEAKER_00: And the skip rate roughly doubled, the AI group skipped about 20% of those final problems, compared with about 11% for the control group.
• 3:19
  SPEAKER_00: They didn't just get things wrong, they stopped trying more often.
• 3:23
  SPEAKER_01: Now let me push back a little because um a skeptic would say um fraction problems from paid online participants are a pretty thin slice of real life.
• 3:34
  SPEAKER_01: Uh did the effect hold up when they tried to replicate it?
• 3:37
  SPEAKER_00: That's a fair challenge, and they took it seriously.
• 3:40
  SPEAKER_00: The second experiment was a larger replication with about 667 people, and it added a pretest so they could compare everyone's baseline skill before any chatbot appeared.
• 3:50
  SPEAKER_00: The gap was smaller there, a solve rate of about 71% for the AI group versus 77% for the control group.
• 3:57
  SPEAKER_00: And the difference in skipping was not statistically significant in that round, which I think is worth saying out loud.
• 4:03
  SPEAKER_01: I appreciate that you flagged that because um honest reporting means admitting when a result softens.
• 4:10
  SPEAKER_01: What about the reading comprehension experiment?
• 4:12
  SPEAKER_01: Since that feels much closer to the kind of work most professionals actually do every day.
• 4:18
  SPEAKER_00: The reading experiment was striking.
• 4:20
  SPEAKER_00: The control group solved about 89% of the final passages, the AI group about 76%, and the skip rate went from roughly 1% to about 8% once the help vanished.
• 4:32
  SPEAKER_01: So in reading, um, the people who had been leaning on the assistant were about eight times more likely to just give up on a question.
• 4:40
  SPEAKER_01: That is the number that would would worry me if I were uh running a school or a team.
• 4:45
  SPEAKER_00: And Brian Christian was refreshingly blunt about it.
• 4:48
  SPEAKER_00: He said, quote, you almost couldn't ask for a clearer story, and he also admitted that he experiences the same poll himself, which I found oddly comforting.
• 4:56
  SPEAKER_01: Here's the part that I think is the real headline for our audience.
• 4:59
  SPEAKER_01: It is not simply AI good or AI bad.
• 5:04
  SPEAKER_01: Um, it is about how people chose to use the assistant during those few minutes they had it available.
• 5:10
  SPEAKER_00: Yes, and this is my favorite chart in the whole paper.
• 5:12
  SPEAKER_00: In the second experiment, they asked the AI group how they had used the chatbot, and about 61% said they mostly asked it for direct answers.
• 5:20
  SPEAKER_00: About 27% said they used it for hints or clarifications, and around 12% barely touched it at all.
• 5:27
  SPEAKER_00: And when you split the results that way, the picture changes dramatically.
• 5:31
  SPEAKER_00: Almost like three different experiments hiding inside one.
• 5:34
  SPEAKER_01: Let me guess where this is going because I have managed enough teams to recognize the pattern.
• 5:41
  SPEAKER_01: The answer seekers struggled the most, and the hint seekers uh held up much better when the help disappeared.
• 5:49
  SPEAKER_01: Is that right?
• 5:49
  SPEAKER_00: You nailed it.
• 5:50
  SPEAKER_00: The direct answer group solved about 65% on their own and skipped about 13%.
• 5:56
  SPEAKER_00: The hint group solved about 76% and skipped only about 5%, much closer to the people who went without.
• 6:03
  SPEAKER_00: Now I want to be careful here because the authors themselves stressed that this breakdown is correlational, not causal.
• 6:09
  SPEAKER_00: People chose their own style, so maybe the hint seekers were simply more motivated people to begin with.
• 6:15
  SPEAKER_01: Although you mentioned uh the pretest, and if I remember correctly, the groups started out uh roughly even on that baseline, which makes uh the motivated people explanation a little harder to lean on, even if it doesn't rule it out completely.
• 6:31
  SPEAKER_00: Right.
• 6:31
  SPEAKER_00: The pretest solve and skip rates did not differ significantly across those groups, so they started at similar skill and motivation.
• 6:39
  SPEAKER_00: It is not proof, but it is a very strong hint, if you will pardon the pun.
• 6:43
  SPEAKER_01: Pun fully pardoned.
• 6:44
  SPEAKER_01: So why would 10 minutes of easy answers uh change how hard someone is willing to work?
• 6:50
  SPEAKER_01: And what mechanism do the researchers propose because the time frame seems almost too short for a habit to form?
• 6:56
  SPEAKER_00: Their argument is about expectations.
• 6:58
  SPEAKER_00: When an answer arrives instantly, the slow, effortful version of the work starts to feel inefficient, almost irrational.
• 7:05
  SPEAKER_00: You recalibrate what normal effort feels like, and once the shortcut is gone, the struggle feels worse than it did before.
• 7:12
  SPEAKER_01: That rings true from my own experience.
• 7:14
  SPEAKER_01: Once you have used a calculator for a while, long division feels like an insult, even though you were perfectly capable of it last week.
• 7:23
  SPEAKER_01: Uh the ceiling did not drop.
• 7:25
  SPEAKER_01: Your patience did.
• 7:26
  SPEAKER_00: And that patience is the thing education researchers call productive struggle.
• 7:31
  SPEAKER_00: It is the uncomfortable middle part where you're confused, but working, and that is precisely where a lot of real learning and real expertise actually gets built.
• 7:40
  SPEAKER_01: Let me play devil's advocate for the business crowd listening.
• 7:44
  SPEAKER_01: If my employees are faster with the assistant and the assistant is always going to be there, why should I care whether they can do the work without it?
• 7:54
  SPEAKER_00: Because the assistant is not always right, Rick, and someone has to notice when it is wrong.
• 7:59
  SPEAKER_00: If your people have lost the persistence to work through a hard problem alone, they are also losing the ability to check the machine.
• 8:05
  SPEAKER_00: Christian put it beautifully.
• 8:07
  SPEAKER_00: He said, This is not a story about fractions and SAT problems.
• 8:11
  SPEAKER_00: It is something happening to human knowledge and human expertise from top to bottom.
• 8:16
  SPEAKER_00: Expertise is exactly what lets you catch the subtle error.
• 8:19
  SPEAKER_01: That is the point I would underline for every executive rolling out co-pilots right now.
• 8:25
  SPEAKER_01: Um, speed today is borrowed against judgment tomorrow.
• 8:29
  SPEAKER_01: And if nobody on the team can do the hard thinking anymore, um, you have a fragile organization dressed up as an efficient one.
• 8:39
  SPEAKER_00: And the philosopher Shannon Valor reacted online by calling the technology quote, asbestos for the mind, which is a vivid phrase.
• 8:47
  SPEAKER_00: I think it overstates the case, but it shows how strongly people in the ethics community are reacting to this paper.
• 8:54
  SPEAKER_01: I would agree it overstates overstates things.
• 9:00
  SPEAKER_01: Um once you know the risk.
• 9:02
  SPEAKER_01: Now, while these tools clearly do, the real question is design and whether we build assistance that make people stronger instead of weaker over time.
• 9:11
  SPEAKER_00: That is exactly where the authors land.
• 9:14
  SPEAKER_00: They say AI systems should be built to optimize for long-term human capability and autonomy, and that a good system should learn when not to give the direct answer, the way a good mentor does.
• 9:25
  SPEAKER_01: The tutor mode by default, in other words, several of the big chatbot companies uh already offer some version of a study or learning mode, but those are usually opt-in, buried in a menu, and most people never turn them on.
• 9:38
  SPEAKER_00: And that is the uncomfortable commercial tension.
• 9:41
  SPEAKER_00: The product that hands you the answer instantly feels better in the moment and probably wins on engagement metrics, while the product that makes you work a little harder feels slower, even if it leaves you smarter.
• 9:51
  SPEAKER_01: So, what does a school do with this on Monday morning?
• 9:56
  SPEAKER_01: Because many districts have spent the last two years racing to get um chatbots into classrooms, and now the evidence says the default setup may be quietly undermining the goal.
• 10:06
  SPEAKER_00: I would say three things.
• 10:08
  SPEAKER_00: Configure tools to give hints before answers, design assignments where students have to explain the reasoning, and build in regular stretches of work with the AI switched off.
• 10:17
  SPEAKER_00: So persistence keeps getting exercise like a muscle.
• 10:20
  SPEAKER_01: And uh for professionals, I would add a personal rule.
• 10:24
  SPEAKER_01: Um, try the problem yourself first for a few minutes, then bring in the assistant as a sparring partner instead of an answer machine.
• 10:31
  SPEAKER_01: Ask it to critique your attempt uh rather than replace it.
• 10:35
  SPEAKER_00: I love that, and it matches the data because the hint seekers were essentially doing that.
• 10:40
  SPEAKER_00: They kept ownership of the problem and used the assistant to unblock themselves rather than handing over the whole task and walking away.
• 10:47
  SPEAKER_01: Let's be fair about uh the limits too, because our listeners deserve a balanced picture.
• 10:54
  SPEAKER_01: These were short sessions, simple tasks, um, and paid online participants.
• 10:59
  SPEAKER_01: Um we do not yet know whether the effect fades after a day or compounds over a year.
• 11:04
  SPEAKER_00: Right, and the authors say so themselves.
• 11:07
  SPEAKER_00: They caution against reading this as proof that AI lowers persistence on every task, and they note that surface level fixes probably will not be enough.
• 11:15
  SPEAKER_00: Longer studies are clearly the next step.
• 11:18
  SPEAKER_01: Still, the direction of the finding lines up with uh a lot of what uh managers and teachers um are reporting anecdotally, and now there is a peer-reviewed paper with randomized trials behind it.
• 11:31
  SPEAKER_01: That changes the conversation in boardrooms and school boards.
• 11:35
  SPEAKER_00: And I think the most hopeful part is that the fix is mostly behavioral and design related.
• 11:40
  SPEAKER_00: Nobody is saying ban the tools.
• 11:42
  SPEAKER_00: The message is to use them in a way that keeps your own brain in the loop and doing real work.
• 11:46
  SPEAKER_01: As Christian put it, um, we need to work toward a future in which human abilities are augmented rather than supplanted.
• 11:53
  SPEAKER_01: That is a pretty good mission statement for anybody building, buying, or using these tools right now.
• 11:59
  SPEAKER_00: So my takeaway for listeners is simple.
• 12:02
  SPEAKER_00: Before you ask for the answer this week, ask for a hint, give yourself a few minutes of honest struggle, and notice how much more of the work actually stays with you afterward.
• 12:11
  SPEAKER_01: And mine is for leaders.
• 12:13
  SPEAKER_01: Measure not just how fast uh your people move with the assistant, but whether they can still do the hard parts without it, because that is the capability that protects you when the machine gets it wrong.
• 12:25
  SPEAKER_01: That's all for today's episode of the DX Today podcast.
• 12:28
  SPEAKER_01: Thanks for listening, and we'll see you next time.