Latest / Soft Ratios Research / It felt like knowing (on learning)

Transcript

It felt like knowing (on learning) · Soft Ratios Research

• 0:09
  Figure: This is Soft Ratios Radio.
• 0:12
  Ground: The voices are synthetic [pause] no act, just the thinking.
• 0:18
  Figure: Hoyd Breton, a designer at Soft Ratios Studio, teaching himself in the open.
• 0:23
  Ground: One subject at a time. [pause] Today, how learning actually works.
• 0:29
  Figure: Here's the thing almost everyone's sure of. You sit down with the material, read it through, read it again, and by the second pass it's smooth. No snags. You close the book and you think — I've got this.
• 0:43
  Ground: And that feeling's real. Say that first. The ease isn't imaginary. Something did happen in there.
• 0:50
  Figure: It did. The question is whether the thing that happened is the thing you think happened.
• 0:55
  Ground: You think it's learning.
• 0:57
  Figure: Learning, yes. What it usually is, though, is fluency. The page went down easy the second time because you just saw it the first time. That smoothness is recognition. It's not the same as pulling the thing out of your head a week later, cold, with nothing in front of you.
• 1:14
  Ground: There's a cousin of this people build a whole identity on. "I'm a visual learner." "I'm hands-on, I have to do it." That one feels true from the inside too.
• 1:25
  Figure: It feels true and it doesn't hold. The claim underneath it — the meshing idea — is that teaching works best when it matches your preferred mode. To prove that you'd need a clean crossover: visual people learn best taught visually, verbal people best taught verbally, each doing worse in the other's mode. Pashler and his colleagues laid out exactly that test in 2008, and they concluded there's no adequate evidence base for using learning-styles assessments in general educational practice.
• 1:56
  Ground: And more recently?
• 1:58
  Figure: Worse for the myth. Hattie and Bustamante pulled it apart in 2025 — the studies that actually test the matching, not just the ones that ask people their preference. A hundred and forty-three studies, four hundred and one effects. The effect size came out at zero point zero four. That's noise wearing a lab coat.
• 2:17
  Ground: So the preference is genuine — people do prefer pictures, or prefer doing — but the payoff isn't there.
• 2:24
  Figure: The preference is real, the prescription is false. And I want to hold onto why that lie's so comfortable, because it's the same trick running under everything else we're about to say. It flatters you. If you fail, it wasn't the effort — it was a mismatch. Somebody taught you wrong for your type.
• 2:41
  Ground: That's the tell, isn't it. Any story about learning that makes you feel better about how it's going — be suspicious. Because the real ones tend to feel worse.
• 2:53
  Figure: That's the spine of everything we're about to say. A lot of the techniques that build durable learning feel harder and slower while you're doing them, and fluency can be a misleading cue. Smooth performance on its own isn't evidence the learning will last. Which means the feeling — the exact instrument you're using to judge your own progress — is pointed the wrong way.
• 3:15
  Ground: Then let's put the sharpest version of it on the table. Where's the cleanest proof that easy fools you?
• 3:21
  Figure: Roediger and Karpicke, 2006. They give people science passages to learn. One group keeps rereading — they get through the material about fourteen times on average. The other group reads it far less, maybe three, four times, but instead of rereading they get tested — they have to recall it, blank page. [pause] Now you test everyone. If the final test is five minutes later, the rereaders win. Fresh, fluent, it's all sitting right there. So in the room, in the moment, more studying looks better. [pause] But wait a week. Now it flips, and it flips hard. The rereaders — fourteen passes — come back at forty percent. The ones who tested themselves, on a fraction of the exposure, come back at sixty-one. [pause] And here's the knife. The rereaders were more confident. They read it four times as much, it felt locked in, they were sure they'd remember — right at the point they were about to remember the least.
• 4:18
  Ground: So the confidence tracked the reading, not the knowing.
• 4:22
  Figure: The confidence tracked the fluency. It's a beautiful, brutal little result. The strategy that produced the feeling of mastery produced less of the thing.
• 4:32
  Ground: I want to understand the mechanism of the trick, not just that it happens. Why does rereading feel like knowing when it isn't?
• 4:40
  Figure: Koriat and Bjork have the cleanest account. They call it foresight bias. When you study, the answer's right there on the page in front of you. When you're tested, the answer's gone and you have to produce it. Those are two different conditions, and your judgment gets made in the first one.
• 4:56
  Ground: So I rate how well I know it while I'm looking at it.
• 5:00
  Figure: While it's present. And its being present makes the whole thing feel obvious, connected, easy to grab. Then at test the cue shows up alone, the answer's missing, and the obviousness evaporates — because the obviousness was borrowed from the answer being there. You were reading the room wrong because the room changes between study and test.
• 5:21
  Ground: That's almost unfair. The moment you're supposed to judge is the moment you have the most help.
• 5:27
  Figure: Which is why the fix, when we get to it, is so simple it's almost rude: judge yourself later, from the cue alone, when the help is gone.
• 5:36
  Ground: Before we get there — I want to press on the frame. You keep saying performance in the room versus learning that lasts. Are those really two different things, or is it just a slow test versus a fast one?
• 5:51
  Figure: They're genuinely two things, and Soderstrom and Bjork made the distinction the center of the field in 2015. Performance is what you can do right now, while you're training. Learning is the relatively permanent change — and you can only see it later, in delayed retention and in transfer. The trap is that performance is visible and learning isn't, so we manage the one we can see.
• 6:16
  Ground: Give me the physical version. I trust my hands more than my confidence.
• 6:21
  Figure: Then take practice schedules. Say you're drilling three variations of a movement. You can block them — all of variation one, then all of two, then all of three. Or you can randomize — one, three, two, one, two, three, no pattern.
• 6:38
  Ground: Blocked feels incredible. I know exactly this. You do the same thing twenty times and by rep fifteen it's silk. You feel yourself getting good in real time. Randomized feels like being thrown — every rep you're resetting, you never get into a groove, you leave the session feeling clumsy and behind. [pause] And the retention tests say the clumsy sessions are the ones that stick. The random practice, the one that felt worse the whole way through, transfers better and holds longer. [pause] There's a version of this in an old typing study Bjork points to — postal workers, Baddeley and Longman. The ones on the spaced schedule learned it more thoroughly, and they were the least happy about it. They felt like they were falling behind the massed group. They weren't. They were falling ahead, and it felt like falling behind. [pause] So the body reports the same lie the mind does. It feels like smooth means I'm learning. But smooth performance on its own isn't evidence the learning will last.
• 7:50
  Figure: The Bjorks' account of why difficulty helps is worth stating precisely, because it's not mysticism — harder isn't magic. They split memory into two strengths. Storage strength: how deeply something's built in. That never decreases. And retrieval strength: how reachable it is right now. That fluctuates. [pause] Rereading pumps retrieval strength — it's right on the surface, easy to grab, feels great — but difficult retrieval is the valuable part, because that's what builds storage strength. Struggling to pull something back from a distance, when it's gone a little cold, is the thing that builds storage. The difficulty is the mechanism. That's why rereading alone is a poor basis for durable learning.
• 8:36
  Ground: So the effort isn't a tax on learning. The effort is the learning.
• 8:41
  Figure: When it's the right kind of effort, yes. That's the whole banner over this field — desirable difficulties. Conditions that slow you down now and pay you later.
• 8:52
  Ground: Here's where I want to push, though, because "harder is better" is one slogan away from being its own comfortable lie. If any difficulty helped, you could just make everything maddening and call it learning. That can't be right.
• 9:09
  Figure: It isn't, and the field is honest about the edges. Take one large classroom result in the desirable-difficulty file — interleaving. Rohrer, Dedrick and Burgess, 2014. Seventh graders, math, nine weeks. One group blocks the problem types, one group mixes them. Unannounced test two weeks after. Blocked group: thirty-eight percent. Interleaved group: seventy-two.
• 9:35
  Ground: That's not a nudge. That's nearly double.
• 9:38
  Figure: A very large effect. But here's the boundary — interleaving helps when the categories are confusable, when the hard part is knowing which kind of problem you're even looking at. Because mixing forces you to choose the method, not just run one you were already handed. Mixing in unrelated stuff isn't automatically better. The difficulty has to be aimed at the categories you actually need to tell apart.
• 10:03
  Ground: So it's not difficulty in general. It's difficulty that maps onto the real confusion.
• 10:11
  Figure: Which is a much more disciplined claim than the slogan. And the preference gap persists here exactly like the postal workers — later studies keep finding students prefer blocking, rate the mixed practice as more effortful and less useful, even in the studies where it's plainly teaching them more. In one later college study, only eighteen percent of the students used interleaved practice before the intervention.
• 10:35
  Ground: They can feel it working against them and read that as it not working.
• 10:40
  Figure: Same instrument, pointed the same wrong way. Let me give you the version that unsettles me most, though, because it's not about lazy studying. [pause] Karpicke and Blunt, 2011. They pit retrieval practice against concept mapping — building those elaborate diagrams, boxes and arrows, everything connected. That is not passive. That's deep, effortful engagement.
• 11:02
  Ground: And I'd bet on the mapping. It sounds like real thinking.
• 11:05
  Figure: A week later, the recall test: the mapping group scores forty-five percent, the retrieval group sixty-seven. A gap of about one and a half standard deviations. And the mappers had done the work well — their diagrams captured around eighty percent of the ideas. The engagement was real. It just wasn't the engagement that builds retrieval.
• 11:26
  Ground: And let me guess the confidence.
• 11:28
  Figure: The students predicted mapping would work as well or better. The one that lost, they expected to win. So it's not that effort was missing. It's that the felt sense of depth doesn't track what actually lasts, even when you're working hard and working thoughtfully.
• 11:43
  Ground: Let me sit in the middle of this and say the question out loud, because it's gotten stranger than where we started. We began with: does the ease of rereading fool me. Now it's bigger. If a real, deep, effortful strategy can also fool me about how much it's teaching me — then what exactly am I allowed to trust about my own sense of how it's going?
• 12:07
  Figure: And the honest first answer is: less than you'd like. Kruger and Dunning put a floor under how bad it gets. Nineteen ninety-nine — humor, grammar, logic. The people in the bottom quarter, whose scores put them around the twelfth percentile, pegged themselves at the sixty-second.
• 12:25
  Ground: Fifty points of daylight between where they were and where they thought they were.
• 12:30
  Figure: On the humor test the bottom group figured they'd gotten about sixty-nine percent of the items. They'd gotten twenty-three. And the argument is that the skill you'd need to do the task well is the same skill you'd need to know you did it badly — so the least able are also the least able to see it.
• 12:47
  Ground: Okay. But now I have to push the other direction, because you've been building a very clean story and I don't fully trust clean stories about messy things. You keep saying the feeling runs overconfident. Is it always overconfident?
• 13:02
  Figure: No. And this is where I have to give ground. Koriat, Sheffer and Ma'ayan, 2002 — they had people run the same material through repeated study-test cycles. First cycle, calibration's decent, slight overconfidence, the story you'd expect. But from the second cycle on it flips into underconfidence. People start predicting they'll remember less than they actually do, and it stays that way cycle after cycle.
• 13:28
  Ground: So once you've practiced something a few times, you start selling yourself short.
• 13:33
  Figure: Consistently. It held up across several of their manipulations — feedback or none, self-paced or fixed, different incentives. So the feeling isn't just "too high." In that repeated study-test setup, the estimates shifted toward underconfidence from the second cycle on. But later work found boundary conditions, so the direction of the bias isn't universal.
• 13:55
  Ground: That actually rescues something for me. Because if the feeling were only ever inflated, you'd say — fine, throw it away, distrust it wholesale. But it's not a simple bias you can just subtract off. It's genuinely uncalibrated. Which is a harder problem and, I think, a truer one.
• 14:17
  Figure: It's harder. You can't apply a correction if you don't know which way you're wrong today.
• 14:22
  Ground: And I want to push once more, on the word illusion itself. You've called fluency an illusion of competence. But when they actually watched what students do and how they end up — it's not so tidy, is it.
• 14:37
  Figure: It isn't, and I won't pretend it is. The prevalence part holds — Karpicke, Butler and Roediger surveyed students in 2009 and most of them reread; very few tested themselves. So the weak strategy is the popular one, that's solid. But Hartwig and Dunlosky, 2012 — they surveyed study habits against actual grade point average, and this is the uncomfortable finding — both self-testing and rereading came out positively associated with grades.
• 15:08
  Ground: Rereading too. Not just the good strategy.
• 15:11
  Figure: Both. Now, the strongest students — the ones above a 3.6 — those were the reliable self-testers, so the self-testing signal is real and it's where success clusters. But rereading didn't fall on its face in the real world the way the clean lab story predicts. Their own read is that it's confounded — grades pool across courses where testing matters and where it doesn't, and motivation and interest and everything else is in the mix.
• 15:39
  Ground: Here's what I take from that, and I don't think it softens the science — I think it locates it. The lab tells you what a technique does to durable memory, all else equal. Real life is never all else equal. That survey can't tell us why rereading was positively associated with GPA — motivation, interest, differences between courses, and other confounds may all be in it. A strategy that feels awful, that makes you feel stupid every session, is one most people quietly abandon. [pause] So the feeling isn't just a liar to be caught and punished. It's also the thing that keeps the whole enterprise going. You can't run purely on the number from the delayed test, because a person who feels like a failure every day stops showing up. The trick isn't to kill the feeling. It's to stop letting it be your evidence.
• 16:34
  Figure: That's the reconciliation I can live with. Keep the feeling as fuel. Fire it as a witness. It can motivate you; it can't be allowed to testify about whether you've actually learned the thing.
• 16:45
  Ground: Then give me the instrument that can testify. Because we've spent a long time saying the built-in gauge is broken. What replaces it?
• 16:55
  Figure: The single cleanest tool in the whole field, and it's almost free. Nelson and Dunlosky, 1991 — the delayed judgment. If you ask people to predict their own recall right after studying, the accuracy of that prediction, measured as a correlation, sits around point three eight. Above chance, but far below the delayed judgments.
• 17:16
  Ground: Because they're judging while the answer's still warm.
• 17:19
  Figure: But make the same person wait — even a few minutes — and predict from the cue alone, with the answer gone, and the accuracy of the prediction jumps to about point nine three. Nearly perfect.
• 17:33
  Ground: That's an enormous swing for such a small change. All you did was move when the guess happens.
• 17:40
  Figure: You moved it from a moment governed by how fluent it felt to a moment governed by whether you can actually get it back. You made the judgment happen under the test's conditions instead of the study's. Same person, same material — you just stopped asking them at the one moment guaranteed to fool them.
• 18:00
  Ground: So if I wanted one habit out of all this, it's not even a study technique. It's a judging technique. Don't ask yourself "do I know this" while you're staring at it. Shut the book, wait, then ask.
• 18:14
  Figure: And keep a little record of it — what you predicted, what you actually got, delayed by at least a day. The gap between those two numbers is the most useful thing you can measure about yourself, because it tells you which way your gauge is bent this week. Overshooting by fifteen points for a stretch means you're still trusting fluency; lean harder into pulling it out cold. Undershooting means trust the schedule and stop over-studying out of fear.
• 18:41
  Ground: And the spacing — how long do I wait between pulls?
• 18:44
  Figure: Cepeda's group mapped that in 2008, and it's tidier than you'd expect. The best gap scales with how long you want to keep it. To hold something for a week, space it about a day. For a month, about a week. For a year, about a month. Get the gap right and, in some of their conditions, recall improved by up to a hundred and fifty percent.
• 19:05
  Ground: Now I want to talk about the part that doesn't come for free, because everything so far has been about holding onto one thing. What about carrying a thing across — using what I learned here somewhere else.
• 19:19
  Figure: Transfer. And the sober news is that far transfer — one domain leaping to a distant one — is rare. It mostly doesn't happen on its own. Skills sit stuck to the context you learned them in unless you do something deliberate to unstick them.
• 19:34
  Ground: That runs against the whole romance of the generalist, doesn't it. The idea that you get good at learning itself and it flows everywhere.
• 19:42
  Figure: It punctures it. One evidence-backed way to actually build transfer is almost mechanical. Gick and Holyoak showed it in the eighties — study one example of a principle and it stays welded to that example. Study two examples side by side and force yourself to write down what they share, the structure underneath — and now you've got something abstract that travels.
• 20:05
  Ground: Two cases, and the deliberate act of naming what's common between them.
• 20:11
  Figure: The comparison is the move. Not the examples, the comparing. That's how you make a principle that isn't chained to where you found it.
• 20:21
  Ground: I like that because it makes transfer a thing you build, not a thing you have. It's construction work. You lay two things next to each other and you go looking for the beam that runs through both. And you write it down, so it becomes an object you can carry into the next room instead of a mood that fades by morning. [pause] It also means the generalist isn't someone with a magic portable mind. It's someone who's in the habit of asking, every time — what is this an instance of? What else works this way? That's not talent. That's a practice you can just decide to do.
• 20:59
  Figure: And one honest limit on the whole ambition, so we don't oversell it. There's a temptation, once you know all this, to think enough deliberate practice reshapes anyone into anything. Macnamara and her colleagues ran the numbers in 2014 — how much of the difference in skill deliberate practice actually explains. In games, about a quarter. Music, a fifth. Sports, under a fifth. In education, four percent. In the professions, under one.
• 21:29
  Ground: The share deliberate practice explains varies wildly by domain — about a quarter in games, a fifth in music, under a fifth in sports, four percent in education, and under one percent in professions.
• 21:44
  Figure: Starting point, working memory, when you began, how predictable the domain even is — all of it matters, and it varies wildly by field. Practice is necessary. It's nowhere near sufficient. The method we've been describing is a real lever, but a lever still needs something to push against.
• 22:04
  Ground: Which brings it back around, for me, to the oldest line in this whole subject.
• 22:09
  Figure: Ausubel, 1968. If he had to reduce all of educational psychology to one principle, it was this: the most important single factor influencing learning is what the learner already knows. Find that out and teach accordingly. Prior knowledge explains somewhere between thirty and sixty percent of the variance in the final scores, and it strongly predicts where you end up — though on average it doesn't predict the size of the gains. It often out-predicts intelligence.
• 22:39
  Ground: That's almost a relief after all the ways the feeling betrays us. The biggest lever isn't a trick you feel or fail to feel. It's just the ground you're already standing on.
• 22:51
  Figure: New things stick to old things. A fact with nowhere to attach slides off. Which is why the expert and the novice can't use the same methods — the scaffolding that saves a beginner becomes clutter for someone who's already built the structure. What counts as one difficulty for you depends entirely on what you already are.
• 23:12
  Ground: So we end up somewhere honest and a little dizzying. The feeling of understanding can mislead you in either direction, depending on the conditions, and you can't turn it off, and you shouldn't, because it's also the thing that keeps you in the seat.
• 23:28
  Figure: You can't put the feeling down. It's the only thing that makes the work feel worth doing. You can only stop mistaking it for the verdict — hand it a stopwatch, make it wait a day, and let the cold recall be the witness instead.
• 23:41
  Ground: The ease was never the enemy. Believing the ease — that was the whole mistake.
• 23:48
  Ground: So when the material finally goes smooth and it feels like you've got it — is that the moment to trust yourself, or the exact moment to stop?