Latest / The Mastery Project Podcast / Stop Rereading and Start Retrieving to Improve Retention
Transcript
- Dr. T-Michael Dougherty, Ed.D.: In 2006, two researchers at Washington University in St. Louis published a study that should have changed how every student in this country studies. It did not. The finding was simple, though, and it's been repeated hundreds of times since. It holds up in laboratories, in middle school classrooms, in medical schools, and even in professional exam prep. And roughly nine out of ten students still do the opposite of what it says. And I want to tell you why. Not because students are lazy. In fact, the people that I've taught are some of the hardest working students that I've ever met. They study the wrong way because the wrong way feels better while you are doing it. Now, there's kind of a gauge in your head that tells you how well you're doing when you're studying. That gauge is reading the wrong thing. And once you know what it's actually measuring, you can stop trusting it and start measuring something that actually matters. Welcome to the Mastery Project Podcast. I'm Dr. T. Michael Dougherty This show is about how learning works, how high performance actually occurs, and how ordinary people get very good at doing hard things. I spent my working career in physical therapy education, which means I've watched people try to learn enormous amounts of very difficult information very quickly with a licensing exam that waits at the end of it. And I've watched people work incredibly hard and still struggle. Now, usually the difference between those two outcomes isn't ours, it's method. So today we're going to begin with the most repeated finding in the entire science of learning, and with the reason almost nobody uses it. Let's discuss this study. Henry Rediger and Jeffrey Karpicki at Washington University in St. Louis published a study in the journal Psychological Science in 2006. The title was Test Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. And here's what they did. They gave college students short reading passages, ordinary academic material, science writing, it was a few hundred words. And then they split the students into groups, and they gave every group the exact same amount of extra time with that material. One group read the passage again and again and again. The other group read it once and then spent all their remaining time taking some recall tests. Blank sheet of paper, write down everything you remember without peeking. Same material, same total time, the only difference was how they spent it. And then everybody got tested. Some students five minutes later, some two days later, and some a full week later. At five minutes, the rereading group won. They remembered about 80% of the passages that they read. The self-testing group remembered about 70% of what they had read. Now, if the experiment had ended right there, the lesson would be obvious. Just study more. Testing just gets in the way, so don't do it. But a week later, the results flipped things completely. The students who reread remembered about 40%. The students who tested themselves remembered about 61%. So think about those two numbers, 40 and 61. Now let's put it another way. Out of every hundred facts in that passage, one group walked away with 40 of them, and the other walked away with 61 of them. Same reading, same clock, 21 more facts And here's the piece that I want you to think about for the rest of this episode. The researchers also asked the students to predict how much they thought they could remember. The students who reread predicted they would do better. They were confident and they were wrong. Now, why it happens? Why would the method that makes you feel confident be the same method that makes you forget? That doesn't make any sense. Well, turns out there are two reasons. The first one is about where confidence actually comes from. When you reread a page, the words go down easy. Nothing catches in your head. The sentences show up already familiar. Your eyes move quickly and it feels smooth. Researchers have a name for that smoothness, and we call it fluency. And your brain treats fluency as sort of evidence that you already know it. If it goes down easy, the logic goes, I must have already read it. Now, two other researchers studied this in 2005, Koriat and Bjork. They called it an illusion of competence. The conditions that make something feel easy to read aren't the same conditions that make it easy to retrieve later. And you can't feel that gap from the inside. Nothing warns you about it. So here's how I think about it. Fluency is the difference between recognizing a face and remembering the name that goes with it. Recognizing it is effortless. I've seen this before. Yes, it's familiar. Remembering the name is a completely different job inside your brain. You have to reach into a big messy space and come back with the one specific thing. Rereading practices recognizing For most students, things like exams and clinical experiences, or the moment somebody asks you a question where everybody turns to look at you, those all require remembering. You've been practicing the easy one. Now, the second reason, and this one explains the process and it's the more useful of the two. Bjork and Bjork at UCLA laid out a framework in 1992 that has held up remarkably well in this sense. They said that every memory has two separate strengths, and the two can drift apart. The first is how deeply the memory is planted, or how solid it is, whether it's really locked in. The second is how close at hand it happens to be right this second, how fast it comes back when you recall it. Those two things kind of sound like the same thing, but they aren't. Rereading makes a memory very easy to reach for a very short period of time. It does almost nothing to implant it deeper into your brain, which is exactly why the rereading group won at five minutes and fell apart at one week. Everything was sitting right at hand, but nothing was actually built in. the second thing the Bjork said is the part that changes how you should study. They propose that a memory gets planted deeper in proportion to how hard you had to work to pull it out, as long as you do pull it out. The harder the reach, the deeper the memory gets when it goes back down. Think of a path through the woods. Rereading is looking at a map of the woods. Retrieving is walking the path. Every time you walk that path, it gets a little clearer and the walk gets a little bit easier for you. Staring at the map doesn't wear a path into anything. Two other researchers in 2009 tested this directly and got the same result. The harder you reach for a memory, as long as you get there, means better memory later. Now, I want to be careful here because this idea can sometimes get twisted. Difficulty on its own is probably not worth much. Struggling, failing, and walking away teaches you nothing at all about what you're trying to learn. The Bjorks were specific about this, and they used the phrase. Desirable difficulty. A difficulty is desirable if you can eventually overcome it, or if somebody shows you the right answer afterwards. Struggle plus the answer is where learning kind of lives. Struggle alone is just struggle. We're going to spend an entire episode on that distinction later on in this season, but because most of what we know about learning kind of depends upon it. Now, I want to talk about this Swahili study. I said earlier that students predict incorrectly, and I want to show you just how incorrectly they often do. Karpiki and Rediger in 2008 published a study in the Journal Science, and the title was The Critical Importance of Retrieval for Learning. And they taught college students 40 different Swahili words and their English meanings. Everybody learned all 40 of them until they had gotten each one right at least once. Then The group split four different ways. The first group kept studying all 40 words and kept testing all 40. The second group stopped studying a word once they got it right, but kept testing everything. And the third group kept studying everything but stopped testing a word once they got it right. The fourth group dropped words from both studying and testing. A week later, everyone came back for one final test. The two groups that kept testing themselves remembered about 80% of the words. The two groups that kept studying but stopped testing remembered only about 35%. 80 versus 35. out of 40 words, that is 32 words against 14. And look at what that tells you. Studying a word you already got right added basically nothing. Testing a word you already got right more than doubled what people kept. Now here's the part that I find hard to believe. At the end of the session, the researchers asked every student to predict how much they would remember in a week. All four groups gave about the same answer, which was about 50%. The group that would go on to remember 32 words and the group that would go on to remember 14 felt exactly the same about how their studying went. The gauge was indifferent. It had no idea. And that's the actual problem that this episode is about. The weaker method doesn't announce itself. There is no alarm, there's no warning light, that nothing feels wrong. Right up until the moment somebody asks you the question, but you can't retrieve the answer. Now I want to talk about concept mapping Because in another study, there's an objection that I want to close. Somebody always says, sure, testing beats rereading, but rereading is a low bar. What about the good study methods? What about concept mapping when you draw out all the ideas and show their connections? It's a fair question, and they tested that. Karpicki and Janelle Blunt in 2011 also published in the journal Science. The retrieval practice produces more learning than elaborative studying with concept mapping. Students read science texts. In this study, students read science texts. One group built concept maps with the material open in front of them. It was a real technique and it's very well respected. They're drawing out relationships and linking ideas visually. The other group practiced recall. They would read it, close the book, write down what they could, open it, close it, write it down again, etc. one week later the recall group did better. They did better on straight factual questions, and they did better on the harder questions, the ones that required connecting two ideas that the text never connected for them. And here's the detail that ties this back to everything else. The researchers ran a version where the final test itself was building a concept map. The recall group still won. The students who spent their time practicing recall built better concept maps than the students who spent their time building concept maps. And once again, the students had predicted the exact opposite. They were sure the mapping would serve them better. So, does this hold up in the real world? At this point, you might reasonably think these are college students, these are short passages, vocabulary words, under controlled conditions. Does any of this survive in an actual classroom? It does, and there are three lines of evidence for it. First, the big pooled reviews. When researchers want to know whether something is really true, they do what's called a meta-analysis. They gather every experiment ever run on the question that they can find, and they pool all the results together and look at what the whole body of evidence suggests. One experiment can be a fluke. A hundred pooled together usually isn't. Christopher Rowland did this in 2014. Across the whole body of research, the advantage for self-testing came out at about half a standard deviation, which sounds like nothing until you translate it. Take a student sitting right in the middle of the class, switch them from rereading to self-testing. The change alone moves them into roughly the top third. In 2017, another group of researchers did the same thing, and they pooled 272 separate comparisons, but they got a slightly larger number. In plain terms, when you line up two students who are equally capable, where one rereads and one self-tests, about two times out of three, the one who self-tested comes out ahead. For education research, this is a big effect. Most things schools try don't move the needle anywhere near that far. In terms of real classroom research with real kids, a group of researchers ran a series of studies in a middle school science class and social studies class in Columbia, Illinois. Teachers taught normally, and some of the material got a short quiz during the unit that was presented, and other material got reviewed, but it was never quizzed. And then the students took the teachers' own unit exam that they created. The quizzed material came back roughly a full letter grade higher than the material that was never quizzed. Same teacher, same kids, same classroom, same week. The only difference was whether they had ever been asked to produce it from memory before. Third, and this matters most for anybody whose work involves judgment. In 2010, Andrew Butler showed that students who practiced recall did better when the questions changed. New questions they had never seen, questions in a different subject built on the same underlying idea. The old complaint about testing was always that it just produces shallow memorization, sort of parroting. The evidence goes the other direction, though. The students who practiced recall were the ones who could use the knowledge somewhere new. And one more finding I want to get on the record because I think it addresses something people can really appreciate. In 2014, another research group looked at test anxiety in middle school and high school classrooms that used frequent low pressure quizzing. The anxiety levels among students went down, and students in those classes reported feeling much less anxious about big exams, not more, which makes sense once you think about it. A lot of what we call test anxiety is really uncertainty, not knowing whether or not you know the material. if you have already quizzed yourself 20 times, you are not uncertain anymore. You already know exactly where you stand with this information. Now, I want to discuss some of the limitations of some of this research. And here is where some of the findings get a bit shaky. First, testing yourself without checking the answer loses most of its power when you can't recall the item. Roland's review found that getting the answer afterward made a real difference in how big the benefit was. If you can't remember something and nobody ever tells you the answer, you have mostly just practiced not knowing it. So checking isn't optional, checking is half the process. Number two, there's a live argument about complicated material. Van Gogh and Sweller argued in two thousand fifteen that the benefit shrinks. and maybe disappears as material gets more complex. By complex, they meant material where you have to hold several different concepts in your head at the same time to make sense of any of it. Not a vocabulary word, something more like working through a multi-step clinical case, for example. Kropicki and William A.U. pushed back the same year and said the evidence doesn't support that. the effect looks strongest for facts and clear concepts, but somewhat less certain for genuinely complex problem solving, and that the argument isn't over. Number three, the benefit depends on how far away the test is. Almost every study I described today found that rereading wins in the very short term and loses over any real distance. If your exam is in 90 minutes from now, cramming isn't a crazy idea. If your exam is in six weeks, or you actually intend to use this knowledge on a human being someday, a year and a half or two years from now, cramming is a really bad trade-off. So What you can do this week. Let me make this as usable as possible. Four changes, none of them cost you a dime. None of them cost you any time. They only cost a change in your habits. First, the blank page. At the end of anything that you read, any lecture, close everything. Take a blank sheet of paper, write down everything you remember for five to ten minutes. Don't look. Whatever your memory hands you, in whatever order it hands it to you, write it down. Then open the book and find out what you missed. That second part is where the learning occurs. Don't skip it. Two, answer before you read. Before you open the chapter, ask yourself what you already think you know about it. Try to answer questions nobody's even taught you yet. You'll get things wrong, and that's fine. That's actually the point. Nate Cornell and colleagues showed in 2009 that guessing wrong first and then being shown the right answer afterwards actually beats simply studying the right answer. and Barbara Butterfield and Janet Metcalfe found in 2001 that the mistakes people fix most reliably are the ones they were most sure about. Being confidently wrong and then corrected is one of the most efficient learning events available to you. It just doesn't feel good when it's occurring. 3. Stop retiring cards the moment you get them right. This comes straight out of the Swahili study that we mentioned earlier. And I've seen many of my former students do this exact same thing. Everybody's instinct is to pull a card out of the deck, and once they've answered it correctly, it kind of feels like progress. And that instinct costs those students more than half of what they could have kept. Leave it in the deck, space it out further, but keep asking. Four, put down your highlighter. John Dunloskey and colleagues in 2013 went through 10 common study techniques and graded each one on how much good evidence actually supports it. Self-testing and spacing your practice out got the top grade. Highlighting, underlining, rereading, and summarizing got the bottom grade. I'm not going to tell you that highlighting is absolutely useless in every possible studying situation. It's not. I am telling you though that when researchers graded every tool available to you, the one most students lean on hardest came in near the bottom, and the one most students almost never use came in first. And on that point, Karpicki, Butler, and Rediger surveyed college students in 2009 about how they actually studied. The large majority named rereading as their main strategy. About one in ten named testing themselves. The gap is the reason I made this episode. Now, I want to leave you with the one idea that sits underneath all of this. Every study I described today found the same thing generally. The method that felt better worked worse. The method that felt worse worked better. And the people doing it couldn't tell the two apart from the inside, which means the feeling of learning and the fact of learning are two different things. And most of us have been steering by the wrong one our entire lives. You can fix that today with one change. Stop asking yourself whether the material felt familiar to you. Ask whether you could say it out loud with the book closed. The question has a real answer, and you can check it in about 10 minutes. In the next episode, we're going to go one level down from here and into the framework that sort of explains all of this. Robert and Elizabeth Bjork's work on desirable difficulty and the strange finding that the things which slow us down while we are practicing are often the exact things that tend to help make it stick. If you found today's episode useful, the most helpful thing that you can do is to close this and spend 10 minutes writing down what you remember. You'll find the holes in it right away. And that's the whole point. I'm Dr. T. Michael Doherty, and this has been the Mastery Project Podcast. Every study that I mentioned today is in the show notes with the authors, the journal, and the year it was published, so that you can go read it yourself. thank you for listening to this episode. And if you found it helpful, please share it with a colleague who you think might benefit from it as well.