Latest / The Psychology Undergrad Podcast / Experimental Design: Between-Subjects vs Within-Subjects & Random Assignment
Transcript
- 0:00You know, usually when we talk about discovering
- 0:03the truth, there is this underlying expectation
- 0:06of certainty. Oh, for sure. People want concrete
- 0:09answers. Right. We like to think of science as
- 0:12this perfectly clean magnifying glass that we
- 0:14just hold up to the universe. You run a test,
- 0:16the numbers pop out, and boom, you have an objective
- 0:18fact. Well, welcome to the Psychology undergrad.
- 0:21We are so glad you're here with us. Yeah, we
- 0:23are basically just... Two eager undergrad psychology
- 0:27students who spend way too much time studying
- 0:29together. And today, we're taking you along on
- 0:32a deep dive into our chapter on experimental
- 0:35design. Which is honestly one of the most eye
- 0:37-opening chapters so far. It really is. Because
- 0:39the mission for today is to decode how psychological
- 0:43studies actually discover the truth. And it turns
- 0:46out that magnifying glass of science is actually
- 0:49completely warped by the way you designed the
- 0:51glass in the first place. Exactly. I mean, as
- 0:53undergrads, we have a tendency to look at the
- 0:55results of a study, especially in psychology,
- 0:58and just accept them as ground truth. Because
- 1:00it's in a textbook. Right. But the actual truth
- 1:02is buried in the methodology. The architecture
- 1:05of the experiment is where the secret livers
- 1:07of human behavior are really being pulled. If
- 1:10the design is flawed, well, the data is just
- 1:13noise masquerading as a fact. So whether you
- 1:16are cramming for an exam right now, prepping
- 1:19for a work presentation where you have to defend
- 1:22some data, or you are just intensely curious
- 1:25about how the human mind works, knowing how experiments
- 1:29are designed, changes how you see the world.
- 1:32It's a total shortcut to being well informed.
- 1:34Yep. Once you understand the bones of an experiment,
- 1:37you are never going to be tricked by a flashy
- 1:39news headline again. So to understand how we
- 1:41test human behavior, we really have to start
- 1:43at the absolute foundation. Before you can measure
- 1:46any kind of reaction, you have to figure out
- 1:48how to divide up the people you are testing.
- 1:50The participant pool. Exactly. You have a pool
- 1:52of people and you have to decide what specific
- 1:54environment they are going to experience. And
- 1:57this creates a massive core divide in experimental
- 2:00design. Right, you're either going to use a between
- 2:02-subject design or a within -subjects design.
- 2:06And, okay, let's unpack this, starting with between
- 2:08subjects. Let's do it. The defining feature here
- 2:11is total isolation. Each participant is tested
- 2:15in only one single condition. Like, the text
- 2:18gives this really classic example. A researcher
- 2:20takes 100 university students. Pretty standard
- 2:23sample for psych study. Oh, totally. So they
- 2:25take these 100 students, assign 50 of them to
- 2:27a room, and ask them to write a detailed essay
- 2:30about a highly traumatic event from their past.
- 2:32Heavy stuff. Yeah, very heavy. And then the other
- 2:3550 go to a completely different room, and they
- 2:38are asked to write about a totally neutral event,
- 2:41like their morning commute. And the key thing
- 2:44here is that those two groups never cross paths.
- 2:47They are isolated. You are comparing one distinct
- 2:50group of people against a completely different
- 2:53distinct group. You know, you want to see if
- 2:55the act of writing about trauma changes a physiological
- 2:58marker like heart rate or stress hormones compared
- 3:01to the baseline of the people who just wrote
- 3:03about sitting in traffic. But then you have the
- 3:05alternative, which is the within subjects design.
- 3:08And the mechanics here are just totally different
- 3:10because every single participant experiences
- 3:12every single condition. So no isolated groups.
- 3:16Exactly. You use the exact same people. The textbook
- 3:19gives this amazing courtroom example. You put
- 3:22a participant in a room and ask them to judge
- 3:25the guilt of an incredibly attractive defendant
- 3:27based on a case file. Then, after they give their
- 3:31rating, you hand that exact same person a new
- 3:34case file and ask them to judge the guilt of
- 3:37a very unattractive defendant. So you are looking
- 3:39at the changes within a single individual's cognitive
- 3:42process, rather than trying to measure the differences
- 3:45between two completely distinct groups. Right.
- 3:47And you know, a good way to remember this, this
- 3:49really helped me, is that between is like a competition
- 3:51between two rival houses. Right. They are separate.
- 3:54They are fighting it out. They're like that.
- 3:55But within is like an inner journey. Everything
- 3:58is happening within one person's house. That's
- 4:00actually a great memory aid. Thanks. But OK,
- 4:04let's unpack this a little more. Because looking
- 4:06at this, Isn't a within -subjects design always
- 4:09superior? How so? Well, you are comparing a person's
- 4:13reaction to their own baseline. You are totally
- 4:17eliminating messy individual differences like
- 4:20IQ or motivation or just having a bad day. Doesn't
- 4:24that make it objectively better? What's fascinating
- 4:27here is that mathematically, you're completely
- 4:29right. It does control for all those extraneous
- 4:32variables. Right. But practically, it's not always
- 4:34possible, which is why we need to build fair
- 4:37between groups. Like, think about a treatment
- 4:39that causes a permanent change. Oh, like curing
- 4:41something. Exactly. If you are testing a new
- 4:44exposure therapy for severe agoraphobia and it
- 4:46works perfectly in condition A, you can't just
- 4:49erase that participant's memory and put them
- 4:51through a different therapy in condition B. Because
- 4:53their baseline is gone. They aren't agoraphobic
- 4:55anymore. Right. They are permanently altered.
- 4:57So you are literally forced to rely on a between
- 5:00-subjects design. OK. That makes sense. But since
- 5:04between -subjects designs use entirely different
- 5:08people in each group, how do we ensure those
- 5:10groups are actually comparable? Like, how is
- 5:13it fair? That brings us to the fairness doctrine
- 5:15of science. Random assignment. Oh, yeah. This
- 5:18is huge. It really is. And we have to strictly
- 5:21distinguish it from random sampling. Which people
- 5:23mix up all the time. Constantly. Random sampling
- 5:26is how you get your participant pool from the
- 5:28general population, which is actually super rare.
- 5:31Usually researchers just take whoever is available
- 5:33on campus. Right. The classic college sophomore
- 5:36problem. Exactly. But random assignment happens
- 5:39after those people are already in your lab. It's
- 5:42using a random process to decide who goes to
- 5:45which group. Like flipping a coin, heads you
- 5:48write about trauma, tails you write about your
- 5:50commute. It gives everyone an equal chance. Mathematically
- 5:53equal, yes. But there is a flaw with just flipping
- 5:55a coin, right? Especially in small studies. Because
- 5:58if you flip a coin 20 times, you could easily
- 6:01get 14 heads and 6 tails. And that completely
- 6:05ruins your statistical power. Having unequal
- 6:08group sizes makes the smaller group way too vulnerable
- 6:11to outliers. So to fix that, they use block randomization,
- 6:17which I always picture like dealing a deck of
- 6:19cards at a poker table. That's a perfect analogy.
- 6:22Right. You make sure everyone at the table gets
- 6:24one card before anyone gets a second card, so
- 6:27your groups always stay perfectly equal in size.
- 6:29It's an elegant solution, but it does bring up
- 6:32some ethical stuff. I was just going to say that.
- 6:34Yeah. Because we are talking about real human
- 6:36beings here. Let me ask you a moral question.
- 6:39Is it ethically fair to leave a person's potential
- 6:42treatment, especially for something severe like
- 6:44depression, up to a literal coin flip. It sounds
- 6:48so cold when you say it like that. It does. It
- 6:50sounds awful. But if we connect this to the bigger
- 6:52picture, random assignment is actually a moral
- 6:56necessity for science. How so? Because it accounts
- 6:59for human fallibility. If researchers handpicked
- 7:02who got the experimental drug, they might subconsciously
- 7:04pick the patients they feel bad for or the ones
- 7:07they think will respond best. Oh, wow. Yeah,
- 7:09because they want to help them. Exactly. They
- 7:11are human. But doing that introduces massive
- 7:14biases that completely ruin our search for objective
- 7:18truth. Randomness ensures we aren't rigging the
- 7:21game. That makes a lot of sense. So once we randomly
- 7:24assign people, what exactly are we assigning
- 7:27them to? Because this gets into the whole treatment
- 7:29versus control conditions thing. Which is where
- 7:31things get wild. Totally. So treatments are the
- 7:35interventions for betterment, the new drug, the
- 7:37new therapy. But control conditions are the baseline.
- 7:41Like a no treatment group a wait list or a placebo
- 7:44and the placebo effect I mean we have to talk
- 7:47about this heavily We absolutely do because you
- 7:50hear about these folk remedies all the time People
- 7:52drinking chicken soup to cure a cold or I read
- 7:54this one in the text putting a bar of soap under
- 7:56their bedsheets to stop leg cramps The soap thing
- 7:59is wild and from a chemical standpoint. There
- 8:01is no active medical ingredient there But people
- 8:04genuinely report relief. Because the brain's
- 8:07expectation literally alters the physical body.
- 8:10It can trigger natural painkillers, lower stress
- 8:12hormones. It's incredible. Well, and the text
- 8:14explicitly details this 2002 study by Moseley.
- 8:18This absolutely blew my mind. The arthroscopic
- 8:21knee surgery one? Yes. They took people with
- 8:24debilitating knee pain. Half got the real surgery,
- 8:27the other half got a sham surgery. Which means
- 8:29they wheeled them in, gave them a tranquilizer,
- 8:32and the surgeon just made small incisions in
- 8:34their knee. But they didn't do the actual surgery.
- 8:36They even splashed water around to make it sound
- 8:38real. And over the next two years... the sham
- 8:41surgery group healed just as much as the real
- 8:44surgery group. Just because they expected to
- 8:46heal. Right. And here's where it gets really
- 8:48interesting. If the mind's expectation can literally
- 8:51alter the immune system or heal a knee joint,
- 8:54isn't a placebo just a secular form of faith
- 8:56healing? That is a fascinating way to look at
- 8:59it. I mean, it operates on belief. You believe
- 9:01in the ritual of the operating room and your
- 9:03body physically heals. It's faith. This raises
- 9:06an important question, though, about informed
- 9:08consent and fairness. Because ethically, you
- 9:11can't just lie to a patient, cut them open, and
- 9:13leave. Right. The deception is a huge moral issue.
- 9:16So researchers compromise. They use a weightless
- 9:19control condition. So the control group eventually
- 9:21gets the real treatment after a few months. Nobody
- 9:24is denied care forever. Oh, that's smart. Or,
- 9:27even better, they compare the new treatment against
- 9:30the best available alternative. So if you are
- 9:33testing a new depression med, you don't give
- 9:36the control group a sugar pill. You give them
- 9:38the best current medication on the market. So
- 9:41everyone gets help? And everyone has the expectation
- 9:44of getting better. That feels way more ethical.
- 9:46It definitely is. OK, so we've solved the problems
- 9:49with splitting people up in between subjects'
- 9:51designs. But let's pivot back to within subjects.
- 9:55What happens when one person experiences multiple
- 9:57conditions in a row? Ah, the order of things.
- 10:01This introduces carryover effects. Right, because
- 10:03our brains aren't just hard drives we can wipe
- 10:05clean. The residue of the first test carries
- 10:08over to the second. Exactly. You've got the practice
- 10:10effect where people just get better at the task
- 10:13because they've done it already. And you've got
- 10:15the fatigue effect where they just get exhausted
- 10:17and their performance drops. But the context
- 10:19effect is the one that really got me. Going back
- 10:22to that courtroom example, if you judge an average
- 10:25looking defendant all by themselves, you give
- 10:26them a moderate sentence. But if you judge them
- 10:29right after looking at an incredibly attractive
- 10:32defendant. Your baseline is completely skewed.
- 10:35Yes, you judge the average defendant significantly
- 10:38more harshly just because of the contrast. And
- 10:41honestly, it highlights how fragile our moral
- 10:43judgments are. What do you mean? Well, we think
- 10:45of our perception of guilt and justice as these
- 10:48solid, objective things. But... They can be completely
- 10:52warped simply by the sheer order in which information
- 10:55is presented to us. It's terrifying if you think
- 10:57about a judge sitting on the bench all day. Exactly.
- 11:00So how do researchers deal with this? Through
- 11:02counterbalancing. You randomly assign participants
- 11:05to experience the conditions in different orders.
- 11:07It isolates those carryover effects. Right. And
- 11:10the text has this awesome memory aid for complex
- 11:13counterbalancing called a Latin Square design.
- 11:15It's literally just exactly like a Sudoku puzzle.
- 11:18Oh, I love Sudoku. Right. In Sudoku, no number
- 11:21repeats in a row or column. In the Latin square,
- 11:24no treatment repeats in the same position across
- 11:26your groups. So everyone experiences the fatigue
- 11:30differently, and it mathematically washes out
- 11:32in the end. It's brilliant. It isolates the real
- 11:35data from the context. But you know, context
- 11:37is everything. We've seen how too much context
- 11:39ruins within -subjects designs, but the lack
- 11:42of context in between -subjects designs can be
- 11:44even wilder. Oh my gosh, yes. The 1999 Michael
- 11:48Birnbaum study. Yes. This is a perfect example
- 11:51of what happens when you isolate groups too much.
- 11:53Let me just break this down for everyone listening.
- 11:55It sounds impossible. Birnbaum used a between
- 11:58-subjects design. Two totally separate groups.
- 12:01Group A was asked to rate how large the number
- 12:039 is on a scale of 1 to 10. Just the number 9?
- 12:05Just 9. And they gave it an average rating of
- 12:075 .13. Okay. Group B, totally isolated from Group
- 12:11A, was asked to rate the number 221 on that exact
- 12:14same 1 to 10 scale, and they gave it an average
- 12:16rating of 3 .0. Wait, really? Yes. Meaning Group
- 12:20A statistically rated the number 9 as mathematically
- 12:22larger than Group B rated the number 221. That
- 12:25is insane. It is, because group A lacked context.
- 12:29They just implicitly compared 9 to other single
- 12:32digits, and 9 is huge for a single digit. Right,
- 12:34and group B compared 221 to other triple digits,
- 12:38so they rated it small. So what does this all
- 12:41mean? Does this mean our perception of reality
- 12:44is entirely relative to the experimental design?
- 12:47Like, we aren't discovering truth, we're just
- 12:48building it. It honestly does confirm that design
- 12:51shapes reality. The rules of the test dictate
- 12:54what you can discover, which gives us our final
- 12:57rule of thumb for this chapter. Which is? If
- 12:59you can use a within subjects design without
- 13:02carryover effects ruining it, do it. But if the
- 13:05treatment causes a permanent change, you absolutely
- 13:07must use a between subjects design. And just
- 13:10be super careful about random assignment. Exactly.
- 13:13Well, I think that wraps up our study session
- 13:15for today. For you listening, we really hope
- 13:17this helps. The next time you are scrolling and
- 13:19you read a flashy headline about a new study,
- 13:21don't just look at the results. Look at the bones.
- 13:24Look at the bones. Ask yourself. Who was in the
- 13:26control group? What order did they see the information
- 13:29in? Was it a Sudoku puzzle or a coin flip? Right.
- 13:32And I want to leave you with a final kind of
- 13:35mind -bending thought to mull over. We talked
- 13:37about how our physical bodies can heal from a
- 13:40fake surgery just because we believe we were
- 13:42treated. The placebo effect. Yeah. And we talked
- 13:45about how... our strict moral judgments of guilt
- 13:48are totally swayed by the sheer sequence of what
- 13:51we see. The context effect. So, how much of our
- 13:54everyday quote -unquote objective reality is
- 13:57actually just an accidental byproduct of how
- 13:59our daily lives happen to be designed? Oh, wow.
- 14:02That is a heavy thought to end on. I know, right.
- 14:05We'll keep questioning the design, everyone.
- 14:07Thanks for hanging out with us, Psychology Undergrad.
- 14:09Catch you next time.