Latest / Stuff You Should Know / Research tips from SYSK

Transcript

Research tips from SYSK · Stuff You Should Know

0:01
Speaker 1: Welcome to Stuff You Should Know from House Stuff Works
0:04
Speaker 1: dot com. Hey, and welcome to the podcast. I'm Josh
0:13
Speaker 1: Clark with Charles W. Chuck Bryant and Jerry This Stuff
0:17
Speaker 1: you Should Know. Um, Josh, We're gonna do something weird today.
0:22
Speaker 1: I'm gonna do a listener mail at the head of
0:25
Speaker 1: the podcast. I know, all right, what? All right, let's
0:29
Speaker 1: do it. Okay, this is from b Wait, hold on,
0:32
Speaker 1: can you have the listener mail? Music going? Oh? I
0:34
Speaker 1: don't know, Jerry, should we go the whole nine yards?
0:37
Speaker 1: So let's do it? People might freak out? All right?
0:41
Speaker 1: This is from Bianca. Uh voice is what I'm gonna say.
0:46
Speaker 1: I think that's great. Hey, guys wrote you not too
0:48
Speaker 1: long ago asking about Hey, research your own podcast. It
0:51
Speaker 1: just got back from a class where we talked about
0:53
Speaker 1: research misrepresentation and journal articles. Apparently journals don't publish everything
0:58
Speaker 1: that is submitted, and lot of researchers don't even publish
1:01
Speaker 1: their studies they don't like the results. Uh. Some laws
1:04
Speaker 1: have been put into place to prevent misrepresentation, such as
1:07
Speaker 1: researchers having to register their studies before they get results
1:10
Speaker 1: and journals only accepting preregistered studies. But apparently this is
1:14
Speaker 1: not happening at all, even though it is now technically law.
1:18
Speaker 1: This ends with the general public being misinformed about methods
1:21
Speaker 1: and drugs that work. For example, there are twenty five
1:24
Speaker 1: studies proving a drug works, twenty five that don't. It's
1:28
Speaker 1: more likely that twenty of the positive results have been
1:31
Speaker 1: published and only one or two of the negative. Uh.
1:34
Speaker 1: And that is from Bianca, and that led us to
1:37
Speaker 1: this article on our own website. Ten signs that that
1:42
Speaker 1: study is bogus, and here it is nice Chuck. Well,
1:46
Speaker 1: we get asked a lot about research from people usually
1:49
Speaker 1: in college. Are like, you guys are professional researchers. How
1:52
Speaker 1: do I know I'm doing a good job and getting
1:54
Speaker 1: good info? And it's getting harder and harder these days,
1:59
Speaker 1: it really is. You know. One sign that I've learned
2:01
Speaker 1: is if you are searching about his study, and all
2:06
Speaker 1: of the hits that come back are from different news organizations,
2:11
Speaker 1: and they're all within like a two three day period
2:14
Speaker 1: from a year ago, nothing like, nothing more recent than
2:18
Speaker 1: that than somebody released a sensational study and no one
2:22
Speaker 1: did any actual effort into investigating it, and there was
2:25
Speaker 1: no follow up. If you dig deep enough, somebody might
2:28
Speaker 1: have done follow up or something like that, but for
2:29
Speaker 1: the most part, it was just something that splashed across
2:32
Speaker 1: the headlines, which more often than not is the is
2:35
Speaker 1: the case as far as science reporting goes. So that's
2:38
Speaker 1: a bonus. That's the eleven boom. How about that? Yeah,
2:43
Speaker 1: should we just start banging these out let's do it?
2:46
Speaker 1: Al Do you have some other clever well apart and
2:51
Speaker 1: parcel with that, I don't know if it's clever. You
2:53
Speaker 1: do come across people who you know can be trusted
2:56
Speaker 1: and relied upon to do good science reporting. So like
3:00
Speaker 1: Ed Young is one, another guy named Ben gold Acre
3:04
Speaker 1: has something called bad Science I don't remember what, Yeah,
3:06
Speaker 1: outlet he's with. And then there's a guy I think
3:09
Speaker 1: scientific American named John Horgan who's awesome. Yeah, or some
3:13
Speaker 1: journalism organizations that have been around instood the test of
3:16
Speaker 1: time that you know are really doing it right, like
3:18
Speaker 1: a nature Yeah. Scientific Americans are like really science Yeah,
3:23
Speaker 1: Like I feel I feel really good about using those sources. Yeah,
3:26
Speaker 1: but even they can you know, there's there's something called
3:29
Speaker 1: scientis um where there's a lot of like faith in
3:32
Speaker 1: dogma associated with the scientific process, and you know, you
3:36
Speaker 1: have to root through that as well. Try it. I'm done. Uh.
3:40
Speaker 1: The first one that they have here on the list
3:42
Speaker 1: is that it's unrepeatable, and that's a big one. UM.
3:46
Speaker 1: The Center for Open Science did a study, UH was
3:49
Speaker 1: a project really where they took two hundred and seventy
3:52
Speaker 1: researchers and they said, you know what, take these one
3:55
Speaker 1: hundred studies that have been published already psychological studies and
4:00
Speaker 1: pour over them. And uh in just last year. They
4:03
Speaker 1: took him a while, took them several years. They said,
4:06
Speaker 1: you know what, more than half of these can't even
4:08
Speaker 1: be repeated using the same methods. They're not reproducible. Nope,
4:11
Speaker 1: not reproducible. That's a big one. And and ones that
4:15
Speaker 1: means that they they when they carried out they followed
4:18
Speaker 1: the methodology. UM Scientific method podcasts. You should listen to
4:22
Speaker 1: that one. That was a good one. That they found
4:25
Speaker 1: that their results were just not what the what the
4:27
Speaker 1: people published, not anywhere near them. UM. For example, they
4:31
Speaker 1: used one as an example where a study found that
4:34
Speaker 1: men were terrible at a determining whether a woman was
4:40
Speaker 1: giving them UM some sort of like a clues to
4:44
Speaker 1: attraction or just being friendly, sexy, sexy stuff or yeah,
4:49
Speaker 1: or good to meet you or buzz off jerky um.
4:53
Speaker 1: And they did the study again and as part of
4:56
Speaker 1: this Open Science Center for Open Science study or survey,
5:00
Speaker 1: and they found that that was not reproducible or that
5:04
Speaker 1: they came up with totally different results. And that was
5:06
Speaker 1: just one of many. Yeah, and in this case specifically,
5:09
Speaker 1: they looked into that study and they found that it was, um,
5:13
Speaker 1: one was in the United Kingdom, one was in the
5:15
Speaker 1: United States. May have something to do with it. But
5:17
Speaker 1: the point is, Chuck, is if you're talking about humanity,
5:22
Speaker 1: I don't think that the study was like the American
5:25
Speaker 1: male is terrible at it. It's men are terrible at it. Right.
5:28
Speaker 1: So that means that whether it's in the UK, which
5:30
Speaker 1: is basically the US with an accent and a penchant
5:33
Speaker 1: for t I'm just kidding UK s soon, Um, the
5:39
Speaker 1: it should be universal. Yeah, you know, unless you're saying, no,
5:44
Speaker 1: it's just this only applies to American men, the American men, right,
5:51
Speaker 1: then it's not even study. Yeah. The next one we
5:54
Speaker 1: have is, uh, it's it's plausible, not necessarily a provable.
6:00
Speaker 1: And this is a big one because and I think, um,
6:04
Speaker 1: we're talking about observational studies here more than lab experiments,
6:07
Speaker 1: because with observational studies, you know, you sit in a
6:10
Speaker 1: room and get asked three questions about something, and all
6:14
Speaker 1: these people get asked the same questions, and then they
6:15
Speaker 1: pour over the data and they draw out their own observations.
6:20
Speaker 1: And one of the very famously an observational study that
6:23
Speaker 1: led to false results found a correlation between having a
6:26
Speaker 1: type A personality and UM being prone to risk for
6:31
Speaker 1: heart attacks and UM. For a long time, you know
6:35
Speaker 1: that the news outlets were like, oh, yes, of course,
6:38
Speaker 1: that makes total sense. This study proved what we've all
6:40
Speaker 1: known all along, UM, And then it came out that no,
6:44
Speaker 1: actually what was going on was a well known anomaly
6:50
Speaker 1: where you have a five percent um risk that chance
6:54
Speaker 1: will produce something that looks like a statistically significant correlation
6:59
Speaker 1: when it's not all when really it's just total chance.
7:02
Speaker 1: And science is aware of this, especially with observational studies,
7:07
Speaker 1: because the more questions you have, the more opportunity you
7:10
Speaker 1: have for that five percent chance to create a seemingly
7:13
Speaker 1: statistically significant correlation when really it's not there. It was
7:19
Speaker 1: just random chance where if somebody else goes back and
7:22
Speaker 1: does the same same study, they're not going to come
7:25
Speaker 1: up with the same results. But the if a researcher
7:31
Speaker 1: is I would guess willfully blind to that five percent chance, um,
7:36
Speaker 1: they will go ahead and produce the study and be like, no,
7:39
Speaker 1: it's true, here's the results right here, go ahead and
7:41
Speaker 1: report on it and make my career. Yeah. Well, and
7:43
Speaker 1: they also might be looking for something inter affect Chances
7:45
Speaker 1: are they are, Um, it's not just some random studying. Look,
7:50
Speaker 1: let's just see what we get if we ask a
7:51
Speaker 1: bunch of weird questions. It's like, hey, we're looking to
7:54
Speaker 1: try and prove something most likely so that better minehoff
7:57
Speaker 1: thing might come into play where you're kind of chair
8:00
Speaker 1: picking data. Yeah, that's a big problem that kind of
8:03
Speaker 1: comes up. A lot of these are really kind of
8:05
Speaker 1: interrelated to totally. The other big thing that's in related
8:08
Speaker 1: is how the media reports on science these days. Yeah,
8:11
Speaker 1: you know, he's a big deal. Yeah, Like in John
8:14
Speaker 1: Oliver just recently went off on this and NPR did
8:17
Speaker 1: a thing on it, like they might even like the
8:21
Speaker 1: researcher might say plausible, but it doesn't get portrayed that
8:23
Speaker 1: way in the media. Sure, remember that poor kid who
8:27
Speaker 1: thought he found the ancient Mayan city. The media just
8:31
Speaker 1: took it and ran with it. You know, yeah, I
8:33
Speaker 1: think there was a lot of maybe or it's possible,
8:36
Speaker 1: we need to go check kind of thing. That mean,
8:38
Speaker 1: He's like, no, he discovered an ancient Mayan city never
8:41
Speaker 1: known before. Yeah, and let's put it in a headline.
8:43
Speaker 1: And that's I mean, that's the That's just kind of
8:45
Speaker 1: the way it is these days. Do you have to
8:48
Speaker 1: be able to sort through it? I guess that's what
8:50
Speaker 1: we're doing here, aren't we, Chuck, We're telling everybody how
8:52
Speaker 1: to sort through it, or at the very least take
8:55
Speaker 1: scientific reporting with a grain of salt, right, Not like
9:00
Speaker 1: you don't necessarily have the time to go through and
9:04
Speaker 1: double that research and then check on that research and
9:07
Speaker 1: you know, right, so take it with a grain of salt.
9:10
Speaker 1: Um unsound samples. Uh, here was this study that basically said, um,
9:17
Speaker 1: how you lost your virginity. It's going to have a
9:20
Speaker 1: very large impact and play a role on how you
9:24
Speaker 1: feel about sex and experienced sex for the rest of
9:27
Speaker 1: your life. Yeah, it's possible. Sure, it seems logical, so
9:33
Speaker 1: we'll just go with it. But when you um only
9:38
Speaker 1: interview college students and uh, you don't you only interview
9:44
Speaker 1: heterosexual people, Then you can't really say you've done a
9:47
Speaker 1: robust study, now, can't you. Plus you also take out
9:50
Speaker 1: of the sample size your sample population, anybody who reports
9:55
Speaker 1: having had a violent encounter, Throw them out, that data out.
10:00
Speaker 1: That's not gonna inform how you feel about sex, right exactly.
10:02
Speaker 1: You're just narrowing it down further and further and again,
10:05
Speaker 1: cherry picking the data by throwing people out of your
10:08
Speaker 1: population sample that don't that will throw off the data
10:12
Speaker 1: that you want. Yeah, and I've never heard of this
10:14
Speaker 1: acronym weird and um. A lot of these studies are
10:18
Speaker 1: conducted by professors in academics. So a lot of times
10:21
Speaker 1: you've got college students as your sample, and there's something
10:25
Speaker 1: called weird Western educated from industrialized rich in democratic countries. Right,
10:31
Speaker 1: those are the participants in the studies studies subject. But
10:34
Speaker 1: then they will say, men, right, well, what about the
10:39
Speaker 1: gay man in Africa? Like you didn't ask him? So
10:44
Speaker 1: that was that's actually a really really big deal. Um.
10:47
Speaker 1: In two thousand and ten, these three researchers did a
10:50
Speaker 1: a survey of a ton of social science and behavioral
10:55
Speaker 1: science studies found that eight percent of them used your
11:00
Speaker 1: study participants. So basically it was college kids for eighty
11:04
Speaker 1: percent of these papers. And they surveyed a bunch of
11:07
Speaker 1: papers and they took it a little further and they
11:10
Speaker 1: said that, um, people who fit into the weird category
11:15
Speaker 1: only make up twelve percent of the world population, but
11:18
Speaker 1: they represent eight percent of the population of these studies.
11:21
Speaker 1: And a college student, chuck in North America, Europe, Israel,
11:28
Speaker 1: or Australia is four thousand times more likely to be
11:32
Speaker 1: in a scientific study than anyone else on the planet.
11:36
Speaker 1: And they're basing psychology and behavioral sciences are basing their
11:40
Speaker 1: findings onto everybody else based on this this small tranch
11:45
Speaker 1: of humanity. And that's a that's a big problem. That's
11:49
Speaker 1: extremely misleading. Yeah, and it's also a little insulting because
11:52
Speaker 1: what they're essentially saying is like, this is who matters.
11:57
Speaker 1: Well also, yeah, but what's sad is this is who
12:00
Speaker 1: I am going to go to the trouble of recruiting
12:03
Speaker 1: for my study. It's just sheer laziness. And I'm sure
12:07
Speaker 1: a lot of them are like, well, I don't have
12:09
Speaker 1: the funding to do that. I guess I see that.
12:13
Speaker 1: But at the same time, I guarantee there's a tremendous
12:16
Speaker 1: amount of laziness involved. Yeah, or maybe if you don't
12:19
Speaker 1: have the money, maybe don't do that study. Is it
12:24
Speaker 1: that simple? I'm probably over simple plying. I don't know.
12:26
Speaker 1: I'm sure we're going to hear from some people in
12:28
Speaker 1: academia about this one. We'll stop using weird participants, or
12:33
Speaker 1: at the very least say, um, like this is sexual
12:38
Speaker 1: Dartmouth students. This applies to them, not everybody in the world.
12:46
Speaker 1: Of these studies where used those people as study participants,
12:50
Speaker 1: and they're not even they're not even emblematic of the
12:52
Speaker 1: rest of the human race. Like college students are shown
12:56
Speaker 1: to see the world differently than other people around the world.
13:01
Speaker 1: And so it's not like you can be like, well,
13:03
Speaker 1: it still works, you can still extrapolate. It's like flawed
13:06
Speaker 1: in every way, shape and form. Probably take a break. Come, yeah,
13:09
Speaker 1: let's take a break because, uh, you're get a little
13:12
Speaker 1: hot under the collar. I love it, man. Uh, we'll
13:15
Speaker 1: be right back after this. Just so much, All right,
13:40
Speaker 1: what's next, buddy? Uh? Very small sample sizes. Right, if
13:47
Speaker 1: you do a study with twenty mice, then you're not
13:54
Speaker 1: doing a good enough study. No, So They used this
13:58
Speaker 1: um in the the article. They use the idea of
14:02
Speaker 1: ten thousand smokers and ten thousand non smokers, and they said, okay,
14:07
Speaker 1: if you have a population sample that size, that's not bad.
14:11
Speaker 1: It's a pretty good start. And you find that fifty
14:13
Speaker 1: of the smokers developed lung cancer but only five percent
14:16
Speaker 1: of non smokers did, then your study has what's called
14:19
Speaker 1: a high power. UM. It's if if you had something
14:24
Speaker 1: like ten smokers and ten non smokers, and two of
14:27
Speaker 1: the smokers developed lung cancer and one developed lung cancer
14:30
Speaker 1: as well, you have very little power and you should
14:35
Speaker 1: have a very little confidence in your findings. But regardless,
14:39
Speaker 1: it's still going to get reported if it's a sexy
14:42
Speaker 1: idea yea for sure. Um. And because these are kind
14:47
Speaker 1: of overlapping in a lot of ways, it was I
14:49
Speaker 1: want to mention this guy, a scientist named Ulrich uh
14:53
Speaker 1: Dirnegle Uh. He and his colleague Malcolm McCloud have been trying,
14:57
Speaker 1: I mean, and there are a lot of scientists that
14:59
Speaker 1: are trying to clean this up because they know it's
15:01
Speaker 1: a problem. But he co wrote an article in Nature.
15:04
Speaker 1: Uh that's called robust research. Colon institutions must do their
15:10
Speaker 1: part for reproducibility. So this kind of ties back into
15:14
Speaker 1: the reproducing things, like we said earlier, and his whole ideas,
15:18
Speaker 1: you know what good funding they should tie funding too
15:21
Speaker 1: good institutional practices, like you shouldn't get the money if
15:25
Speaker 1: you can't show that you're doing it right. Um, and
15:28
Speaker 1: he said that would just weed out a lot of stuff.
15:31
Speaker 1: Here's one staggering stat for reproducibility and small, simple size. Uh.
15:36
Speaker 1: Biomedical researchers for drug companies reported that of their only
15:42
Speaker 1: of the papers that they published or even reproducible, that
15:46
Speaker 1: was like an insider stat And it doesn't matter. They
15:50
Speaker 1: still drugs are still going to market, yeah, which is
15:54
Speaker 1: that's a really good example of why this does matter
15:56
Speaker 1: to the average person. You know, like if you hear
15:59
Speaker 1: something thing like um uh, monkeys like to cuddle with
16:05
Speaker 1: one another because they are reminded of their mothers. Study shows.
16:10
Speaker 1: Do you could just be like, oh, that's great, I'm
16:12
Speaker 1: going to share that on the internet. Doesn't really affect
16:15
Speaker 1: you in any way. But when their studies being conducted
16:19
Speaker 1: that are that are creating drugs that could kill you
16:24
Speaker 1: or not treat you or that kind of thing, And
16:26
Speaker 1: is it's attracting money and funding and that kind of
16:30
Speaker 1: stuff that's like that's harmful. Yeah. Absolutely. I found another survey,
16:37
Speaker 1: did you like that terrible study idea? That it came up?
16:41
Speaker 1: The monkeys like the cuddle A hundred and forty trainees
16:47
Speaker 1: at the MD Anderson Cancer Center in Houston, Texas, Thank
16:52
Speaker 1: you Houston for being so kind to us at a
16:54
Speaker 1: recent show. They found that nearly a third of these
16:58
Speaker 1: UM trainees felt pressure to support their mentors work. It's
17:04
Speaker 1: like to get ahead or not get fired. So that's
17:07
Speaker 1: another issue. As you've got these trainees or residents, uh,
17:11
Speaker 1: and you have these mentors, and even if you disagree
17:13
Speaker 1: or don't think it's a great study, you're you're pressured
17:15
Speaker 1: into just going along with it. I could see that
17:18
Speaker 1: for sure. There's there seems to be a huge hierarchy.
17:21
Speaker 1: And UM science can a lab. You know, you've got
17:24
Speaker 1: the person who runs the lab, it's their lab, and
17:27
Speaker 1: go against them. But there are people UM like Science
17:30
Speaker 1: and Nature to great journals are updating their guidelines right now.
17:34
Speaker 1: They're introducing checklists. UM Science hired statisticians to their panel
17:40
Speaker 1: of reviewing editors, not just other you know, peer reviewed
17:44
Speaker 1: like they actually actually hired numbers people specifically because that's
17:49
Speaker 1: a bigger process. That's a huge part of studies. It's
17:52
Speaker 1: like these this mind breaking statistical analysis that can be
17:56
Speaker 1: used for good or ill. And I mean, I don't
17:59
Speaker 1: think the average scientists necessarily is a whiz at that,
18:04
Speaker 1: although I it has to be part of training, but
18:06
Speaker 1: not necessarily. And that's a different kind of beast altogether.
18:09
Speaker 1: Um Stats, we talked about it earlier. I took a
18:12
Speaker 1: stats class in college, had so much trouble that was
18:16
Speaker 1: awful at it. It really just it's a special kind
18:19
Speaker 1: of is it even matter? Yeah, I didn't get it.
18:25
Speaker 1: I passed it, though I passed it because my professor
18:29
Speaker 1: took pity on me. Um that Ulric during durnagal Ulric
18:37
Speaker 1: narnagal Um. He is a he's a big time crusader
18:42
Speaker 1: for his jam making sure that science is good science.
18:47
Speaker 1: One of the things um he crusades against is the
18:49
Speaker 1: idea of remembering that virginity study where they just threw
18:52
Speaker 1: out anybody who had a violent encounter for their first
18:56
Speaker 1: sexual experience. UM. Apparently that's a big deal with annam
19:00
Speaker 1: studies as well. If you're studying the effects of a
19:02
Speaker 1: drug or something like there was this one in the article. Um,
19:06
Speaker 1: if you're studying the effects of a stroke drug and
19:09
Speaker 1: you've got a control group of mice that are taking
19:12
Speaker 1: the drug or that aren't taking the drug, and then
19:15
Speaker 1: a test group that are getting the drug. Um, and
19:18
Speaker 1: then like three mice from the test group die even
19:22
Speaker 1: though they're on the stroke drug. They die of a
19:24
Speaker 1: massive stroke, and you just literally and figuratively throw them
19:27
Speaker 1: out of the study, um, and don't include them in
19:30
Speaker 1: the results. That changes the data. And he's been on
19:35
Speaker 1: a peer of you on a paper before. He's like, no,
19:37
Speaker 1: this doesn't pass peer of you. You can't just throw
19:39
Speaker 1: out what happened to these three rodents? You started with ten,
19:42
Speaker 1: there's only seven reported in the end. What happened to
19:45
Speaker 1: those three? And how many of them just don't report
19:49
Speaker 1: the ten? They're like, oh, we only started with seven.
19:51
Speaker 1: Were going, you know, well, I was about to say
19:55
Speaker 1: I get the urge. I don't get it because it's
19:56
Speaker 1: not right. But I think what happens is you work
19:59
Speaker 1: so hard at something yeah yeah, and you're like, how
20:01
Speaker 1: can I just walk away from two years of this
20:04
Speaker 1: because it didn't get a result? Okay, The point of
20:08
Speaker 1: real science, though you have to walk away from it. Well,
20:11
Speaker 1: you have to publish that. And that's the other thing too,
20:14
Speaker 1: And I guarantee scientists will say, hey, man, try getting
20:18
Speaker 1: a negative paper published in a good journal. These days,
20:21
Speaker 1: you don't want that kind of stuff. But part of
20:23
Speaker 1: it also is I don't think it's enough to just
20:26
Speaker 1: have to be published in like a journal. You want
20:28
Speaker 1: to make the news cycle as well. That makes it
20:31
Speaker 1: even better, right, Um So, I think there's a lot
20:34
Speaker 1: of factors involved. But ultimately, if you take all that
20:36
Speaker 1: stuff away, if you take the culture away from it,
20:39
Speaker 1: you're if you get negative results, you're supposed to publish
20:43
Speaker 1: that so that some other scientists can come along and
20:45
Speaker 1: be like, oh, somebody else already did this using these
20:48
Speaker 1: methods that I was going to use. I'm not gonna
20:50
Speaker 1: waste two years of my career because somebody else already did.
20:54
Speaker 1: Thank you, buddy for saving me this time and trouble
20:57
Speaker 1: and effort to know that this does not work. You've
21:00
Speaker 1: proven this doesn't work when you saw it to prove
21:02
Speaker 1: it does work, you actually proved it didn't work. That's
21:05
Speaker 1: part of science. Yeah, I wish there wasn't a negative
21:10
Speaker 1: connotation to a negative result, because to me, it's the
21:14
Speaker 1: value is the same as proving something does work as
21:17
Speaker 1: proving something doesn't work. Right again, it's just not Yeah,
21:22
Speaker 1: but I'm not sexy either, so maybe that's why I
21:24
Speaker 1: get it. Uh. Here's one that I didn't know was
21:27
Speaker 1: a thing, predatory publishing. You never heard of this. So
21:31
Speaker 1: here's the scenario. You're a doctor or a scientists and um,
21:35
Speaker 1: you get an email from a journal that says, hey,
21:39
Speaker 1: you got anything interesting for us. I've heard about your
21:42
Speaker 1: work and you say, well, actually, do I have this
21:44
Speaker 1: this study right here? They say, cool, we'll publish it.
21:47
Speaker 1: You go great, my career is taking off. Then you
21:50
Speaker 1: get a bill he says, where's my three grand for
21:54
Speaker 1: publishing your article? And you're like, I don't owe you
21:57
Speaker 1: three grand, all right, give us two? And you know
22:00
Speaker 1: I can't even give you two. And if you fight
22:02
Speaker 1: them long enough, maybe they'll drop it and never work
22:06
Speaker 1: with you again. Or maybe it'll just be like, well,
22:09
Speaker 1: we'll talk to your next court. Exactly. That's called predatory publishing.
22:13
Speaker 1: And it is I'm not sure how new it is.
22:16
Speaker 1: Maybe it's pretty new. Is it pretty new? But it's
22:19
Speaker 1: a thing now where uh, you can pay essentially to
22:24
Speaker 1: get something published. Yes, you can, um it kind of
22:29
Speaker 1: it's kind of like who's who in behavioral science is
22:32
Speaker 1: kind of thing, you know. Um. And apparently it's new
22:36
Speaker 1: because it's a result of open source academic journals, which
22:39
Speaker 1: a lot of people push for, including Aaron Schwartz very
22:43
Speaker 1: famously who like took a bunch of academic articles and
22:46
Speaker 1: published him online and was prosecuted heavily for it. Persecuted.
22:50
Speaker 1: You could even say, um, but the idea that science
22:53
Speaker 1: is behind this paywall, which is another great article from
22:56
Speaker 1: Priceonomics by the way, um, really just takes a lot
23:00
Speaker 1: of people off. So they started open source journals right,
23:04
Speaker 1: and as a result, predatory publishers came about and said, okay, yeah,
23:08
Speaker 1: let's make this free, but we need to make our
23:10
Speaker 1: money anyway, so we're going to charge the academic who
23:14
Speaker 1: wrote the study for publishing it. Well yeah, and and
23:16
Speaker 1: sometimes now it's just a flat out scam operation. There's
23:22
Speaker 1: this guy named Jeffrey Beale who is a research librarian.
23:26
Speaker 1: He is my new hero because he's truly like one
23:30
Speaker 1: of these dudes that has uh, he's trying to make
23:33
Speaker 1: a difference and he's not profiting from this, but he's
23:36
Speaker 1: spending a lot of time by creating a list of
23:41
Speaker 1: of predatory publishers. Yeah, a significant list too. Yeah, how
23:45
Speaker 1: many four thousand of them right now? Um. Some of
23:51
Speaker 1: these companies flat out lie, like they're literally based out
23:54
Speaker 1: of Pakistan or Nigeria and they say no, we're in
23:57
Speaker 1: New York. Yea publisher, So it's just a flat out scam.
24:02
Speaker 1: Or they lie about their review practices. Um, like they
24:05
Speaker 1: might not have any review practices and they straight up
24:08
Speaker 1: lie and say they do. There was one called Scientific
24:11
Speaker 1: Journals International out of Minnesota that he found out was
24:15
Speaker 1: just one guy like literally working out of his home,
24:19
Speaker 1: just lobbying for articles, charging to get them published, not
24:23
Speaker 1: reviewing anything, and just saying I'm a journal, I'm a
24:27
Speaker 1: scientific journal. He shut it down apparently or tried to
24:31
Speaker 1: sell it. I think he was found out. Um and
24:35
Speaker 1: this other one, the International Journal of Engineering Research and Applications.
24:40
Speaker 1: They created an award and then gave it to itself
24:45
Speaker 1: and even modeled the award from an Australian TV award
24:48
Speaker 1: like the physical Statute. That's fascinating. I didn't they could
24:52
Speaker 1: do that. I'm gonna give ourselves. Yeah, let's the best
24:56
Speaker 1: podcast in the Universal Award. It's gonna look like the Oscar. Yeah, okay,
25:02
Speaker 1: the Oscar crusted with the Emmy. Uh this other one
25:05
Speaker 1: med med No publications actually confused the meaning of STM
25:11
Speaker 1: Science Technology Medicine. They thought it meant sports technology in Medicine. No. Well,
25:16
Speaker 1: a lot of UM science journalists or scientists too. But
25:20
Speaker 1: watchdogs like to send like gibberish articles into those things
25:25
Speaker 1: to see if they publish him, and sometimes they do.
25:27
Speaker 1: Frequently they do sniff them off the case it's the
25:30
Speaker 1: big time. How about that call back? It's been a while,
25:33
Speaker 1: it needs to be a T shirt. Did we take
25:37
Speaker 1: a break? Yeah, all right, we'll be back and finish
25:39
Speaker 1: up right after this. Just so much so, here's a
26:05
Speaker 1: big one. You ever heard the term follow the money? Hm.
26:09
Speaker 1: That's applicable to a lot of realms of society, and
26:14
Speaker 1: most certainly in journals. UM. If something looks hinky, just
26:20
Speaker 1: do a little investigating and see who's sponsoring their work. Well,
26:23
Speaker 1: especially if that person is like, no, everyone else is wrong.
26:28
Speaker 1: Climate change is not man made kind of thing. Sure,
26:31
Speaker 1: you know, if you look at where their funding is
26:34
Speaker 1: coming from, you might be unsurprised to find that it's
26:37
Speaker 1: coming from people who would benefit from the idea that
26:41
Speaker 1: anthropogenic climate change isn't real. Yeah, well we might as
26:44
Speaker 1: well talk about him Willie Soon. Yeah, Mr Soon. Is
26:48
Speaker 1: he a doctor? He's a He's a physicist of some sort. Yeah,
26:52
Speaker 1: all right, m M. I'm just gonna say Mr. Or
26:56
Speaker 1: doctor Soon because I'm not positive. Uh. He is one
27:00
Speaker 1: of a few people on the planet Earth. Um professionals,
27:05
Speaker 1: that is, who deny human climate change, human influence climate
27:11
Speaker 1: change like you said, he said the fancier word for
27:13
Speaker 1: it though, and anthropogenic um. And he works at the
27:19
Speaker 1: Harvard Smithsonian Center for Astrophysics. So hey, he's with Harvard.
27:24
Speaker 1: He's got the cred right. Um. Turns out when you
27:28
Speaker 1: look into where he's getting his funding. Uh. He received
27:30
Speaker 1: one point two million dollars over the past decade from
27:34
Speaker 1: Exxon Mobile, the Southern Company, the Cokes, and the Koke brothers,
27:39
Speaker 1: their foundation, the Charles G. Coke Foundation. Excen stopped in
27:44
Speaker 1: stopped funding him, but the bulk of his money and
27:47
Speaker 1: his funding came and I'm sorry, I forgot the American
27:50
Speaker 1: Petroleum Institute came from people who clearly had a dog
27:54
Speaker 1: in this fight. And it's just how can you trust this?
27:59
Speaker 1: You know? Yeah, well you trusted because there's a guy
28:02
Speaker 1: and he has a PhD in aerospace engineering by the way,
28:05
Speaker 1: all right, he's a duck. He works with this, um,
28:08
Speaker 1: this organization, the Harvard Smithsonian Center for Astrophysics, which is
28:12
Speaker 1: a legitimate place. Um, it doesn't get any funding from Harvard,
28:16
Speaker 1: but it gets a lot from NASA and from the Smithsonian. Well,
28:20
Speaker 1: and Harvard's very clear to point this out when people
28:22
Speaker 1: ask him about Willie Soon, Um, they're kind of like, well,
28:26
Speaker 1: here's the quote. Willie Soon as a Smithsonian staff researcher
28:29
Speaker 1: at Harvard Smithsonian Center for Astrophysics, a collaboration of the
28:33
Speaker 1: Harvard College Observatory in the Smithsonian Astrophysical Observatory. Like they
28:38
Speaker 1: just want to be real clear. Even though he uses
28:40
Speaker 1: a Harvard email address, he's not our employee. No, but
28:44
Speaker 1: again he's getting lots of funding from NASA and lots
28:46
Speaker 1: of funding from the Smithsonian. This guy, Um, if his
28:51
Speaker 1: scientific beliefs are what they are, and he's a smart guy,
28:55
Speaker 1: then yeah, I don't know about like getting fired for saying,
28:59
Speaker 1: you know, here's a paper on on the idea that
29:03
Speaker 1: climate change is not human made. Yeah, he thinks it's
29:06
Speaker 1: the Sun's fault. But he didn't. He doesn't reveal in
29:10
Speaker 1: any of his um conflicts of interest. Uh, that should
29:15
Speaker 1: go at the end of the paper. He didn't reveal
29:16
Speaker 1: where his funding was coming from. And I get the
29:20
Speaker 1: impression that in academia, if you're are totally cool with
29:23
Speaker 1: everybody thinking like you're a shill, you can get away
29:28
Speaker 1: with it. Right. Well, this stuff, a lot of this
29:32
Speaker 1: stuff is not illegal. Right, Even predatory publishing is not illegal,
29:36
Speaker 1: just unethical. And if you're counting on people to police
29:40
Speaker 1: themselves with ethics, a lot of times will disappoint you.
29:43
Speaker 1: The Heartland Institute gave Willie Soon a Courage Award, and
29:47
Speaker 1: if you're not caring about what other scientists think about
29:50
Speaker 1: if you've heard the Heartland Institute, you might remember them.
29:53
Speaker 1: They are a conservative think tank. You might remember them
29:56
Speaker 1: in the nineties when they worked alongside Philip Morris to
30:00
Speaker 1: deny the risks of secondhand smoke. Yeah, that's all chronicle
30:04
Speaker 1: In that book. I've talked about merchants of doubt, A
30:08
Speaker 1: bunch of scientists, legitimate bona fide scientists who are like
30:11
Speaker 1: up for for um, being bought by groups like that said,
30:19
Speaker 1: it is sad um and the whole the whole thing
30:21
Speaker 1: is they're saying like, well, you can't stay without beyond
30:25
Speaker 1: a shadow of a doubt with absolute certainty, that that's
30:29
Speaker 1: the case, and science is like, no, science doesn't do that.
30:32
Speaker 1: Science doesn't do absolute certainty. But the average person reading
30:35
Speaker 1: a newspaper sees that, oh you can't stay with absolute certainty, Well,
30:38
Speaker 1: then maybe it isn't man made. And then there's that
30:41
Speaker 1: doubt that the people just go and get the money
30:44
Speaker 1: for for saying that, for writing papers about it. It's
30:47
Speaker 1: millions of Yeah, it really is. Um self reviewed. Uh,
30:54
Speaker 1: you've heard of peer review. We've talked about it quite
30:56
Speaker 1: a bit. Your reviews. When you have a study and
30:59
Speaker 1: then one or more ideally more of your peers reviews
31:03
Speaker 1: your study and says, you know what, you had best practices,
31:05
Speaker 1: You did it right. Um, it was reproducible, you follow
31:08
Speaker 1: the scientific method. Um, I'm gonna give it my stamp
31:11
Speaker 1: of approval and put my name on it, not literally
31:14
Speaker 1: or is it? I think so? It says who reviewed
31:16
Speaker 1: it in the journal when it's published, but not my
31:20
Speaker 1: name as the author of study. You know what I mean? Um,
31:24
Speaker 1: And the peer reviewer. Yeah, as a peer reviewer, and
31:26
Speaker 1: that's a wonderful thing. But people have faked this and
31:31
Speaker 1: been their own peer reviewer, which is not how it works. No,
31:36
Speaker 1: who is this guy? Uh? Well, I'm terrible at pronouncing
31:42
Speaker 1: Korean names, so all apologies, but I'm gonna say nung
31:46
Speaker 1: In Moon. Nice, Dr Moon, I think, yeah, let's call
31:50
Speaker 1: m Dr Moon. Okay. So Dr Moon Um worked on
31:54
Speaker 1: natural medicine, I believe, and was submitting all these papers
31:59
Speaker 1: that were getting very quickly because apparently part of the
32:02
Speaker 1: process of peer of views to say, this paper is great,
32:05
Speaker 1: Can you recommend some people in your field that can
32:08
Speaker 1: review your paper? And Dr Moon said, I sure can.
32:12
Speaker 1: He was on fire. Let me go make up some
32:14
Speaker 1: people and make up some email addresses that actually come
32:18
Speaker 1: to my inbox and just posed as all of his
32:20
Speaker 1: own peer reviewers. He was lazy, though, is the thing, Like,
32:25
Speaker 1: I don't know that he would have been found out
32:26
Speaker 1: if he hadn't been um careless. I guess because he
32:33
Speaker 1: was returning the reviews within like twenty four hours. Sometimes
32:36
Speaker 1: a peer of view of like a real um study
32:40
Speaker 1: should take I would guess weeks, if not months, like
32:44
Speaker 1: the the study the publication schedule for the average study
32:48
Speaker 1: or paper. I don't think it's a very quick thing.
32:50
Speaker 1: There's not a lot of quick turner And this guy
32:53
Speaker 1: was like twenty four hours. Dr Moon. I see your
32:57
Speaker 1: paper was reviewed and accepted by Dr Mooney. It's like
33:03
Speaker 1: I just added a Y to the end. It seemed easy. Uh.
33:06
Speaker 1: If you google peer review fraud, you will be shocked
33:11
Speaker 1: at how often this happens and how many legit science
33:16
Speaker 1: publishers are having to retract studies. Uh. And it doesn't
33:21
Speaker 1: mean they're bad. Um, they're getting duped as well. But
33:24
Speaker 1: there was one based in Berlin that had sixty four
33:28
Speaker 1: retractions because of fraudulent reviews. And they're just one publisher
33:33
Speaker 1: of many. Every publisher out there probably has been duped. Um.
33:37
Speaker 1: Maybe not everyone, I'm surmising that, but it's a big problem.
33:44
Speaker 1: I'll review it. It'll end up in the headlines. Now,
33:48
Speaker 1: every single publisher duped, says Chuck. Uh. And speaking of
33:52
Speaker 1: um the headlines, Chuck. One of the problems with science
33:56
Speaker 1: reporting or reading science reporting is that what you usually
34:01
Speaker 1: are hearing, especially if it's making a big splash, is
34:04
Speaker 1: what's called the initial findings. Somebody carried out a study,
34:07
Speaker 1: and this is what they found, and it's amazing and
34:10
Speaker 1: mind blowing and it um, it supports everything everyone's always known.
34:15
Speaker 1: But now there's a scientific study that says, yes, that's
34:17
Speaker 1: the case. And then if you wait a year or two,
34:21
Speaker 1: when people follow up and reproduce the study and find
34:25
Speaker 1: that it's actually not the case, it doesn't get reported on.
34:28
Speaker 1: Usually yeah, and and sometimes the science scientists or the
34:33
Speaker 1: publisher is they're doing it right, and they say initial findings,
34:38
Speaker 1: but the public and sometimes even the reporter will say
34:41
Speaker 1: initial findings. But we as a people that ingest this
34:46
Speaker 1: stuff need to understand what that means, um, And the
34:49
Speaker 1: fine print is always like you know, you know, more
34:52
Speaker 1: studies needed, but no one if it's something that you
34:56
Speaker 1: want to be true, you'll just say, hey, look at
34:59
Speaker 1: the study, right. You know it's brand new and they
35:02
Speaker 1: need to study for twenty more years, but hey, look
35:05
Speaker 1: what it says. And the more the more you start
35:08
Speaker 1: paying attention to this kind of thing, the more kind
35:10
Speaker 1: of disdain you have for that kind of just off
35:14
Speaker 1: hand um sensationalist science reporting. But you'll still get caught
35:20
Speaker 1: up in it. Like every once in a while, I'll
35:22
Speaker 1: catch myself like saying something you'd be like, oh, did
35:24
Speaker 1: you hear this? And then as I'm saying it out loud,
35:26
Speaker 1: I'm like, that's preposterous. Yeah, there's no way that's going
35:29
Speaker 1: to pan out to be true. I got collebated, I know.
35:34
Speaker 1: I mean, we we have to avoid this stuff. It's
35:36
Speaker 1: stuff because we have our name on this podcast. But
35:40
Speaker 1: luckily we've given ourselves the back door of saying, hey,
35:44
Speaker 1: we make mistakes a lot. It's true though we're not
35:48
Speaker 1: we're not scientists. Uh. And then finally we're gonna finish
35:51
Speaker 1: up with the header on this one is it's a
35:55
Speaker 1: cool story. And that's a big one because, um, it's
35:59
Speaker 1: not enough these days. And this all ties in with
36:02
Speaker 1: the media and how we read things as people. But
36:05
Speaker 1: it's not enough just to have a study that might
36:08
Speaker 1: prove something. You have to wrap it up in a
36:10
Speaker 1: nice package to deliver people, get it in the news cycle.
36:14
Speaker 1: And the cooler the better. Yep. It almost doesn't matter
36:19
Speaker 1: about the science as far as the media is concerned.
36:23
Speaker 1: They just want a good headline and a scientist who
36:26
Speaker 1: will say, yeah, that's that's cool. Here's what I found.
36:30
Speaker 1: This is going to change the world. Lockness Monster is real.
36:35
Speaker 1: This is a kind of ended up being depressing somehow
36:38
Speaker 1: not somehow Yeah, like, yeah, it's kind of depressing. We'll
36:45
Speaker 1: figure it out, Chuck. Well, we do our best. I'll
36:48
Speaker 1: say that science will prevail, I hope. So. Uh, if
36:52
Speaker 1: you want to know more about science and scientific studies
36:55
Speaker 1: and research fraud and all that kind of stuff, just
36:57
Speaker 1: type some random words into the search part how to
37:00
Speaker 1: work dot com. See what comes up. And since I
37:02
Speaker 1: said random, it's time for a listener mail. Oh no, oh, yeah,
37:06
Speaker 1: you know what. It's time for administrator All right, Josh,
37:16
Speaker 1: administrative details. If you're new to the show, you don't
37:19
Speaker 1: know what it is. That's the very clunky title. We're saying,
37:22
Speaker 1: thank you to listeners who send us neat things. It
37:26
Speaker 1: is clunky and generic and I've totally gotten used to
37:28
Speaker 1: it by now. Well you're the one who made it
37:30
Speaker 1: up to be clunky and generic, and it's stuck. Yeah.
37:33
Speaker 1: So people send us stuff from time to time, and
37:35
Speaker 1: it's just very kind of you to do so, yes,
37:38
Speaker 1: and we like to give shout outs whether or not
37:40
Speaker 1: it's just out of the goodness of your heart, or
37:42
Speaker 1: if you have a little small business that you're trying
37:44
Speaker 1: to plug either way, it's a sneaky way of getting
37:47
Speaker 1: it in there. Yeah, but I mean I think we
37:49
Speaker 1: we brought that on, didn't we didn't We say like,
37:51
Speaker 1: if you have a small business, then you send us something,
37:53
Speaker 1: we'll we'll be happy to say something exactly. Thank you.
37:56
Speaker 1: All right, so let's get it going here. We got
37:58
Speaker 1: some coffee right from uh, from one thousand faces right
38:02
Speaker 1: here in Athens, Georgia from Kayla. Yeah, delicious, Yes it was.
38:08
Speaker 1: We also got some other coffee too, from Jonathan at
38:10
Speaker 1: Steamworks Coffee. He came up with a Josh and Chuck blend.
38:13
Speaker 1: Oh yeah, it's pretty awesome. I believe it's available for
38:16
Speaker 1: sale to Yeah. That Josh and Chuck blend is dark
38:19
Speaker 1: and bitter. Uh. Jim Simmons, he's a retired teacher who
38:26
Speaker 1: sent us some lovely handmade wooden bowls and a very
38:30
Speaker 1: nice handwritten letter, which is always great. Thanks a lot, Jim. Uh.
38:34
Speaker 1: Let's see. Chamberlayne sent us homemade pasta, including a delicious
38:39
Speaker 1: savory pumpkin fettuccini. It was very nice. Um. Jake graft
38:44
Speaker 1: two F's send us a postcard from Great Wall of China.
38:47
Speaker 1: It's kind of neat. Sometimes we get those postcards from
38:50
Speaker 1: places we've talked about. I was like, thanks, here, let's
38:56
Speaker 1: see the Hammer Press team. They sent us a bunch
38:58
Speaker 1: of Mother's Day cards that are wonderful. Oh those were
39:00
Speaker 1: really nice, really great. You should check them out. The
39:02
Speaker 1: Hammer Press team. Yeah. Uh, Misty Billy and Jessica. They
39:07
Speaker 1: sent us a care package of a lot of things.
39:10
Speaker 1: There were some cookies, um, including one of my favorite
39:13
Speaker 1: white chocolate dipped rits, and peanut butter crackers. Oh yeah, man,
39:17
Speaker 1: I love those homemade right yeah. And uh, then some
39:21
Speaker 1: seventies Macromay for you, along with seventies Macromay magazines because
39:27
Speaker 1: you're obsessed with Macromay. We have a Macromay plant holder
39:30
Speaker 1: hanging from my um microphone arms a coffee mug sent
39:36
Speaker 1: to us by Joe and Lynda heckt oh that's right,
39:38
Speaker 1: and it has some pens in it. Uh. And they
39:40
Speaker 1: also sent us a Misty Billy and Jessica a lovely
39:43
Speaker 1: little hand drawn picture of us with their family, which
39:45
Speaker 1: was so sweet, awesome. Um. We've said it before, we'll
39:48
Speaker 1: say it again. Huge thank you to Jim Ruaine. I
39:52
Speaker 1: believe that's how you say his name and the Crown
39:54
Speaker 1: Royal people for sending us all the Crown Royal We
39:57
Speaker 1: are running low. Uh. Mark Silberg at the Rocky Mountain
40:02
Speaker 1: Institute sent us a book called Reinventing Fire. They're great
40:06
Speaker 1: out there, man, they know what they're talking about. And
40:09
Speaker 1: I think it's reinventing Fire. Colon Bold Businesses Bowld business
40:12
Speaker 1: Solutions for the New Energy Era. Yeah, they're they're basically
40:16
Speaker 1: like um, green energy observers, but I think they um,
40:20
Speaker 1: they're experts in like all sectors of energy, but they
40:23
Speaker 1: have a focus on green energy, which is awesome. Yeah,
40:25
Speaker 1: they're pretty cool. Um john whose wife makes Delightfully Delicious
40:30
Speaker 1: doggie treats. Delightfully Delicious is the name of the company.
40:33
Speaker 1: There's no artificial colors or flavors. And they got um
40:36
Speaker 1: sweet little Momo hooked on sweet potato dog treats. I
40:40
Speaker 1: thought you're gonna say, hooked on the junk, the the
40:43
Speaker 1: sweet potato junk. She's crazy cuckoo for sweet potatoes. Nice.
40:47
Speaker 1: That's good for a dog too. It is very h
40:50
Speaker 1: Stratt Johnson sent us his band's LP And if you're
40:53
Speaker 1: in a band, your name is Strat. That's pretty cool. Uh,
40:57
Speaker 1: die omeya still mhm. I think that was great. Yeah,
41:04
Speaker 1: I'm not sure if I pronounced all right, d I
41:05
Speaker 1: O M A e a uh. Frederick, this is long overdue.
41:11
Speaker 1: Frederick at the store one five to one store dot
41:15
Speaker 1: com sent us some awesome low profile cork iPhone cases
41:20
Speaker 1: and passport holders, and I was telling him, Jerry walks
41:23
Speaker 1: around with her iPhone in the cork holder and it
41:26
Speaker 1: looks pretty sweet. Yeah, so he said, AWESO, I'm glad
41:29
Speaker 1: to hear. Joe and Holly Harper sent us some really
41:31
Speaker 1: cool three D printed stuff you should know, things like
41:36
Speaker 1: s y s k uh, you know, like a little
41:39
Speaker 1: desk oh as, like after Robert Indiana's love sculpture. Yeah,
41:42
Speaker 1: that's what I couldn't think of what that was from. Yeah,
41:44
Speaker 1: it's awesome. It's really neat and like a bracelet um
41:48
Speaker 1: made out of stuff you should know, three D carved
41:51
Speaker 1: like plastics, really neat. Yeah, they did some good stuff.
41:54
Speaker 1: Thanks Joe and Holly Harper for that. And then last
41:57
Speaker 1: for this one, we got a postcard from Yosemite National
42:01
Speaker 1: Park from Laura Jackson, So thanks a lot for that.
42:04
Speaker 1: Thanks to everybody who sends us stuff. It's nice to
42:06
Speaker 1: know we're thought of and we appreciate it. Yeah, we're
42:09
Speaker 1: gonna finish up with another set on the next episode
42:13
Speaker 1: of Administrative Details. You got anything else, No, it's it.
42:17
Speaker 1: Oh yeah. If you guys want to hang out with
42:19
Speaker 1: us on social media, you can go to s Y
42:22
Speaker 1: s K Podcast on Twitter or on Instagram. You can
42:26
Speaker 1: hang out with us at Facebook dot com, slash stuff
42:28
Speaker 1: you Should Know. You can send us an email to
42:30
Speaker 1: Stuff Podcast at how stuff Works dot com and has
42:33
Speaker 1: always joined us at our home on the web, Stuff
42:35
Speaker 1: you Should Know dot com For more on this and
42:43
Speaker 1: thousands of other topics. Is it how stuff Works dot com.