Latest / Why We Do What We Do / Mini: Science Cheat-Prevention

Transcript

Mini: Science Cheat-Prevention · Why We Do What We Do

• 0:03
  Speaker 1: You're listening to Why we do what we do. Welcome
• 0:15
  Speaker 1: to Why we Do what we do?
• 0:17
  Speaker 4: Many.
• 0:18
  Speaker 2: I'm your host, Abraham, I'm your host, Shane. We are
• 0:21
  Speaker 2: a psychology podcast. We talk about all of the things
• 0:24
  Speaker 2: that humans and non human animals do, and sometimes we
• 0:27
  Speaker 2: talk about the science that we do and how we
• 0:30
  Speaker 2: make that science better or at least we try to.
• 0:33
  Speaker 3: Yes, and so on this MANY, we are going to
• 0:35
  Speaker 3: talk about the idea of pre registering your studies. And
• 0:38
  Speaker 3: for those of you who are in the academic world
• 0:40
  Speaker 3: and maybe you do research, this may be familiar. But
• 0:44
  Speaker 3: as somebody who has done research before, I didn't know
• 0:46
  Speaker 3: this was a thing up until like the last year,
• 0:48
  Speaker 3: and so sometimes you just kind of discover new stuff,
• 0:51
  Speaker 3: and I think this is a really great process that
• 0:53
  Speaker 3: we should at least consider.
• 0:54
  Speaker 2: For those of you who are not scientists, it's probably
• 0:58
  Speaker 2: worth us actually doing a deep dive at some point
• 1:00
  Speaker 2: on like how people lie with research and data, because
• 1:05
  Speaker 2: what's cool about this is like this is actually one
• 1:07
  Speaker 2: of those strategies that's meant to specifically make it near
• 1:11
  Speaker 2: impossible to lie with your data. And there are going
• 1:15
  Speaker 2: to be industries where they will never pre register their
• 1:18
  Speaker 2: study because their whole thing is to lie with their data.
• 1:20
  Speaker 2: So chiropractors, acupuncturists, homeopaths like these are people who are
• 1:25
  Speaker 2: not going to be preregistering their studies because they've got
• 1:28
  Speaker 2: to do that p hack to get to statistic significance.
• 1:32
  Speaker 2: I will say that although we're talking super science nerd stuff,
• 1:35
  Speaker 2: even if that's not your jam, we always try and
• 1:37
  Speaker 2: make our content as consumable and accessible as possible. So
• 1:41
  Speaker 2: we will do our best to make this funny and
• 1:43
  Speaker 2: interesting and informative and have you walk away being more
• 1:46
  Speaker 2: well rounded in your understanding of the scientific process. And
• 1:49
  Speaker 2: if you are already in that field, then just feel
• 1:52
  Speaker 2: validated that we're discussing things that you gotta do and
• 1:55
  Speaker 2: you know sort of how they work already, but maybe
• 1:57
  Speaker 2: there's some tidbits in here that even you didn't know.
• 2:00
  Speaker 4: Hell yeah, yeah, we're going to discover a lot today, all.
• 2:04
  Speaker 2: Right, and quickly before we dive into that, I will
• 2:06
  Speaker 2: let you all know that if you would like to
• 2:08
  Speaker 2: support us, then you can do so by joining us
• 2:09
  Speaker 2: on Patreon, leaving us a rating and review, liking, subscribing,
• 2:13
  Speaker 2: telling a friend, or picking up some merch at our
• 2:15
  Speaker 2: merch store I'll talk a little bit more about that
• 2:17
  Speaker 2: at the end of this discussion. And as we mentioned,
• 2:20
  Speaker 2: this is a mini that means it's a short dive
• 2:22
  Speaker 2: into a specific topic. We put out these episodes every Monday.
• 2:25
  Speaker 2: Our full length episodes come out on Wednesdays. I think
• 2:28
  Speaker 2: that's enough preamble, though, Let's start getting into the pre
• 2:30
  Speaker 2: registering stuff and what that means.
• 2:33
  Speaker 3: Yeah, so when we talk about pre registering your preregistration,
• 2:36
  Speaker 3: it refers to the practice of formally registering elements of
• 2:39
  Speaker 3: a research study, not necessarily the entire study, but maybe
• 2:42
  Speaker 3: even parts prior to researchers conducting the study. So this
• 2:45
  Speaker 3: can include pre registering a hypothesis, a methods section of
• 2:50
  Speaker 3: your study, how data are analyzed, it could include the
• 2:53
  Speaker 3: entire study. But basically you're saying, hey, this is what
• 2:55
  Speaker 3: I'm going to do. Here's exactly how I have established
• 2:59
  Speaker 3: my study, and then I would like to implement it.
• 3:00
  Speaker 4: I'd like to preregister it ahead of time.
• 3:03
  Speaker 2: Yes, and specifically, like you could anyone could be just
• 3:05
  Speaker 2: making these declarations on their own who are scientists, But
• 3:09
  Speaker 2: this is like going to a third party database industry
• 3:13
  Speaker 2: that sort of it lives now, and like you're codifying
• 3:16
  Speaker 2: this into I cannot change this, because you could like
• 3:20
  Speaker 2: write all this stuff down and make that claim that
• 3:22
  Speaker 2: you were going to do all those things and then
• 3:24
  Speaker 2: do your study and then change your mind to do
• 3:26
  Speaker 2: different things, and no one would be the wiser. But
• 3:28
  Speaker 2: in this instance, you are like publicly committing to this
• 3:31
  Speaker 2: in a way that you can't back out of.
• 3:33
  Speaker 4: Right, exactly.
• 3:34
  Speaker 2: There are multiple reasons people would do this, so there's
• 3:37
  Speaker 2: a lot of purposes. One is that it ensures that
• 3:39
  Speaker 2: research teams share a clear understanding of their research goals
• 3:43
  Speaker 2: and processes, because again, you might change how you would
• 3:48
  Speaker 2: interpreter use those data toward the end of whatever experiment
• 3:51
  Speaker 2: you were doing if you didn't have anyone who already
• 3:55
  Speaker 2: had expectations about what you were trying to accomplish.
• 3:58
  Speaker 4: Right.
• 3:58
  Speaker 3: Another one is it separate confirmatory aspects of research from
• 4:02
  Speaker 3: exploratory aspects. So basically what this is saying is that,
• 4:05
  Speaker 3: like when you preregister your study, you're going, I am
• 4:08
  Speaker 3: looking at this information. I am trying to with the
• 4:11
  Speaker 3: study confirm or deny or discover this particular hypothesis. It
• 4:18
  Speaker 3: allows for you to kind of like register that aspect
• 4:21
  Speaker 3: without changing the goalpost as you go forward.
• 4:24
  Speaker 2: That's definitely important. And another one here is that science is,
• 4:28
  Speaker 2: or at least should be, a lot about communication. Like
• 4:31
  Speaker 2: I would say that at the heart of science is
• 4:33
  Speaker 2: people who are communicating their ideas and being open to
• 4:36
  Speaker 2: feedback on their ideas, being open to the fact that
• 4:39
  Speaker 2: they might be wrong. Like this idea that we don't
• 4:42
  Speaker 2: want to say something because someone might disagree with us,
• 4:45
  Speaker 2: I think is absolutely counter to what science is. Science
• 4:48
  Speaker 2: is like you put yourself out there and then you
• 4:50
  Speaker 2: get feedback on that. And if we just treat each
• 4:52
  Speaker 2: other compassionately, then we can both be prepared to receive
• 4:55
  Speaker 2: that feedback and be good at giving that feedback ourselves.
• 4:58
  Speaker 2: And so an aspect of this, one of the purposes
• 5:01
  Speaker 2: for pre registering is that can allow for input from colleagues.
• 5:05
  Speaker 2: In this public space where we people can comment on this,
• 5:08
  Speaker 2: they can make observations and impossibly even suggestions that might
• 5:12
  Speaker 2: then influence how we go about doing our research in
• 5:14
  Speaker 2: a way that is even more robust now because we
• 5:16
  Speaker 2: had more thoughtful eyes on.
• 5:18
  Speaker 4: That, right.
• 5:19
  Speaker 3: And then this also does this really cool thing where
• 5:21
  Speaker 3: it allows editors to actually accept research papers conditionally, regardless
• 5:26
  Speaker 3: of the outcomes. So Basically, there are a couple processes
• 5:28
  Speaker 3: inside of this that allow for peer review to happen,
• 5:31
  Speaker 3: and a peer ofview process can occur before the study
• 5:34
  Speaker 3: is even done to allow editors to go, yes, we
• 5:37
  Speaker 3: will accept this under the condition that it is implemented,
• 5:40
  Speaker 3: that you have data. No matter what your data say,
• 5:42
  Speaker 3: we have accepted this paper, which is really kind of
• 5:44
  Speaker 3: a unique take on the peer review process and accepting
• 5:47
  Speaker 3: papers for publication.
• 5:49
  Speaker 2: It is and I could see a lot of utility
• 5:51
  Speaker 2: and just increased efficiency with this model because it's going
• 5:55
  Speaker 2: to spend less time going through the process and therefore
• 5:58
  Speaker 2: the what we learn can be communicated more quickly. Because
• 6:01
  Speaker 2: in case y'all didn't know this from the start of
• 6:04
  Speaker 2: a study to that study actually being published in a journal,
• 6:07
  Speaker 2: that can I would say, on average is like at
• 6:11
  Speaker 2: least two.
• 6:11
  Speaker 4: Years, right.
• 6:12
  Speaker 2: It can be shorter than that, for sure, but like
• 6:15
  Speaker 2: most of the time, if you start a project today,
• 6:17
  Speaker 2: you're not going to see that project like reaching like
• 6:20
  Speaker 2: people's awareness for two years, and a lot can happen
• 6:24
  Speaker 2: in two years, believe me, right, And sometimes it is
• 6:26
  Speaker 2: a lot longer than that. Just in case you guys
• 6:28
  Speaker 2: weren't all aware of that, that is definitely a thing
• 6:31
  Speaker 2: that it just slows down the pipeline. And so having
• 6:34
  Speaker 2: anything that increases efficiency is something that I think we
• 6:36
  Speaker 2: can generally be appreciative of and something that does not
• 6:39
  Speaker 2: increase the efficiency of podcasts as ads. All right, we're back.
• 6:49
  Speaker 2: Let's talk about the different types of preregistration for studies.
• 6:53
  Speaker 3: Yeah, one of the types is just a standard pre registration,
• 6:57
  Speaker 3: which basically means that you take the research protocol, you
• 7:00
  Speaker 3: you post it prior to the research being conducted. This
• 7:03
  Speaker 3: can include, like I said, the parts of your study,
• 7:06
  Speaker 3: the entire study, but basically you just go through the process,
• 7:09
  Speaker 3: you submit it to one of these third parties that
• 7:10
  Speaker 3: allow you to pre register the study ahead of time.
• 7:14
  Speaker 4: And so this is just kind of a standard practice
• 7:16
  Speaker 4: in general.
• 7:17
  Speaker 2: There's also registered reports. In this they give a description
• 7:21
  Speaker 2: and the analysis and that is registered prior to data collection.
• 7:25
  Speaker 2: And here a theoretical plan is peer reviewed. So it's
• 7:27
  Speaker 2: just the plan itself right at this point, it's not
• 7:30
  Speaker 2: actual data or conclusions drawn from those data. And as
• 7:33
  Speaker 2: a result, the study is sort of provisionally accepted because
• 7:36
  Speaker 2: it's already been vetted before the data collection took place.
• 7:39
  Speaker 2: Where they say essentially whatever the outcome, we're clear that
• 7:43
  Speaker 2: your methods were sound enough that whatever day you get
• 7:45
  Speaker 2: will be able to speak to them, which is interesting,
• 7:47
  Speaker 2: you know, could be interesting. And then after data collection,
• 7:50
  Speaker 2: a second peer review does confirm that the methods were
• 7:52
  Speaker 2: followed as intended and a corresponds to the declared intentions
• 7:56
  Speaker 2: of that method.
• 7:57
  Speaker 3: Yes, A really important aspect for that it's that they're
• 8:01
  Speaker 3: not reviewing the validity of the results in terms of like,
• 8:03
  Speaker 3: we're the results effective. They're saying, did you do what
• 8:06
  Speaker 3: you set out to do, regardless of what your results were.
• 8:09
  Speaker 3: And that's what that second peer review does, because at
• 8:11
  Speaker 3: this point in time the paper has already been accepted.
• 8:13
  Speaker 4: They're just making sure that you did what you said
• 8:15
  Speaker 4: you were going to do, right.
• 8:16
  Speaker 3: And then there's also specialized preregistration, which means that you
• 8:19
  Speaker 3: can actually submit for preregistration for different types of studies
• 8:22
  Speaker 3: that include like qualitative research, mixed methods research. Single case
• 8:25
  Speaker 3: designs is even listed underneath that too, So for you
• 8:27
  Speaker 3: behavior analytic folks out there, that's I think really critically
• 8:30
  Speaker 3: important to know that they are doing single case designs
• 8:33
  Speaker 3: as part of the pre registration process. So if you
• 8:35
  Speaker 3: are a researcher out there, consider this as you go forward.
• 8:38
  Speaker 3: It's going to make our behavior analytic literature a little
• 8:41
  Speaker 3: bit more robust.
• 8:42
  Speaker 2: Yeah, that's cool, all right, But why would we do this?
• 8:46
  Speaker 2: Because we did mention some purposes, but I think that
• 8:48
  Speaker 2: they're uh most let's dig into the whys we need
• 8:50
  Speaker 2: to explain. There are a few benefits to pre registering studies.
• 8:54
  Speaker 2: One of them is that it shows credibility in your work.
• 8:57
  Speaker 2: Like you are. You can't just change what you thought
• 9:00
  Speaker 2: you were going to say or do about this, or
• 9:03
  Speaker 2: change how you analyze the data to make it fit
• 9:06
  Speaker 2: whatever outcome you were trying to get. You kind of
• 9:08
  Speaker 2: just have to accept it as is, and that means
• 9:10
  Speaker 2: that you're more there than more credible as a researcher
• 9:12
  Speaker 2: as a field.
• 9:14
  Speaker 4: Right.
• 9:14
  Speaker 3: It also helps you maximize data sets because now you're
• 9:17
  Speaker 3: going to be able to like kind of map out
• 9:19
  Speaker 3: your plan and you don't have to worry about, you know,
• 9:22
  Speaker 3: additional outliers and your data. You don't have to worry
• 9:24
  Speaker 3: about additional data sets that might come in as a
• 9:27
  Speaker 3: result of these new discoveries. You can actually kind of
• 9:29
  Speaker 3: streamline your data process and maximize that data set that
• 9:32
  Speaker 3: you've got.
• 9:33
  Speaker 2: It also demonstrates research transparency, like you're just you can't
• 9:37
  Speaker 2: hide behind things. It aims to reduce bias, which it
• 9:40
  Speaker 2: does help accomplish some reductions and bias, and it does
• 9:43
  Speaker 2: support the replication of further studies down the road.
• 9:46
  Speaker 3: It also allows you to collaborate more with the scientific community,
• 9:49
  Speaker 3: which I think is really important because you have you've
• 9:52
  Speaker 3: put this in a public space where people can actually
• 9:54
  Speaker 3: review it and give you feedback, so that's.
• 9:55
  Speaker 4: Kind of cool.
• 9:56
  Speaker 3: It does also reduce the pressure to receive positive results
• 9:59
  Speaker 3: because a lot of when you're doing a study, you
• 10:01
  Speaker 3: want it to turn out really well. But if it's preregistered,
• 10:04
  Speaker 3: it doesn't matter how it turns out. What matters is
• 10:06
  Speaker 3: that your scientific method is sound, which I think is
• 10:08
  Speaker 3: really cool. And like we mentioned, it does shorten the
• 10:11
  Speaker 3: peer of view process. It does make that far more
• 10:13
  Speaker 3: efficient and allows you to kind of skirt, not get
• 10:16
  Speaker 3: around the peer of view process. But it does make
• 10:18
  Speaker 3: it a little bit easier to say, yes, this was preregistered,
• 10:20
  Speaker 3: which means it was approved, which means that we have
• 10:22
  Speaker 3: identified it as a sound already. You don't need to
• 10:25
  Speaker 3: comment on the method's part of this or the data
• 10:27
  Speaker 3: set or whatever. You know, you can actually kind of
• 10:28
  Speaker 3: streamline that a little bit.
• 10:30
  Speaker 2: Yeah, yeah, it's just boosting that efficiency. And I do
• 10:32
  Speaker 2: think this positive results are like null results thing I
• 10:35
  Speaker 2: think is really important. This is one of the things
• 10:37
  Speaker 2: where one of the effects and criticisms of scientific work
• 10:41
  Speaker 2: is how often the fact that like if a study
• 10:43
  Speaker 2: doesn't turn out anything that's like statistically significant, that doesn't
• 10:46
  Speaker 2: get reported on. And I actually do have like I've
• 10:49
  Speaker 2: thought about us doing an episode discussing this very phenomenon. Yeah,
• 10:53
  Speaker 2: because I do think there are more nuanced considerations as
• 10:56
  Speaker 2: opposed to just like, well we should be publishing all results.
• 11:00
  Speaker 2: I'm like part of that. Well, we'll get into that
• 11:02
  Speaker 2: end of the discussion. But the point being the fact that, like,
• 11:04
  Speaker 2: if you have declared your methods and peer reviewers have
• 11:08
  Speaker 2: looked at those and said this is good sound scientific
• 11:12
  Speaker 2: like experimentation that you have arranged, then like if you
• 11:15
  Speaker 2: produce some data that like don't really show anything, well
• 11:18
  Speaker 2: it's kind of already been accepted for publication. So then
• 11:21
  Speaker 2: what then happens is we're less likely to not publish
• 11:24
  Speaker 2: those results that don't show anything, because like those data
• 11:28
  Speaker 2: just are what they are. And if it happens to
• 11:30
  Speaker 2: be like a null hypothesis, like we did not made
• 11:32
  Speaker 2: a statistically significant outcome, Then that's useful information that people
• 11:37
  Speaker 2: can actually like operate off of r and be like,
• 11:39
  Speaker 2: whatever effect we thought was there is not robust enough
• 11:42
  Speaker 2: to have been captured by this study at least, right,
• 11:44
  Speaker 2: And that's something to know that, like that might not
• 11:46
  Speaker 2: be a thing absolutely now, Just like anything, there are
• 11:49
  Speaker 2: definitely concerns related to pre registering and things that could
• 11:52
  Speaker 2: go wrong. So this is not the hunky door it'll
• 11:55
  Speaker 2: solve all problems in the world. There are some things
• 11:57
  Speaker 2: that we might want to consider and here here's what
• 11:59
  Speaker 2: those things are are.
• 12:00
  Speaker 3: Yeah, one of them is a thing called scooping. This
• 12:03
  Speaker 3: refers to when someone comes in and they will see
• 12:05
  Speaker 3: a pre registered study an idea, and what they'll do
• 12:08
  Speaker 3: is they will steal a carefully crafted research idea and
• 12:12
  Speaker 3: maybe go do the experiment and publish the results before
• 12:15
  Speaker 3: the preregistered.
• 12:16
  Speaker 4: Event the study comes out. So that is something that happens.
• 12:19
  Speaker 3: But according to the websites we found, there have been
• 12:21
  Speaker 3: zero instances where that has been the case. But that
• 12:24
  Speaker 3: is a criticism or a concern that people will note
• 12:27
  Speaker 3: about why they don't preregister.
• 12:28
  Speaker 2: Another one is this it really does foster restrict adherence
• 12:32
  Speaker 2: to research design, which could be a good thing in
• 12:34
  Speaker 2: some cases, but some researchers express concern with the inflexibility
• 12:38
  Speaker 2: of the design and how that may prevent further additional
• 12:42
  Speaker 2: sort of findings. Yeah, and actually I was thinking about
• 12:44
  Speaker 2: this specifically with preregistering. Is like, let's just say you're
• 12:47
  Speaker 2: doing a study and you just happen to discover inadvertently
• 12:51
  Speaker 2: something totally outside of what you were expecting, Like pursuing
• 12:55
  Speaker 2: that might be a really good thing, Like that could
• 12:57
  Speaker 2: yield to something way more important than you thought you're
• 13:00
  Speaker 2: onto before. And if you've fully bought into it, are
• 13:02
  Speaker 2: committed to this one thing that you're pursuing in your design,
• 13:05
  Speaker 2: you might miss an opportunity to capture that and then
• 13:09
  Speaker 2: you use it as a jumping off point for some
• 13:11
  Speaker 2: really fruitful lines of research. So there there could be
• 13:14
  Speaker 2: some rigidity that does not foster the kind of exploratory
• 13:18
  Speaker 2: discovery that is I think endemic to what we want
• 13:20
  Speaker 2: in a scientific method. Now, I don't know that it's
• 13:23
  Speaker 2: such a problem that we think it's not worth doing
• 13:26
  Speaker 2: the pre registering. I would still say pre registering is
• 13:28
  Speaker 2: generally worth it. Yeah, and I think what would be
• 13:30
  Speaker 2: best is for us to continue moving forward with the
• 13:32
  Speaker 2: framework of how do we avoid this overly strict adherence
• 13:36
  Speaker 2: to protocol and like allow for the space of discovery
• 13:40
  Speaker 2: while also benefiting from pre registering and all the positives
• 13:45
  Speaker 2: that come from it.
• 13:46
  Speaker 3: You're absolutely right, and I think that goes into the
• 13:48
  Speaker 3: next point too that people are concerned about is like,
• 13:50
  Speaker 3: typically preregistration is only used for confirmatory studies, which means
• 13:53
  Speaker 3: that it limits its use in exploratory studies in some way.
• 13:57
  Speaker 3: This is at least a criticism or a concern that
• 13:58
  Speaker 3: people have. I think from the perspective of preregistering, if
• 14:01
  Speaker 3: you're going to preregister a hypothesis, it's really tough because
• 14:05
  Speaker 3: when you're doing an exploratory study, like let's say you're
• 14:08
  Speaker 3: doing a phenomenological study and you are kind of exploring
• 14:11
  Speaker 3: a potential experience or you know, the lived experience of
• 14:15
  Speaker 3: somebody like through some type of interview or something, you
• 14:17
  Speaker 3: may not be able to possibly or anywhere accurately predict
• 14:22
  Speaker 3: what that experience looks like. So it can be really
• 14:25
  Speaker 3: tough to pre register the at least the hypothesis parts
• 14:28
  Speaker 3: of that study, but you may preregister the survey or
• 14:31
  Speaker 3: the interview methods that you're using something along those lines.
• 14:35
  Speaker 3: So there are still options and specializations that you could
• 14:37
  Speaker 3: look at for pre registering those elements.
• 14:39
  Speaker 2: Fair some also have a fear of criticism. They're they're
• 14:43
  Speaker 2: report anxiety or stress related to peer review or criticisms.
• 14:46
  Speaker 2: I would kind of respond to that by saying, like,
• 14:48
  Speaker 2: this might not be the right field of work for you,
• 14:51
  Speaker 2: because that's kind of like, sure, what science is is
• 14:55
  Speaker 2: like we're just going to tear each other's work apart,
• 14:57
  Speaker 2: and just we got to be open to the fact that, like,
• 15:00
  Speaker 2: while there are I think dogmatic interpretations that get in
• 15:02
  Speaker 2: the way of like good scientific scrutiny of people's work,
• 15:06
  Speaker 2: I would say generally speaking, most people are really trying
• 15:09
  Speaker 2: to support one another in a scientific space.
• 15:12
  Speaker 4: Right.
• 15:12
  Speaker 2: My experiences with publishing and peer review have been exceedingly positive.
• 15:16
  Speaker 2: Even when people didn't necessarily care for what I had done,
• 15:19
  Speaker 2: I still got a lot of value from their feedback.
• 15:21
  Speaker 4: Yeah, same, absolutely, that would be I.
• 15:23
  Speaker 2: Think how I respond to that is like I could
• 15:25
  Speaker 2: see that being a little bit of a concern, and
• 15:26
  Speaker 2: also like, if that's a concern you have maybe try
• 15:29
  Speaker 2: doing something that you can not expose to that level
• 15:32
  Speaker 2: of criticism.
• 15:32
  Speaker 4: Yeah, yeah, consider something different that's okay.
• 15:34
  Speaker 3: And then also time constraints people complain about not complain
• 15:38
  Speaker 3: they are concerned about how much time it takes a
• 15:40
  Speaker 3: preregister but yet at the same time, like it speeds
• 15:43
  Speaker 3: up the peer review process and allows you to plan
• 15:46
  Speaker 3: your designs better, I think true, and be really clear
• 15:49
  Speaker 3: about what your designs are, be more thoughtful, get feedback
• 15:51
  Speaker 3: ahead of time before you waste time doing a study,
• 15:53
  Speaker 3: and then eventually get that feedback later that you've done
• 15:55
  Speaker 3: a study and that it was poorly designed and that
• 15:58
  Speaker 3: nobody will publish your results because you've alread done the
• 16:00
  Speaker 3: work and it wasn't very good. So like I think
• 16:01
  Speaker 3: that this is like it's well, the juice is worth
• 16:04
  Speaker 3: the squeeze when it comes to I think.
• 16:06
  Speaker 2: One of the things is like the time. You also
• 16:09
  Speaker 2: might say of like if you start going down the
• 16:10
  Speaker 2: path of your study having had no one get eyes
• 16:12
  Speaker 2: on your method or anything, and then you have to
• 16:14
  Speaker 2: start over because you didn't like foresee a mistake, right,
• 16:17
  Speaker 2: So like having preregistered your study, gone through all the
• 16:19
  Speaker 2: work and rigorm role that that might entail and granted
• 16:22
  Speaker 2: that might be a lot, but you also then might
• 16:25
  Speaker 2: have gotten some feedback from people that help you then
• 16:27
  Speaker 2: actually avoid unforeseen issues in your design or methodology. So
• 16:33
  Speaker 2: again I think, as you said, the juice is worth
• 16:34
  Speaker 2: the squeeze. Yeah, speaking of squeezing juice, we have run long,
• 16:38
  Speaker 2: so let's go ahead and wrap up this discussion on
• 16:40
  Speaker 2: minis and get you all back to your day. Thank
• 16:42
  Speaker 2: you so much for listening to my quick credit things.
• 16:46
  Speaker 2: If you'd like to support us, as I mentioned, you
• 16:47
  Speaker 2: can join us on Patreon. If you do that, you
• 16:49
  Speaker 2: get behind the scenes content, early content, and all kinds
• 16:51
  Speaker 2: of things over there like subscribe, liber rating, and review.
• 16:54
  Speaker 2: If you would like to tell us your thoughts or
• 16:56
  Speaker 2: your experiences with preregistering, or studies or anything else unrelated
• 16:59
  Speaker 2: to that, you can email us directly info at wwdwwdpodcast
• 17:04
  Speaker 2: dot com or reach out to us on the social
• 17:06
  Speaker 2: media platforms, and we certainly look forward to hearing from
• 17:08
  Speaker 2: you there.
• 17:09
  Speaker 4: Yeah, we love it.
• 17:10
  Speaker 2: If you do join us on patroon, I'll read your
• 17:11
  Speaker 2: name on our full length episodes that we publish on Wednesdays.
• 17:14
  Speaker 2: Otherwise I would say thank you so much to our
• 17:15
  Speaker 2: patron supporters. Thank you for listening, Thank you for recording
• 17:18
  Speaker 2: with me today, Shane, and to everyone on my team
• 17:20
  Speaker 2: for all the work that they do. Is there anything
• 17:22
  Speaker 2: you would like to add or that I forgot before
• 17:24
  Speaker 2: we say our goodbyes. Do good science, do good science? Beautiful?
• 17:29
  Speaker 4: All right?
• 17:29
  Speaker 2: Well said, then, thank you for listening. This is Abraham
• 17:32
  Speaker 2: and this is Shane. Why We Do what we Do?
• 17:33
  Speaker 2: Mini is out i E. You've been listening to Why
• 17:37
  Speaker 2: We Do What We Do.
• 17:38
  Speaker 1: You can learn more about this and other episodes by
• 17:41
  Speaker 1: going to wwdwwdpodcast dot com.
• 17:45
  Speaker 3: Thanks for listening, and we hope you have an awesome day.