Latest / Azadeh and Guy on AI / The Smart Lit Review

Transcript

The Smart Lit Review · Azadeh and Guy on AI

• 0:00
  SPEAKER_00: Hello everyone, hope you are having a great week.
• 0:03
  My name is Azadeh and I'm recording the third episode of Azadeh
• 0:08
  and Guy on AI with
• 0:10
  SPEAKER_00: my co-host Dr.
• 0:11
  SPEAKER_00: Guy Training.
• 0:12
  SPEAKER_01: Hi.
• 0:14
  It's a phenomenal week and I'm looking forward to all the AI
• 0:18
  things that are going to happen.
• 0:20
  SPEAKER_01: Sure.
• 0:21
  SPEAKER_01: Because every week there's something that happens in AI.
• 0:25
  And if nothing happens that is new,
• 0:27
  just finding new ways to use the AI we have.
• 0:30
  SPEAKER_00: Sure.
• 0:31
  SPEAKER_00: Exploring new features and new options.
• 0:33
  SPEAKER_00: Yeah.
• 0:35
  SPEAKER_00: You're always learning with AI.
• 0:36
  SPEAKER_00: Yeah.
• 0:37
  SPEAKER_01: Yeah.
• 0:38
  SPEAKER_01: So what are we going to do today?
• 0:39
  SPEAKER_00: I think we are going to start with exploring two AI tools.
• 0:45
  SPEAKER_00: One of them is free.
• 0:46
  SPEAKER_00: The other one is not.
• 0:47
  SPEAKER_00: It has a free version, but you need to pay for more options.
• 0:52
  SPEAKER_00: The other one that I'm introducing is a free tool.
• 0:56
  And next,
• 0:58
  we are going to talk about a research paper that we read
• 1:02
  recently.
• 1:03
  SPEAKER_00: Yeah.
• 1:03
  SPEAKER_01: Also about that.
• 1:04
  SPEAKER_01: So yes, we are focused on research with AI.
• 1:09
  So we think about the work we do at the university and the work
• 1:12
  we do as graduate
• 1:13
  SPEAKER_01: students,
• 1:14
  SPEAKER_01: as researchers,
• 1:16
  SPEAKER_01: as scholars,
• 1:17
  and see how that fits into this rise of AI and what are the
• 1:23
  tools that can make our
• 1:24
  job easier and what are the boundaries where we stop feeling
• 1:27
  comfortable about what
• 1:29
  SPEAKER_01: to use and how to use.
• 1:30
  SPEAKER_01: So do you want to share first?
• 1:32
  SPEAKER_00: I can do it.
• 1:33
  SPEAKER_00: Yeah.
• 1:33
  SPEAKER_00: All right.
• 1:34
  SPEAKER_01: So let's hear about what you have.
• 1:36
  So I think the tool that I'm going to explore and explain about
• 1:42
  actually is very
• 1:43
  SPEAKER_00: related to what most doctoral students do.
• 1:46
  SPEAKER_00: And it is literature review.
• 1:49
  SPEAKER_00: For some students, it's the most boring part of the research.
• 1:53
  SPEAKER_00: I like it.
• 1:54
  SPEAKER_00: I think it's the most fundamental part.
• 1:56
  And when you explore and find everything about the topic that
• 2:00
  you are writing
• 2:01
  SPEAKER_00: about,
• 2:01
  SPEAKER_00: it gives you more power.
• 2:02
  SPEAKER_00: It gives you better language, I think.
• 2:05
  SPEAKER_00: So one of the tools that I recently found is Asta.
• 2:12
  SPEAKER_00: It's spelled as A-S-T-A.
• 2:15
  SPEAKER_00: And it's a platform.
• 2:17
  SPEAKER_00: I explored the platform a little bit.
• 2:19
  SPEAKER_00: I tried different prompts to see how it brings up paper that I want.
• 2:26
  SPEAKER_00: And I have some points,
• 2:27
  SPEAKER_00: some positive points,
• 2:30
  SPEAKER_00: some areas for improvement,
• 2:32
  SPEAKER_00: some features that I was thinking about.
• 2:34
  SPEAKER_00: Do I like that kind of thing or not?
• 2:37
  SPEAKER_00: And something that I was not sure about.
• 2:41
  So one of the things that I really liked about ASTA was that
• 2:46
  when I logged in,
• 2:48
  SPEAKER_00: it gives me a very short introduction in a two-sentence format.
• 2:55
  It tells me that ASTA is a scholarly research assistant with
• 3:00
  broad and deep
• 3:01
  coverage via a corpus of more than 1 million scholarly
• 3:06
  abstracts and 12 million
• 3:08
  SPEAKER_00: full-text papers.
• 3:10
  SPEAKER_00: And it says ASTA can make mistakes,
• 3:13
  SPEAKER_00: check source documents by following citations,
• 3:16
  SPEAKER_00: and it also talks about the company that supports it,
• 3:21
  SPEAKER_00: and it's a project from AI2.
• 3:24
  SPEAKER_01: So it is to support literature search,
• 3:28
  and it's connected and potentially trained mostly on those
• 3:32
  scholarly articles.
• 3:34
  SPEAKER_00: Yeah, exactly.
• 3:35
  SPEAKER_00: Yeah, that's true.
• 3:36
  SPEAKER_00: And the other thing that I liked about it,
• 3:38
  SPEAKER_00: I mean,
• 3:39
  SPEAKER_00: about the welcome page,
• 3:40
  SPEAKER_00: if I can say that,
• 3:42
  SPEAKER_00: is that they give you some samples of the prompts.
• 3:46
  SPEAKER_00: And I think that's a very good starting point,
• 3:49
  especially if you are an undergrad student and you do not have
• 3:55
  enough knowledge or
• 3:56
  SPEAKER_00: you do not have enough experience for doing the search.
• 4:01
  SPEAKER_00: It could give you just a starting point.
• 4:06
  SPEAKER_00: So I like that thing about them.
• 4:08
  SPEAKER_00: They had sample prompts on their first page.
• 4:10
  SPEAKER_00: You can easily use any of them to start.
• 4:13
  SPEAKER_00: And it also serves as a super mini tutorial,
• 4:17
  SPEAKER_00: which is very useful for novice researchers,
• 4:19
  SPEAKER_00: as I said.
• 4:22
  SPEAKER_01: So have you used it?
• 4:25
  SPEAKER_00: I used it.
• 4:25
  SPEAKER_00: I used it, yeah.
• 4:26
  SPEAKER_01: And what did you learn when you were using it?
• 4:29
  SPEAKER_00: I entered a prompt.
• 4:31
  SPEAKER_00: I'm doing a literature review for one of my projects.
• 4:34
  SPEAKER_00: I just entered a very simple prompt.
• 4:37
  I said,
• 4:37
  find peer-reviewed research papers on AI-generated texts for
• 4:41
  learners.
• 4:41
  SPEAKER_00: I started with something very simple.
• 4:44
  SPEAKER_00: And I wanted to see how it works.
• 4:46
  SPEAKER_00: So the first thing that it did was very similar to...
• 4:51
  what chat gpt does uh it gave me a kind of uh thinking process
• 4:59
  or i think also
• 5:00
  gemini does the same thing they show you their thinking process
• 5:05
  so it said i'm
• 5:07
  going to look for a set of papers these are the criteria and
• 5:10
  the thing that i
• 5:11
  SPEAKER_00: really liked about it is that it kind of
• 5:14
  broke down my prompt and then offered a definition for
• 5:21
  different parts of the prompt.
• 5:23
  SPEAKER_00: So it says, AI generated text.
• 5:25
  The paper discusses methods or techniques for generating text
• 5:29
  using artificial intelligence.
• 5:31
  SPEAKER_00: I think that information just gives you some ideas
• 5:35
  And you can stop the search if you feel that it's not getting
• 5:39
  the right thing.
• 5:41
  SPEAKER_00: And you can go back and correct your prompt.
• 5:43
  SPEAKER_00: Or you can say, no, this is what I mean by that phrase.
• 5:49
  So it gives you a kind of a space for more corrections and
• 5:52
  improvement on your
• 5:53
  SPEAKER_00: prompt,
• 5:54
  SPEAKER_00: which is very important when you interact with any kind of AI tool.
• 5:59
  And then I got a message when it was done with the search and
• 6:03
  everything.
• 6:04
  It said I found 32 papers that look like perfect matches,
• 6:08
  32 relevant ones,
• 6:10
  and 11 others.
• 6:11
  SPEAKER_00: I also like that kind of categorization.
• 6:14
  SPEAKER_01: Mm-hmm.
• 6:17
  SPEAKER_00: And then it came up with all of those papers, the list of the papers.
• 6:24
  The thing that I didn't really like about the results was the
• 6:29
  color palette of the AI.
• 6:32
  SPEAKER_00: So I showed you how it looks.
• 6:34
  SPEAKER_00: And they use different shades of green.
• 6:37
  SPEAKER_00: That makes you a little bit confused when you want to go over that.
• 6:41
  SPEAKER_00: And you know that our minds or our brain is always looking for patterns.
• 6:47
  SPEAKER_00: So it's kind of hard for you when you look at all of those results.
• 6:51
  SPEAKER_00: What is this color green for?
• 6:53
  SPEAKER_00: How can I, you know?
• 6:54
  SPEAKER_01: Yes.
• 6:55
  SPEAKER_00: Kind of makes it hard to navigate through all of those search results.
• 6:59
  The things that I really liked about the search result was all
• 7:04
  of the information
• 7:05
  SPEAKER_00: that they are giving me.
• 7:06
  SPEAKER_00: Like...
• 7:08
  SPEAKER_00: They are giving me the name of the author, the publication venue.
• 7:13
  SPEAKER_00: They gave me the year.
• 7:14
  SPEAKER_00: They had some excerpts from the paper.
• 7:18
  SPEAKER_00: They are not paraphrased.
• 7:19
  SPEAKER_00: So if you are going to use them, you need to paraphrase them yourself.
• 7:22
  SPEAKER_00: But this is still important.
• 7:24
  SPEAKER_00: And they give you the abstract.
• 7:27
  SPEAKER_00: I told you about that.
• 7:29
  SPEAKER_01: They have a kind... So when you kind of look at ASTA as a whole...
• 7:38
  SPEAKER_01: What is your summary?
• 7:39
  SPEAKER_01: What is your advice for students who are looking for,
• 7:42
  SPEAKER_01: or students are actually researchers,
• 7:44
  SPEAKER_01: who are looking for a tool?
• 7:46
  SPEAKER_01: What's your summary?
• 7:48
  I think it would be a great tool if you are okay with all
• 7:53
  shades of green.
• 7:58
  SPEAKER_00: I really like their filtering options.
• 8:00
  I think it gives you a lot of features in terms of going
• 8:04
  through and sifting
• 8:05
  SPEAKER_00: through all of the papers that it pulls up.
• 8:07
  SPEAKER_00: And you can easily search everything by year, by author.
• 8:10
  SPEAKER_00: I find it useful.
• 8:12
  SPEAKER_00: I don't know.
• 8:14
  SPEAKER_00: I really liked it if it...
• 8:16
  SPEAKER_00: If it allowed me to do a kind of snowball search,
• 8:20
  SPEAKER_00: I mean,
• 8:20
  going through the references for each paper and pulling up all
• 8:24
  of the useful things
• 8:25
  SPEAKER_00: from there.
• 8:26
  SPEAKER_00: But I didn't explore it enough.
• 8:28
  Probably if I try new prompts,
• 8:31
  I can come up with new ideas for next episode.
• 8:35
  SPEAKER_00: All right.
• 8:35
  SPEAKER_00: Yeah, yeah.
• 8:36
  SPEAKER_01: So we might revisit these as we get more fluid with them.
• 8:40
  SPEAKER_01: Yeah.
• 8:41
  SPEAKER_01: Mine...
• 8:43
  SPEAKER_01: is SciSpace, and it does something very similar.
• 8:45
  So if you're conducting research...
• 8:47
  Your favorite A...
• 8:48
  My favorite thing,
• 8:50
  yes.
• 8:50
  SPEAKER_01: I've been talking to everybody about SciSpace.
• 8:53
  SPEAKER_00: I know.
• 8:53
  SPEAKER_01: Because I have tried a few.
• 8:55
  SPEAKER_01: I have not tried Asta, but I have tried a few others, which I was...
• 9:01
  SPEAKER_00: You are using the paid version, right?
• 9:03
  SPEAKER_01: Yes,
• 9:04
  SPEAKER_01: I use the paid version,
• 9:05
  but right now I'm going to talk about the free version because
• 9:09
  it is a freemium
• 9:10
  SPEAKER_01: model.
• 9:10
  SPEAKER_01: And so you get a lot of access, at least right now.
• 9:15
  SPEAKER_01: My guess is sooner or later they will charge for it.
• 9:18
  But right now you're getting a tremendous amount of capacity,
• 9:21
  even without paying.
• 9:23
  Do you have any idea about the number of papers that they pull
• 9:28
  their data from or their corpus?
• 9:31
  I didn't see anything that talked about that specifically when
• 9:36
  I looked at their data.
• 9:37
  SPEAKER_01: So I don't know.
• 9:38
  SPEAKER_01: But I do know that they go to Google Scholar.
• 9:40
  SPEAKER_01: Mm-hmm.
• 9:41
  I do know that they're sifting through a number of other
• 9:45
  databases to get everything.
• 9:49
  And what I love about them is that the window is not in green,
• 9:55
  so that would
• 9:56
  make you happy oh there's very little green in there but you
• 9:59
  can do fairly
• 10:00
  sophisticated searches so for example i did a search for the
• 10:04
  development literature
• 10:06
  on how gasky's idea about professional development intersect
• 10:09
  with literature about
• 10:10
  professional development for professional learning in arts
• 10:12
  education and this is
• 10:13
  something i recommend to anybody who's looking at this kind of
• 10:18
  software and that is
• 10:20
  SPEAKER_01: you start where you know a lot yeah
• 10:24
  SPEAKER_01: because that allows you to know if it's doing the things you want to do.
• 10:29
  SPEAKER_01: Because when you're going to something new,
• 10:30
  SPEAKER_01: and we talked about this when we talked about the novice problem,
• 10:33
  SPEAKER_01: right?
• 10:33
  SPEAKER_00: Yeah.
• 10:34
  SPEAKER_01: When you're the novice, you don't know anything.
• 10:38
  SPEAKER_01: Therefore,
• 10:38
  SPEAKER_01: you don't know if it's doing a good job or a bad job,
• 10:41
  SPEAKER_01: especially at the point where you're trying things out.
• 10:43
  SPEAKER_01: You want to go to a place where you have expertise and say,
• 10:46
  SPEAKER_01: is this similar to what I would have wanted?
• 10:49
  SPEAKER_01: Because if the answer is no, it's probably not a great tool.
• 10:51
  SPEAKER_00: Yeah, that's very interesting.
• 10:53
  SPEAKER_00: Actually, the same idea.
• 10:54
  SPEAKER_00: We will talk about that.
• 10:55
  SPEAKER_00: We will discuss that idea in the paper that we are going to talk about.
• 10:59
  SPEAKER_00: Yeah, they talked about the same thing.
• 11:02
  SPEAKER_00: Like these tools are for people that have the basic knowledge,
• 11:06
  SPEAKER_00: the fundamental knowledge to do the search and do the work,
• 11:09
  SPEAKER_00: yeah.
• 11:09
  SPEAKER_01: Yeah, and to play with it.
• 11:11
  SPEAKER_01: What do I love about SciSpace?
• 11:13
  It seems to be capturing the things I want you to capture when
• 11:16
  I know a lot.
• 11:18
  It gives you immediately access to abstract,
• 11:21
  a PDF,
• 11:22
  and the DOI so you can find it online.
• 11:24
  SPEAKER_01: All of those links are there.
• 11:26
  SPEAKER_01: I love the fact that when they summarize something,
• 11:29
  SPEAKER_01: they actually make a few moves that are great.
• 11:33
  SPEAKER_01: They start with the TLDR, and so they give you just here's the top line.
• 11:39
  SPEAKER_01: This is where we're at.
• 11:41
  SPEAKER_00: I have a question, Guy.
• 11:43
  SPEAKER_00: How big is their summary of the paper?
• 11:46
  SPEAKER_00: Because for ASTA,
• 11:48
  SPEAKER_00: for most of the papers,
• 11:49
  SPEAKER_00: as I'm scrolling through my list,
• 11:51
  SPEAKER_00: is a two-sentence summary.
• 11:53
  SPEAKER_01: So there in the main window in SciSpace, you don't get the papers.
• 11:59
  SPEAKER_01: What you get is a summary of the field first.
• 12:03
  SPEAKER_01: So this is what we saw as we summarize across articles.
• 12:07
  SPEAKER_01: That's what you see on the main page now.
• 12:10
  SPEAKER_01: on the right hand in the activity,
• 12:13
  you can see where it's pulling from and you can say what do you
• 12:18
  want to see.
• 12:19
  So for example,
• 12:20
  it'll create a table based on what you say you want to see.
• 12:25
  SPEAKER_01: It can create a table just looking at the column for conclusions and
• 12:30
  SPEAKER_01: It can create just a column with the abstract.
• 12:32
  SPEAKER_01: It can separate out results or methods.
• 12:36
  I love looking at methods because it tells me it's a different
• 12:40
  way to look at the field.
• 12:41
  SPEAKER_01: Is this mostly a qualitative set of papers or is this more qualitative?
• 12:45
  SPEAKER_01: quantitative or is this a good mix of both?
• 12:48
  So this is a really useful way where you can tailor to the
• 12:54
  needs that you have as
• 12:55
  SPEAKER_01: you're searching.
• 12:56
  If I'm just looking for abstracts because I just want to get a
• 13:00
  sense,
• 13:00
  that's one way.
• 13:01
  SPEAKER_01: But if I want to go and dig much,
• 13:03
  SPEAKER_01: much deeper and look at methodology as I'm entering a field and I want to
• 13:07
  SPEAKER_01: understand how people are investigating this field,
• 13:10
  SPEAKER_01: then
• 13:11
  SPEAKER_01: That flexibility in how you can do things is great.
• 13:15
  SPEAKER_01: And you mentioned filters.
• 13:17
  You can filter by whether there's a PDF,
• 13:20
  whether it's an open access paper.
• 13:22
  SPEAKER_01: Really important if you don't have access.
• 13:24
  SPEAKER_01: We have access to university library.
• 13:26
  SPEAKER_01: We can get access to almost anything digitally.
• 13:29
  SPEAKER_01: And if not, we can ask for it.
• 13:30
  SPEAKER_01: Yeah.
• 13:30
  SPEAKER_01: But you can filter and say only open access.
• 13:34
  SPEAKER_01: You can say, I just want top tier papers.
• 13:36
  SPEAKER_01: You can filter by year.
• 13:38
  SPEAKER_01: So there are lots of ways and publication types.
• 13:41
  SPEAKER_01: So you can say,
• 13:42
  SPEAKER_01: no,
• 13:43
  SPEAKER_01: I want just journal article or just chapter or books or everything,
• 13:47
  SPEAKER_01: patents,
• 13:47
  SPEAKER_01: proceedings.
• 13:48
  SPEAKER_01: So you can,
• 13:49
  SPEAKER_01: again,
• 13:50
  SPEAKER_01: if you have preferences,
• 13:52
  SPEAKER_01: really think about what is it that I'm trying to do?
• 13:55
  SPEAKER_01: What is it that I want?
• 13:57
  So,
• 13:57
  for example,
• 13:58
  as we're writing a meta-analysis,
• 14:00
  we want specific,
• 14:03
  right,
• 14:04
  specific sources.
• 14:05
  SPEAKER_01: So this would do that.
• 14:07
  SPEAKER_01: Yeah.
• 14:10
  SPEAKER_00: Yeah.
• 14:11
  SPEAKER_01: And that's a way for us to think.
• 14:13
  SPEAKER_01: So I love SciSpace.
• 14:14
  SPEAKER_01: I think it is a great tool.
• 14:17
  But again,
• 14:17
  I think just like you said,
• 14:19
  the main thing to do is try it out.
• 14:22
  SPEAKER_01: Use something you know.
• 14:24
  SPEAKER_01: Try it out and say, is it...
• 14:27
  SPEAKER_01: Accurate enough for me to use?
• 14:29
  SPEAKER_01: Is it a good starting point?
• 14:30
  I don't think,
• 14:31
  as you pointed out,
• 14:32
  it's not the final product,
• 14:34
  but it is a great first step.
• 14:37
  SPEAKER_01: It is making it so instead of spending hours,
• 14:41
  SPEAKER_01: I mean,
• 14:41
  SPEAKER_01: I remember even in the digital age,
• 14:44
  we're still spending sometimes hours on finding just the right
• 14:46
  article on Google
• 14:48
  SPEAKER_01: Scholar.
• 14:48
  SPEAKER_01: Yeah.
• 14:50
  and it's getting harder and harder because there are lots of
• 14:52
  publications.
• 14:53
  SPEAKER_01: This allows you to have that access super quickly,
• 14:56
  SPEAKER_01: and you get potentially – it starts with the top 10,
• 14:59
  SPEAKER_01: but it gives you 100 articles,
• 15:01
  SPEAKER_01: and you can get beyond that.
• 15:03
  SPEAKER_01: In this search that I did, there are over 250 papers, for example.
• 15:10
  So I have a quick question,
• 15:11
  Guy,
• 15:11
  before we summarize on Science Space and both of these tools.
• 15:15
  SPEAKER_00: It wasn't in your prompt,
• 15:17
  and you didn't ask it to categorize all of those papers based
• 15:21
  on their research
• 15:21
  SPEAKER_00: methodology.
• 15:22
  SPEAKER_00: It did it by itself, right?
• 15:24
  SPEAKER_01: You can add it on.
• 15:26
  SPEAKER_01: So I did not,
• 15:27
  SPEAKER_01: but there's columns,
• 15:29
  SPEAKER_01: so you can add a column to their results page,
• 15:33
  SPEAKER_01: and it says you can create your own column,
• 15:36
  SPEAKER_01: so you can actually invent your own take.
• 15:38
  SPEAKER_00: I love that.
• 15:39
  SPEAKER_01: You know, what kind of art, for example.
• 15:41
  SPEAKER_00: So you can customize the AI's output based on your… Completely.
• 15:46
  SPEAKER_01: So methods and results are in the suggestions of what you can look at.
• 15:52
  SPEAKER_00: Even in your free version, you can still customize those columns.
• 15:55
  SPEAKER_00: That's very helpful.
• 15:56
  SPEAKER_00: Okay.
• 15:56
  SPEAKER_00: Okay.
• 15:56
  SPEAKER_01: And so there's a tremendous amount of utility there.
• 16:01
  And it gives you these things that Notebook LM does,
• 16:06
  which is you can have a conversation.
• 16:07
  SPEAKER_01: You can chat with the article.
• 16:09
  SPEAKER_01: So you have basically an augmented input from a specific article.
• 16:15
  SPEAKER_01: So you can do a little bit more IAMs.
• 16:21
  SPEAKER_01: deeply in love with this piece of software.
• 16:24
  I'm also a little bit worried because I am worried always that
• 16:27
  these will make us a
• 16:29
  SPEAKER_01: little bit too lazy.
• 16:30
  There's something about the searching and the happenstance that
• 16:33
  happen as you look around.
• 16:35
  SPEAKER_01: But as a tool,
• 16:37
  SPEAKER_01: when we're super busy and we need to get,
• 16:40
  especially if we need to get into a new field and we just need
• 16:43
  to get the lay of
• 16:44
  SPEAKER_01: the land,
• 16:44
  SPEAKER_01: this is a great opening move.
• 16:47
  SPEAKER_00: Yeah.
• 16:47
  SPEAKER_00: So, Guy, I have my last question.
• 16:49
  SPEAKER_00: I hope my last question.
• 16:52
  SPEAKER_00: So compared to you, I have zero experience in research.
• 16:55
  But as a super experienced scholar,
• 16:57
  what stands out when you are working with those AI tools?
• 17:03
  SPEAKER_00: What is the most prominent feature in research?
• 17:05
  The most prominent feature is the ability to do really
• 17:12
  sophisticated searches that
• 17:16
  SPEAKER_01: are not relying just on the keywords.
• 17:19
  SPEAKER_01: Because traditionally,
• 17:20
  SPEAKER_01: we could always do Boolean searches of one kind or another,
• 17:24
  SPEAKER_01: but there was always the same problem,
• 17:26
  SPEAKER_01: and that is people are not very good at choosing keywords.
• 17:30
  And so you don't find what you're actually looking for,
• 17:33
  but you find,
• 17:34
  like,
• 17:35
  SPEAKER_01: 50 things that don't connect at all.
• 17:37
  SPEAKER_01: So whatever magic works within LLMs,
• 17:40
  it does allow a more sophisticated version of the search to
• 17:46
  really manifest a
• 17:48
  SPEAKER_01: series of articles.
• 17:49
  SPEAKER_01: We recently did a search.
• 17:51
  SPEAKER_01: We were looking for a specific framework.
• 17:54
  SPEAKER_01: We couldn't put our words to it.
• 17:56
  SPEAKER_01: But with the LLMs effort, we found something and it was very clear.
• 17:59
  SPEAKER_01: This was exactly what we were looking for.
• 18:02
  SPEAKER_01: So there's this moment of discovery,
• 18:04
  SPEAKER_01: and the advantage here is it is happening very quickly,
• 18:08
  which means we're spending – and this is the people who created
• 18:11
  SciSpace talk about
• 18:13
  SPEAKER_01: this.
• 18:13
  The idea for them was how do we minimize the amount of time
• 18:18
  you're doing the menial –
• 18:20
  less important things so you can have more time to do the
• 18:24
  important things.
• 18:25
  SPEAKER_01: And that is really where my brain is.
• 18:28
  SPEAKER_01: Can I save some time by doing it this way?
• 18:31
  SPEAKER_01: I still think we need to read the key articles.
• 18:33
  I still think we need to summarize them and have a good
• 18:36
  understanding and not just
• 18:38
  SPEAKER_01: rely on what the AI is doing,
• 18:40
  but the AI is opening a window where we can choose where we're
• 18:44
  entering and it's a
• 18:45
  SPEAKER_01: good choice.
• 18:46
  SPEAKER_00: So we are not skipping literature review.
• 18:48
  SPEAKER_00: We are improving the quality of our literature review using AI.
• 18:52
  SPEAKER_01: Yes.
• 18:53
  SPEAKER_01: And we're also admitting that we have more publications than ever,
• 18:58
  SPEAKER_01: more articles published every year.
• 19:00
  SPEAKER_01: There's just a tremendous number.
• 19:02
  SPEAKER_01: And we have stopped being able to read deeply in our area.
• 19:07
  SPEAKER_01: Mm-hmm.
• 19:08
  SPEAKER_01: So even if you take 10 or 20 hours a week, nobody's got that.
• 19:12
  SPEAKER_01: But let's say you are.
• 19:13
  SPEAKER_00: I think that's true for every discipline.
• 19:15
  SPEAKER_01: Yes.
• 19:16
  SPEAKER_01: It is universal.
• 19:17
  SPEAKER_01: There are more researchers than ever.
• 19:19
  SPEAKER_01: There are more publications than ever.
• 19:21
  SPEAKER_01: And the challenge is real.
• 19:23
  And so having tools that help you get to the right papers at
• 19:27
  the right time is
• 19:28
  SPEAKER_01: actually,
• 19:28
  SPEAKER_01: I think,
• 19:29
  SPEAKER_01: critically important.
• 19:30
  SPEAKER_00: Yeah, yeah, yeah.
• 19:31
  SPEAKER_00: I totally agree.
• 19:34
  So the next part that we are going to talk about is about a
• 19:37
  research paper that we
• 19:38
  SPEAKER_00: read recently,
• 19:39
  SPEAKER_00: and it's very related to what we just talked about.
• 19:44
  The title of the paper is Doctoral Students' Reflections on
• 19:48
  Generative Artificial
• 19:49
  SPEAKER_00: Intelligence,
• 19:50
  SPEAKER_00: Gen-AI Use in the Literature Review Process.
• 19:53
  SPEAKER_01: So exactly what we were talking about.
• 19:55
  SPEAKER_01: And this is by Kumar and Gunn.
• 19:58
  SPEAKER_01: And this was published in November 24, so not too long ago.
• 20:02
  SPEAKER_01: And you have to appreciate this because we've talked about this before.
• 20:07
  We live in an age where things are shifting quickly and we need
• 20:10
  to get to publication quickly.
• 20:12
  SPEAKER_01: And academic research is slow.
• 20:16
  SPEAKER_00: Yes.
• 20:17
  It's you send the paper and then there's peer review and they
• 20:20
  send it back if they
• 20:21
  SPEAKER_01: can find reviewers.
• 20:22
  SPEAKER_01: Because with this rise in the number of articles,
• 20:25
  SPEAKER_01: it is harder and harder to find reviewers and to find good reviewers.
• 20:30
  SPEAKER_01: So the process is slow.
• 20:31
  So anybody that can publish anything within a short cycle
• 20:35
  should be blessed.
• 20:36
  SPEAKER_00: And reviewers are not allowed to use AI.
• 20:39
  SPEAKER_00: Yes.
• 20:40
  SPEAKER_01: That is true, too.
• 20:41
  SPEAKER_01: And for a variety of reasons, maybe we'll talk about it another time.
• 20:44
  SPEAKER_01: But let's talk about this paper.
• 20:45
  SPEAKER_01: So what did they do?
• 20:47
  SPEAKER_00: That's a qualitative study.
• 20:48
  They explore doctoral students' perceptions of the value of Gen
• 20:52
  AI tools in the
• 20:54
  SPEAKER_00: literature review process.
• 20:56
  SPEAKER_00: And they had 26 participant reflections.
• 21:02
  They asked doctoral students to use different kinds of AI tools
• 21:07
  and then reflect on their use.
• 21:09
  SPEAKER_00: They had two research questions.
• 21:12
  In what ways can GenAI can be used in the lit review process by
• 21:16
  doctoral students?
• 21:17
  And what are doctoral students' perceptions of the value of
• 21:20
  GenAI to their
• 21:22
  SPEAKER_00: dissertation literature review?
• 21:24
  And what is important to say is the context includes the fact
• 21:28
  that these are
• 21:29
  SPEAKER_01: doctoral students in an online program.
• 21:32
  And I think the reason it's important that they're in an online
• 21:35
  program is because
• 21:36
  SPEAKER_01: when you're in an online program,
• 21:38
  your set of expectations is a little bit different and your
• 21:41
  time management is a
• 21:42
  SPEAKER_01: little bit different.
• 21:43
  But maybe the most important thing is you're already working in
• 21:47
  a digital environment.
• 21:49
  So the transition is a little bit easier,
• 21:51
  whereas if you're coming to lecture,
• 21:54
  SPEAKER_01: there's another layer there before you get to the technology.
• 21:58
  SPEAKER_00: Yeah, so students use different Gen.AI technologies.
• 22:02
  SPEAKER_00: One of them is LeetMaps.
• 22:03
  SPEAKER_00: We can also talk about LeetMaps.
• 22:05
  SPEAKER_00: I love that tool, to be honest.
• 22:07
  SPEAKER_01: We'll talk about it another time.
• 22:09
  SPEAKER_01: Promise for a future episode.
• 22:11
  SPEAKER_00: Scholar C, I never used it.
• 22:14
  SPEAKER_00: Sight, I used it a little bit.
• 22:16
  SPEAKER_00: Research Rabbit, I also used it.
• 22:18
  SPEAKER_00: Yeah, it's a free tool.
• 22:20
  SPEAKER_00: Okay.
• 22:22
  SPEAKER_00: Elicit and Perplexity AI, I also use Perplexity AI.
• 22:26
  SPEAKER_00: That's a good tool.
• 22:27
  SPEAKER_00: Yeah.
• 22:28
  SPEAKER_00: For our future episodes.
• 22:30
  SPEAKER_00: And also Listening.io.
• 22:32
  SPEAKER_00: Okay.
• 22:37
  SPEAKER_01: All right, so what did they find out?
• 22:41
  SPEAKER_00: So students had different ideas when they tried all of those tools.
• 22:45
  SPEAKER_00: For example,
• 22:47
  SPEAKER_00: when they tried ScholarC,
• 22:51
  it seems that it creates exportable flashcards with key
• 22:55
  information from the
• 22:56
  SPEAKER_00: literature,
• 22:56
  SPEAKER_00: and the students found it kind of helpful.
• 23:01
  SPEAKER_00: When they tried LitMaps,
• 23:04
  SPEAKER_00: They explain that lead maps find recurring authors and themes,
• 23:08
  SPEAKER_00: and it creates a kind of network.
• 23:10
  SPEAKER_00: You remember?
• 23:11
  SPEAKER_00: We tried it once together to see all of those maps.
• 23:15
  SPEAKER_01: I love the visual aspect of that, so I understand why.
• 23:18
  SPEAKER_00: They also call it seed maps.
• 23:20
  SPEAKER_00: They provide you with the numbers,
• 23:22
  SPEAKER_00: number of citations,
• 23:23
  SPEAKER_00: or how many times a specific paper has been used.
• 23:29
  SPEAKER_00: And talking about site,
• 23:32
  students found that it provided details on metrics and impact
• 23:36
  factor and focused on
• 23:38
  SPEAKER_00: articles that were most cited in the academic community.
• 23:41
  SPEAKER_00: So each of these tools, they touched specific aspects of it.
• 23:47
  SPEAKER_00: Literature.
• 23:48
  And so doctoral students found that Gen AI can help them
• 23:52
  discover seminal works,
• 23:54
  SPEAKER_00: make connections between researchers and articles,
• 23:57
  SPEAKER_00: conduct snowball searches,
• 23:59
  SPEAKER_00: and also determine key themes and gaps in the literature.
• 24:04
  SPEAKER_01: And most of them,
• 24:06
  SPEAKER_01: so 17 out of the total number,
• 24:09
  SPEAKER_01: which is 65%,
• 24:11
  SPEAKER_01: said they would see themselves using Gen.ai in the future.
• 24:16
  SPEAKER_01: The other 35% maybe were delusional.
• 24:19
  SPEAKER_00: No disrespect.
• 24:20
  SPEAKER_01: No disrespect.
• 24:22
  But as somebody who does a lot of research but also has limited
• 24:26
  time and people in
• 24:27
  SPEAKER_01: an online program are often working full time,
• 24:30
  SPEAKER_01: they have limited time.
• 24:31
  SPEAKER_01: My guess is they are.
• 24:32
  SPEAKER_01: Because what are they going to use if they're not going to use that?
• 24:34
  SPEAKER_01: They're going to use Google Scholar.
• 24:36
  SPEAKER_01: Google Scholar is also AI.
• 24:38
  SPEAKER_01: It's just not as good AI.
• 24:39
  SPEAKER_01: Yeah.
• 24:40
  SPEAKER_01: But it is still AI.
• 24:42
  You put a search term and it has an algorithm and then it puts
• 24:46
  the order of the
• 24:47
  SPEAKER_01: results in some order of usually citations.
• 24:51
  SPEAKER_01: And it's a combination of citations, relevance.
• 24:54
  SPEAKER_00: Relevance, date also.
• 24:56
  SPEAKER_00: You can sort it, but yeah.
• 24:58
  SPEAKER_01: But that's AI too.
• 25:01
  It's just they're not outwardly saying that and you can't have
• 25:04
  a conversation with
• 25:05
  SPEAKER_01: them,
• 25:06
  which makes it a product that you actually control less because
• 25:09
  your prompting
• 25:10
  SPEAKER_01: ability is not as good as AI.
• 25:12
  SPEAKER_01: But this is just me being a little bit oppositional about this.
• 25:16
  SPEAKER_01: Yeah.
• 25:18
  They were concerned about accuracy and they were concerned
• 25:20
  about privacy.
• 25:22
  SPEAKER_01: So those were two concerns that really rose to the top.
• 25:26
  SPEAKER_01: Privacy because they asked them directly,
• 25:29
  SPEAKER_01: you know,
• 25:29
  SPEAKER_01: some of these,
• 25:30
  SPEAKER_01: as we talked about,
• 25:31
  SPEAKER_01: some of these say we're not going to save your information.
• 25:33
  Some of them are saying we will save your searches without your
• 25:36
  name and without
• 25:37
  SPEAKER_01: connecting it to you because we're doing our own research.
• 25:40
  SPEAKER_01: That's at least honest.
• 25:41
  SPEAKER_00: Yeah,
• 25:41
  and sometimes you do not like to share your ideas with any AI
• 25:45
  tool when you are
• 25:46
  SPEAKER_00: doing this.
• 25:46
  SPEAKER_00: Yeah.
• 25:47
  SPEAKER_01: Yeah, that's the difference between us.
• 25:48
  SPEAKER_01: I was never original enough to worry about my ideas.
• 25:52
  SPEAKER_01: Like if anybody wants to take them and run with them.
• 25:55
  SPEAKER_01: May they be blessed.
• 25:58
  SPEAKER_01: But yes, you can worry about that.
• 26:00
  You can also worry about – I think a lot of people talk to me
• 26:04
  about worrying about
• 26:06
  SPEAKER_01: becoming lazy.
• 26:07
  SPEAKER_01: So the idea is not that it's a bad tool,
• 26:10
  SPEAKER_01: but what they're worried about is with time,
• 26:13
  SPEAKER_01: they will start trusting it more than they should,
• 26:16
  and they will not do the due diligence because they are all
• 26:20
  talking about the fact
• 26:21
  SPEAKER_01: that you need human review.
• 26:22
  SPEAKER_01: Yeah.
• 26:23
  You need the human in the process somewhere evaluating in a
• 26:27
  previous conversation.
• 26:28
  You said,
• 26:28
  you need to check that that's a real quote or that's a real
• 26:31
  paper.
• 26:32
  You cannot just assume that because you asked for it and it
• 26:36
  gave it to you that it's real.
• 26:38
  SPEAKER_01: So the following up on those links is an important skill.
• 26:42
  SPEAKER_01: And some people are just worried and they may be right about this.
• 26:46
  SPEAKER_01: that they would become complacent and stop checking.
• 26:50
  SPEAKER_01: And if you stop checking, you're still responsible for the quality.
• 26:54
  SPEAKER_01: You're still responsible for everything that's happening.
• 26:57
  So maybe they're creating a barrier so they would not fall into
• 27:01
  that bucket.
• 27:02
  SPEAKER_01: They want to do it themselves.
• 27:04
  SPEAKER_01: Again,
• 27:04
  SPEAKER_01: they're still using some kind of a Boolean search of one kind or another,
• 27:08
  SPEAKER_01: which is driven by some kind of an intelligence,
• 27:12
  SPEAKER_01: even if it's the limited intelligence of the people choosing keywords.
• 27:16
  SPEAKER_01: You're still doing it.
• 27:17
  SPEAKER_00: Yeah, yeah, yeah.
• 27:18
  SPEAKER_00: So maybe you can use all of those AI tools as a kind of...
• 27:22
  starting point to give you an outline of the or to just show
• 27:28
  you the important or
• 27:30
  most prominent papers that have been published in the area that
• 27:34
  you are looking for
• 27:35
  SPEAKER_00: and then you can do the yeah the bulk of the work by yourself but yeah
• 27:42
  And I go back to – and the paper does too – to one of your
• 27:46
  favorite things,
• 27:46
  and that is you need to have a basic understanding of the
• 27:49
  technology and the
• 27:52
  SPEAKER_01: digital literacy in this case,
• 27:54
  SPEAKER_01: the prompting literacy.
• 27:56
  SPEAKER_01: You need to know how to get the most out of this and how to use it well.
• 28:01
  SPEAKER_01: And again, this is the novice problem all over again.
• 28:06
  SPEAKER_01: The more novice you are, the less likely you are to get top –
• 28:10
  quality results so what you need to be ready is to get a result
• 28:15
  only after a few
• 28:16
  prompts you gotta refine your prompt and you gotta get closer
• 28:19
  to what you wanted
• 28:20
  and not just rely on i just tried it and it kind of worked and
• 28:24
  it's good enough
• 28:25
  SPEAKER_01: that is the danger
• 28:27
  SPEAKER_00: Yeah, yeah.
• 28:27
  SPEAKER_00: And I love two points about the student reflection.
• 28:30
  So all of them said that Gen AI tools,
• 28:34
  they are a supplement to library databases.
• 28:37
  SPEAKER_00: They are not a primary research tool.
• 28:39
  And the other thing that they mentioned is that prior research
• 28:43
  skills are necessary
• 28:44
  SPEAKER_00: and also a human review is necessary.
• 28:48
  SPEAKER_01: Yes.
• 28:48
  So I did want to say Kumar and Gan,
• 28:51
  thank you very much for doing the work.
• 28:54
  SPEAKER_01: I'm teaching a class that's doing something similar.
• 28:56
  It gives me some confidence that my students may be able to use
• 29:01
  generative AI as
• 29:02
  SPEAKER_01: part of the literature search.
• 29:06
  SPEAKER_01: Actually, we're teaching it together.
• 29:07
  SPEAKER_00: Yeah, yeah, that's true.
• 29:08
  SPEAKER_00: I'm sorry, I forgot to say that.
• 29:10
  SPEAKER_00: All right.
• 29:11
  SPEAKER_01: Take us out.
• 29:14
  SPEAKER_00: Okay.
• 29:15
  SPEAKER_00: So I think for next episode,
• 29:18
  we are going to explore more tools or we may try to surprise
• 29:24
  you by something
• 29:25
  SPEAKER_00: totally new.
• 29:26
  SPEAKER_01: All right.
• 29:27
  SPEAKER_01: And just remember,
• 29:28
  SPEAKER_01: you can leave us a note,
• 29:30
  SPEAKER_01: you can send us an email,
• 29:31
  SPEAKER_01: you can communicate back to us.
• 29:33
  SPEAKER_01: We'd love to hear how you're using AI in your research process.
• 29:37
  SPEAKER_00: We love feedback.
• 29:38
  SPEAKER_00: Yeah.
• 29:39
  SPEAKER_01: So goodbye from...
• 29:41
  SPEAKER_00: As I did.
• 29:41
  SPEAKER_01: And guy.
• 29:42
  SPEAKER_00: On AI.