Latest / Azadeh and Guy on AI / Beyond the Answer Key

Transcript

Beyond the Answer Key · Azadeh and Guy on AI

• 0:01
  SPEAKER_01: Hello, everyone. Welcome to another episode of Azadeh and Guy on AI.
• 0:05
  SPEAKER_01: Today, unfortunately, Dr.
• 0:07
  SPEAKER_01: Guy Trainin is not with us, so I'm recording this episode on behalf of two of us.
• 0:14
  SPEAKER_01: We are very excited to have Dr.
• 0:17
  SPEAKER_01: Shuparna Chatterjee from New Mexico State University for this episode.
• 0:21
  SPEAKER_01: I'm going to briefly introduce her and also her research and then we are going
• 0:26
  SPEAKER_01: to start asking our questions.
• 0:29
  SPEAKER_01: So Dr. Shuparna Chatterjee is an assistant professor in the Department of Curriculum
• 0:35
  SPEAKER_01: and Instruction at New Mexico State University.
• 0:38
  SPEAKER_01: Her work focuses on the intersections of STEM education, teacher education,
• 0:44
  SPEAKER_01: and technology, with a particular interest in culturally responsive teaching
• 0:49
  SPEAKER_01: and educational equity.
• 0:51
  SPEAKER_01: She is currently exploring how to create some equitable STEM classrooms
• 0:56
  SPEAKER_01: and how generative AI can support teaching and learning.
• 1:00
  SPEAKER_01: Welcome to this podcast, Dr. Chatterjee.
• 1:02
  SPEAKER_01: We are delighted to have you with us today.
• 1:05
  SPEAKER_01: Do you want to add anything else?
• 1:07
  SPEAKER_01: Talk about your recent research or anything else?
• 1:11
  SPEAKER_00: Thank you so much for inviting me to this episode.
• 1:15
  SPEAKER_00: And I'm very happy and excited to talk with you.
• 1:19
  SPEAKER_00: And I feel bad Guy is not here.
• 1:22
  SPEAKER_01: Yeah, yeah.
• 1:24
  SPEAKER_01: You definitely miss him. Yeah.
• 1:27
  SPEAKER_01: Anything exciting about the research that you are doing right now that you want
• 1:30
  SPEAKER_01: to share with us?
• 1:31
  SPEAKER_00: Yes. You have pretty much said everything,
• 1:34
  SPEAKER_00: but one thing I would like to add is we are very much working towards strengthening
• 1:39
  SPEAKER_00: the STEM workforce in New Mexico.
• 1:42
  SPEAKER_00: Okay.
• 1:43
  SPEAKER_00: Yes, with all the research that I do with technology integration and teaching,
• 1:48
  SPEAKER_00: be it additive manufacturing for teachers and students or for AI.
• 1:54
  SPEAKER_00: Yes.
• 1:57
  SPEAKER_00: Okay.
• 1:57
  SPEAKER_01: Perfect. Yeah.
• 1:59
  SPEAKER_01: So I'm going to ask the first question.
• 2:02
  SPEAKER_01: Your research focuses on creating equitable STEM classrooms.
• 2:06
  SPEAKER_01: How do you see generative AI helping
• 2:09
  SPEAKER_01: or potentially reinforcing existing inequalities in STEM education?
• 2:15
  SPEAKER_00: Thank you for the question.
• 2:17
  SPEAKER_00: So in order to answer, like AI does both and you know which one wins depends
• 2:23
  SPEAKER_00: on who holds the authority in learning.
• 2:25
  SPEAKER_00: And the multiple ways of knowing.
• 2:28
  SPEAKER_00: So, for example, if I give an example of a fifth grade science teacher
• 2:32
  SPEAKER_00: in our workshop, in her fifth grade classroom, when she is teaching a weather unit,
• 2:37
  SPEAKER_00: she asks a question like, how do you know if it is going to rain today?
• 2:42
  SPEAKER_00: And students answer answers from their own observations,
• 2:47
  SPEAKER_00: what they have heard from their family members.
• 2:50
  SPEAKER_00: Then, you know, what they have learned from data or, you know, measurements.
• 2:54
  SPEAKER_00: Absolutely. And then people across cultures,
• 2:59
  SPEAKER_00: you know, that have learned through learned over generations.
• 3:02
  SPEAKER_00: So now, generative AI arrives as a fifth source.
• 3:08
  SPEAKER_00: So AI helps when students hold it
• 3:12
  SPEAKER_00: to the same standard as every other sources
• 3:16
  SPEAKER_00: and sits alongside their own observations, their grandma's weather signals,
• 3:21
  SPEAKER_00: real instrument data.
• 3:23
  SPEAKER_00: And they are able to interrogate AI, like what did AI keep?
• 3:27
  SPEAKER_00: What did AI change?
• 3:29
  SPEAKER_00: Whose voice is missing?
• 3:31
  SPEAKER_00: And so understanding that AI is not my answer key.
• 3:36
  SPEAKER_00: It is another source to examine
• 3:39
  SPEAKER_00: and see what it is doing to my other sources of knowledge.
• 3:44
  SPEAKER_00: So it reinforces inequality when it answers,
• 3:49
  SPEAKER_00: sounds universal, but carry a narrow point of view.
• 3:54
  SPEAKER_00: It risks eraser also.
• 3:56
  SPEAKER_00: So this teacher, she was talking about when she gives the name
• 4:02
  SPEAKER_00: of Native American speaker, And asks about their community observation
• 4:09
  SPEAKER_00: of seasonal birds and monsoons.
• 4:12
  SPEAKER_00: I kept the general pattern, but dropped the name source and link to the Zuni drive farmlands.
• 4:19
  SPEAKER_00: That means what it is getting to the general interpretations.
• 4:24
  SPEAKER_00: And this is how we can see it is bringing inequality.
• 4:29
  SPEAKER_00: But again, it also helps students learn through multiple representation.
• 4:33
  SPEAKER_00: Scaffolding learners and also support multilingual learners with translating.
• 4:40
  SPEAKER_01: Yeah, yeah, that makes a lot of sense.
• 4:42
  SPEAKER_01: So do you mean that AI plays
• 4:46
  SPEAKER_01: a kind of complementary role in the classrooms?
• 4:51
  SPEAKER_00: So this is a question like how you are using AI,
• 4:54
  SPEAKER_00: you are using AI as a collaborator in learning?
• 4:57
  SPEAKER_00: Or as a content generator or as a coach, depending, very much depending on,
• 5:03
  SPEAKER_00: you know, whose authority.
• 5:05
  SPEAKER_00: Like here comes the question of epistemic agency.
• 5:08
  SPEAKER_01: Okay. Yeah.
• 5:10
  SPEAKER_01: And it can, in my imagination, it can play all of these roles even in one class session.
• 5:17
  SPEAKER_01: Yeah. Okay.
• 5:19
  SPEAKER_01: Thank you. So how can educators integrate AI into culturally responsive teaching?
• 5:27
  SPEAKER_00: Yes, to answer this question again, it comes from the previous example, you know.
• 5:31
  SPEAKER_00: So put AI alongside other ways of knowing.
• 5:35
  SPEAKER_00: So, for example, in the weather unit,
• 5:37
  SPEAKER_00: students gather knowledge from their own observation, from, you know, family,
• 5:41
  SPEAKER_00: community, scientific instruments and other cultures.
• 5:44
  SPEAKER_00: Before they meet AI as the fifth source.
• 5:47
  SPEAKER_01: Yeah.
• 5:48
  SPEAKER_00: And then they ask AI what AI did when AI retold them what they know.
• 5:54
  SPEAKER_00: And the other thing I really see helpful is giving students an audit protocol.
• 6:01
  SPEAKER_00: So helping students develop the habit of questioning,
• 6:07
  SPEAKER_00: how certain does the response sound, like the AI response.
• 6:13
  SPEAKER_00: So reviewing the authority.
• 6:15
  SPEAKER_00: What evidence does it actually give?
• 6:19
  SPEAKER_00: That means checking the evidence or information.
• 6:23
  SPEAKER_00: And then whose voice is kept?
• 6:25
  SPEAKER_00: Whose context is disappearing?
• 6:28
  SPEAKER_00: So that is representation.
• 6:30
  SPEAKER_01: And
• 6:33
  SPEAKER_00: let students, and the third thing that I really very much like is let student voice.
• 6:39
  SPEAKER_00: Let students respond and their voice be present there.
• 6:42
  SPEAKER_00: So for example, in a photosynthesis class,
• 6:47
  SPEAKER_00: Some of the things are universal, right?
• 6:50
  SPEAKER_00: Like plants use carbon dioxide, water, light to make glucose and oxygen.
• 6:56
  SPEAKER_00: But then when students are drawing, they can use their own plants of choice.
• 7:01
  SPEAKER_00: They can use their own landscape.
• 7:02
  SPEAKER_00: They can use their own surroundings where they want to see this photosynthesis happening.
• 7:08
  SPEAKER_00: So this means science or reasoning stays at the same time students come
• 7:12
  SPEAKER_00: with their own representations.
• 7:16
  SPEAKER_01: Very interesting.
• 7:17
  SPEAKER_01: And also, I think based on the thing that you said in the beginning,
• 7:22
  SPEAKER_01: so we need to help students develop critical thinking skills.
• 7:26
  SPEAKER_01: Yes.
• 7:29
  SPEAKER_01: They should be also able to go back to AI output,
• 7:33
  SPEAKER_01: do the output verification to make sure
• 7:35
  SPEAKER_01: and not just blindly accept what AI tells them.
• 7:39
  SPEAKER_00: Definitely. Yes. So AI, that's what I say, like treating AI not as an answer key.
• 7:44
  SPEAKER_00: Or as a source of truth, but integrating AI and trying
• 7:48
  SPEAKER_00: to see critically what is there and what is missing.
• 7:52
  SPEAKER_01: Yeah, that makes a lot of sense. Interesting.
• 7:54
  SPEAKER_01: And it's interesting that you are talking about that in a kind of K-12 settings
• 7:58
  SPEAKER_01: because we are seeing the same patterns in higher education.
• 8:01
  SPEAKER_01: I mean, personally, most of my research has been in higher education
• 8:06
  SPEAKER_01: and When I interview undergraduate engineering education,
• 8:11
  SPEAKER_01: I mean engineering undergraduates,
• 8:13
  SPEAKER_01: I see the same patterns in terms of output verification,
• 8:17
  SPEAKER_01: going back to the other resources to make sure that AI output is accurate.
• 8:23
  SPEAKER_01: So that's very interesting.
• 8:26
  SPEAKER_01: So you have a paper in 2024 about social media and artificial intelligence.
• 8:32
  SPEAKER_01: You explored critical questions about these technologies.
• 8:36
  SPEAKER_01: What do you think education should pay the most attention to?
• 8:41
  SPEAKER_00: Yes, in that collaborative autoethnography, actually we tracked our tool,
• 8:46
  SPEAKER_00: we built engagement around, became unstable almost overnight.
• 8:50
  SPEAKER_00: So the lesson is like, don't anchor Your pedagogy to the platform,
• 8:56
  SPEAKER_00: anchor it to the framework and relationships and choose the tool that serve them.
• 9:01
  SPEAKER_00: So organized around the theme of hope, you know,
• 9:04
  SPEAKER_00: we talked about tool volatility and co-creating norms instead of like,
• 9:11
  SPEAKER_00: so we argue that simply prohibiting AI use is not the solution.
• 9:16
  SPEAKER_00: Rather, learning the strategies and tools together
• 9:21
  SPEAKER_00: with the students is what we argued for, what we talked about.
• 9:25
  SPEAKER_00: That is the reason we actually have a document there which talks
• 9:29
  SPEAKER_00: about co-creating norms and values for AI use.
• 9:35
  SPEAKER_00: Also, we talked about assessment redesigning so that we talked more
• 9:40
  SPEAKER_00: about how we can emphasize assessments that AI Struggles to replicate, for example,
• 9:47
  SPEAKER_00: originality, emotional depth, metacognition, personal experience, you know.
• 9:51
  SPEAKER_00: So the whole point is that hopes comes from educators' choice
• 9:57
  SPEAKER_00: of what educators make, how educators make the choice,
• 10:02
  SPEAKER_00: educators design and not from the tools.
• 10:07
  SPEAKER_01: Interesting.
• 10:09
  SPEAKER_01: The other question is about your work in instructional design and teacher education.
• 10:16
  SPEAKER_01: So how should teacher preparation programs evolve to prepare future educators
• 10:21
  SPEAKER_01: to use AI critically, creatively, and also effectively?
• 10:26
  SPEAKER_00: Okay, very interesting and very nice question. Yes.
• 10:30
  SPEAKER_00: So I can talk a little bit about from my own practices, what I am doing in my courses.
• 10:35
  SPEAKER_00: So the first thing is like embed AI in the courses for disciplinary, you know,
• 10:42
  SPEAKER_00: disciplinary teaching and learning, not Using AI as a standalone activity.
• 10:49
  SPEAKER_00: It can be possible in educational technology courses, but in my method courses,
• 10:53
  SPEAKER_00: what I'm trying to use AI is teachers are using AI for role playing.
• 10:59
  SPEAKER_00: So AI is acting as their elementary students and they are teaching a science topic
• 11:06
  SPEAKER_00: to the students who have A dissimulated student who has misconceptions.
• 11:11
  SPEAKER_00: And in that way, teachers are being able to teach a student a science topic
• 11:17
  SPEAKER_00: where they're developing their pedagogical content knowledge
• 11:19
  SPEAKER_00: and their inquiry instruction skills.
• 11:23
  SPEAKER_00: So this is one of the ways I think I really want to emphasize.
• 11:29
  SPEAKER_00: And the other thing is practicing auditing before designing.
• 11:33
  SPEAKER_00: So repeatedly trying to see what AI outputs,
• 11:39
  SPEAKER_00: how AI is generating the output,
• 11:42
  SPEAKER_00: what we are seeking for.
• 11:44
  SPEAKER_00: And I can give you an interesting example.
• 11:47
  SPEAKER_00: So a few days last week only in my instructional design course
• 11:52
  SPEAKER_00: for graduate students, I was talking about content analysis.
• 11:58
  SPEAKER_00: I was after learner and learner analysis.
• 12:02
  SPEAKER_00: And then I gave them three problems, three physics problems.
• 12:07
  SPEAKER_00: And two of the problems were showing that, you know,
• 12:11
  SPEAKER_00: 30 degree inclined plane having a block, a mass,
• 12:15
  SPEAKER_00: and then it was showing the friction coefficient.
• 12:18
  SPEAKER_00: And the other problem was showing, you know, a block attached to a spring.
• 12:23
  SPEAKER_00: And it was also, so these three problems were shown.
• 12:27
  SPEAKER_00: And then I asked students, like, okay, group these problems based on You know,
• 12:31
  SPEAKER_00: based on your understanding about the principles and all the students,
• 12:36
  SPEAKER_00: they group the problems, the first two where it was showing that inclined plane,
• 12:40
  SPEAKER_00: 30 degree ramp and block and that mu versus no one,
• 12:46
  SPEAKER_00: no one connected the problems of the spring.
• 12:50
  SPEAKER_00: So what is happening?
• 12:52
  SPEAKER_00: And while my students were doing it,
• 12:54
  SPEAKER_00: I put the same problem in ChatGPT and I was trying to see how ChatGPT, you know,
• 12:58
  SPEAKER_00: is interesting. Yeah.
• 13:00
  SPEAKER_00: And then I found that all my students, along with ChatGPT,
• 13:03
  SPEAKER_00: they showed that the first two problems go together.
• 13:08
  SPEAKER_00: So the whole idea was around appearance,
• 13:12
  SPEAKER_00: you know, surface appearance signal was strong.
• 13:17
  SPEAKER_00: Because they were not trying to see
• 13:19
  SPEAKER_00: the underlying principles governing
• 13:24
  SPEAKER_00: the physics principle, how it can be solved.
• 13:26
  SPEAKER_00: So I was actually talking about you novice and experts, how novice and experts,
• 13:32
  SPEAKER_00: content experts talk about problems.
• 13:36
  SPEAKER_00: And then using this AI, it was interesting and I showed my students that how important it is for
• 13:44
  SPEAKER_00: Yes,
• 13:49
  SPEAKER_00: and it was very interesting to see AI gave a note there that do not go
• 13:54
  SPEAKER_00: by the numbers, go by the governing principles, and it was not following itself.
• 13:59
  SPEAKER_00: So that was an excellent example to discuss with my students about how AI works.
• 14:06
  SPEAKER_01: Yeah, very interesting.
• 14:07
  SPEAKER_01: So even AI made a mistake.
• 14:11
  SPEAKER_00: AI actually made a mistake and the language was very fluent,
• 14:17
  SPEAKER_00: very well formatted language and so confident and it was showing the answer
• 14:22
  SPEAKER_00: with bold and not bold and bulleted list
• 14:26
  SPEAKER_00: and giving all It sounds
• 14:29
  SPEAKER_01: very convincing
• 14:30
  SPEAKER_01: when it generates something.
• 14:33
  SPEAKER_00: Exactly. So that is the point.
• 14:35
  SPEAKER_00: Very convincing prose is not equivalent to scientific reasoning.
• 14:42
  SPEAKER_00: And the third thing that I would really talk about is...
• 14:46
  SPEAKER_00: Model what we expect, so disclosures about how we use AI, being transparent.
• 14:54
  SPEAKER_00: So you know that in that paper in 2024,
• 14:56
  SPEAKER_00: actually in developing values and norms,
• 15:01
  SPEAKER_00: co-creating values and norms,
• 15:02
  SPEAKER_00: we developed a document built around integrity.
• 15:09
  SPEAKER_00: Transparency and disclosure.
• 15:11
  SPEAKER_00: So if we expect that from our students, from others,
• 15:16
  SPEAKER_00: then we need to practice it and we need to do it explicitly.
• 15:21
  SPEAKER_01: Interesting. Very interesting.
• 15:23
  SPEAKER_01: And it's awesome that you can show those things in classroom, I mean,
• 15:26
  SPEAKER_01: to students so they can practically understand how interactions with AI work
• 15:32
  SPEAKER_01: and it can always make mistakes.
• 15:35
  SPEAKER_00: Yes, it can always make mistakes in our workshops or in the work.
• 15:41
  SPEAKER_00: We really want to emphasize that AI literacy
• 15:45
  SPEAKER_00: and fluency goes along with critical use of AI.
• 15:50
  SPEAKER_01: Absolutely.
• 15:51
  SPEAKER_00: If you're not practicing that, then we are missing something and we are going behind.
• 15:58
  SPEAKER_01: Mm-hmm. Mm-hmm. Yeah.
• 15:59
  SPEAKER_01: It makes a lot of sense. Yeah. Yeah. Okay.
• 16:02
  SPEAKER_01: That was great.
• 16:03
  SPEAKER_01: So how can teachers encourage students to use AI as a tool for deeper learning?
• 16:10
  SPEAKER_00: Right. So this is exactly what we are trying to do,
• 16:13
  SPEAKER_00: work with our teachers in helping them understand, you know, how they can use,
• 16:19
  SPEAKER_00: how they can work with their students to help them in deeper engagement.
• 16:22
  SPEAKER_00: So the first thing is like think first, Then compare.
• 16:26
  SPEAKER_00: So as I gave you the first example,
• 16:28
  SPEAKER_00: so students are here generating their own content for students that, you know,
• 16:33
  SPEAKER_00: trying to answer the question like through their own observations, family,
• 16:37
  SPEAKER_00: community knowledge and data they have seen through scientific measurements.
• 16:42
  SPEAKER_00: But and then they are
• 16:45
  SPEAKER_00: then they are generating the Comparing with the AI-generated output.
• 16:49
  SPEAKER_00: So what is happening here, we are protecting the reasoning or helping students
• 16:54
  SPEAKER_00: to think and then make the difference visible.
• 17:01
  SPEAKER_00: Ask expert or novice.
• 17:03
  SPEAKER_00: So the example that I gave you from the physics example,
• 17:08
  SPEAKER_00: it is exactly what we want our students to learn.
• 17:12
  SPEAKER_00: Like, you know, where you are, like AI answers.
• 17:16
  SPEAKER_00: It's a very confident voice.
• 17:20
  SPEAKER_00: That may not be the true reasoning.
• 17:24
  SPEAKER_00: So do not go with the surface features,
• 17:27
  SPEAKER_00: but go with more with the reasons and the underlying principles.
• 17:33
  SPEAKER_00: Ask the same question again and again,
• 17:35
  SPEAKER_00: or maybe at least twice or thrice so that you can compare the answers
• 17:40
  SPEAKER_00: about the general method and What specific knowledge
• 17:45
  SPEAKER_00: that you want to talk
• 17:49
  SPEAKER_00: about requires evidence for every claim, question AI, interrogate AI,
• 17:56
  SPEAKER_00: whatever it is generating, how it got, work around the process,
• 18:04
  SPEAKER_00: try to understand the process, how it is generated,
• 18:07
  SPEAKER_00: where that reasoning is actually If it is that actual reason, how it is, how it is,
• 18:14
  SPEAKER_00: you know, claiming what it is,
• 18:17
  SPEAKER_00: what it is generating and revise the prompt as an inquiry.
• 18:21
  SPEAKER_00: So this is very important.
• 18:23
  SPEAKER_00: And this is I have learned like, you know,
• 18:24
  SPEAKER_00: writing a better prompt requires understanding of the content.
• 18:28
  SPEAKER_00: So if it is if if it is generating a content,
• 18:31
  SPEAKER_00: try to see how you can ask more
• 18:34
  SPEAKER_00: or what you can ask or refine your question in a way
• 18:39
  SPEAKER_00: so that you can get specific.
• 18:42
  SPEAKER_01: Interesting. Very interesting points.
• 18:44
  SPEAKER_01: So just to summarize what you just said about helping students develop deeper learning.
• 18:51
  SPEAKER_01: So first we need to help them kind
• 18:55
  SPEAKER_01: of gain foundational knowledge.
• 18:59
  SPEAKER_01: So they should be By using that knowledge,
• 19:02
  SPEAKER_01: they will be able to assess what AI gives them, ask AI multiple times,
• 19:09
  SPEAKER_01: not just like one time prompts or I mean,
• 19:13
  SPEAKER_01: going beyond and doing multiple phases of interactions with AI, interrogate AI,
• 19:19
  SPEAKER_01: ask AI to give us references when it generates an output.
• 19:24
  SPEAKER_01: And I think I missed one of the good points that you mentioned, but Yeah.
• 19:31
  SPEAKER_00: Yes, it's like, you know, like ask, I think you have already covered it,
• 19:36
  SPEAKER_00: like ask the prompt, inquire the prompt that you have already given
• 19:41
  SPEAKER_00: and what it generated.
• 19:44
  SPEAKER_00: And all of this, yes, all of this requires that epistemic agency, you know,
• 19:48
  SPEAKER_00: like capacity and authority to shape what counts as knowledge and how
• 19:55
  SPEAKER_00: the claims are justified.
• 19:56
  SPEAKER_00: So this is important for learners to understand.
• 20:00
  SPEAKER_00: Who is having that epistemic agency?
• 20:04
  SPEAKER_00: Who is that authority?
• 20:05
  SPEAKER_00: It's not AI.
• 20:07
  SPEAKER_01: Yeah, absolutely. It's me. Yeah, yeah.
• 20:10
  SPEAKER_01: That makes a lot of sense.
• 20:11
  SPEAKER_01: That makes a lot of sense. Yeah.
• 20:13
  SPEAKER_01: Thank you so much, Dr. Chatterjee.
• 20:15
  SPEAKER_01: It was a very delightful conversation with you.
• 20:18
  SPEAKER_01: If you want to add anything else beyond what I just asked,
• 20:24
  SPEAKER_00: It's so interesting to be here with you and having this conversation.
• 20:30
  SPEAKER_01: Same here.
• 20:31
  SPEAKER_00: Yes. Thank you.
• 20:32
  SPEAKER_01: Thank you so much.
• 20:34
  SPEAKER_01: Bye.