Latest / Azadeh and Guy on AI / Can AI Really Replace Teachers?

Transcript

Can AI Really Replace Teachers? · Azadeh and Guy on AI

• 0:01
  SPEAKER_00: Hi, and welcome.
• 0:03
  SPEAKER_01: Yeah, hello.
• 0:05
  SPEAKER_00: Episode four, season three, episode four.
• 0:09
  SPEAKER_00: We've had this discussion right now.
• 0:12
  And I have decided this is episode four of this marvelous
• 0:17
  season that is going
• 0:18
  SPEAKER_00: throughout the summer.
• 0:20
  SPEAKER_00: And this time we decided to go in a different direction.
• 0:22
  So this time we're looking at two pieces of evidence,
• 0:27
  one from Alpha Schools,
• 0:29
  SPEAKER_00: There are schools that lean heavily on AI as part of the promise.
• 0:33
  SPEAKER_00: And the second one is going back to the ideas around Canmigo.
• 0:37
  SPEAKER_00: I did mention that in a previous episode.
• 0:42
  And I wanted to go back and kind of dig a little bit deeper as
• 0:47
  we're setting it all up.
• 0:49
  SPEAKER_00: But it all starts with Benjamin Bloom and his
• 0:53
  SPEAKER_01: Two-sigma problem.
• 0:55
  SPEAKER_00: His two-sigma problem.
• 0:57
  So he published in the mid-80s a paper that claimed that
• 1:02
  individual tutoring can
• 1:04
  SPEAKER_00: produce two-standard deviation or what he called two-sigma.
• 1:08
  SPEAKER_00: Nowadays we don't use sigma as much,
• 1:11
  SPEAKER_00: but two-standard deviations difference is a huge effect size.
• 1:16
  SPEAKER_00: And that is the source of the claim
• 1:22
  SPEAKER_00: That a lot of tutoring platforms are suggesting might happen with kids.
• 1:29
  And partially it is because intuitively we all think that
• 1:37
  one-to-one teaching is
• 1:38
  SPEAKER_00: the best you can do.
• 1:38
  And the only reason we don't do that is because we don't have
• 1:42
  the capacity.
• 1:43
  It's like if you have a million students,
• 1:45
  you would need a million teachers.
• 1:47
  SPEAKER_00: That does not seem like a reasonable way to spend money.
• 1:51
  SPEAKER_00: That effect of two standard deviation is quite exaggerated most likely.
• 2:02
  SPEAKER_00: And so partially it is that selective reading.
• 2:05
  SPEAKER_00: So two things I think are happening when people make that claim.
• 2:08
  And this especially comes from Silicon Valley and the idea that
• 2:13
  technology would save us all.
• 2:15
  SPEAKER_00: is that they're not aware of the literature as much.
• 2:19
  SPEAKER_00: And so they're going to old studies that are bombastic and saying,
• 2:22
  SPEAKER_00: this is what we're going to produce.
• 2:23
  SPEAKER_00: But in fact, we've got meta-analyses.
• 2:28
  SPEAKER_00: There's a Kulik and Kulik meta-analysis and other meta-analyses.
• 2:32
  Harry's collection of meta-analysis that showed significant
• 2:36
  effect for tutoring.
• 2:38
  SPEAKER_00: But not two standard deviations.
• 2:40
  We're talking about something between half to two thirds of a
• 2:44
  standard deviation,
• 2:45
  SPEAKER_00: which is still at least a moderate effect,
• 2:48
  SPEAKER_00: but it is not an enormous effect like he claimed.
• 2:55
  SPEAKER_01: So today we are talking about Alpha School.
• 2:58
  SPEAKER_01: They promised that with just two hours of learning per day,
• 3:04
  SPEAKER_01: kids will learn a lot,
• 3:05
  SPEAKER_01: much more than conventional classrooms.
• 3:09
  SPEAKER_01: And also Conmigo, which is a chatbot.
• 3:12
  SPEAKER_01: And it was introduced in 2024?
• 3:16
  SPEAKER_00: Yes, after a very long development time.
• 3:20
  SPEAKER_00: And we'll get to that.
• 3:21
  SPEAKER_00: Let's talk about the alpha schools first.
• 3:24
  SPEAKER_00: So alpha schools are private schools that cost a lot of money.
• 3:30
  And what they're doing is they're promising to use AI to
• 3:36
  provide immediate feedback
• 3:37
  SPEAKER_00: to learning with competency goals,
• 3:42
  SPEAKER_00: so meeting the goals and moving to the next thing.
• 3:45
  SPEAKER_00: And then the second part of the day in most of them,
• 3:49
  SPEAKER_00: according to the literature,
• 3:50
  SPEAKER_00: is more Socratic discussion with guides.
• 3:53
  SPEAKER_00: So with teachers who are acting as the guides and do this more social,
• 4:00
  SPEAKER_00: making claims,
• 4:01
  SPEAKER_00: discussing and all of that as a way for kids to make progress in schools.
• 4:07
  And their claim is that it's going to be a lot better and it's
• 4:11
  going to pay
• 4:11
  SPEAKER_00: attention to individual students based on their needs.
• 4:16
  And the alpha schools are interesting because it seems like
• 4:21
  they went too fast to market.
• 4:23
  And that is that they have had a concept and they have made
• 4:27
  promises without having
• 4:29
  SPEAKER_01: a lot of empirical evidence to back it up.
• 4:34
  SPEAKER_01: And when they are talking about two hours,
• 4:37
  SPEAKER_01: many people may think that they use AI to teach kids,
• 4:40
  SPEAKER_01: but this is not what they are doing.
• 4:42
  SPEAKER_00: Well, inside those two hours, they are.
• 4:44
  SPEAKER_00: So they're using things that they have developed internally,
• 4:50
  SPEAKER_00: and in a minute I'll talk about whether it's internal or not,
• 4:53
  but materials that they've developed that provide small
• 4:57
  learning objectives,
• 4:59
  SPEAKER_00: a lot of support.
• 4:59
  SPEAKER_00: So if you think about what Khan did in Khan Academy,
• 5:03
  SPEAKER_00: that's the kind of learning they're talking about.
• 5:05
  A lot of video,
• 5:06
  a lot of small drills leading to bigger objectives,
• 5:10
  and the
• 5:13
  Adult individuals are there to guide that work and make sure
• 5:16
  kids are working and
• 5:17
  SPEAKER_00: then to do the Socratic instruction outside of this.
• 5:19
  SPEAKER_01: So you mean they use AI-generated lessons?
• 5:23
  SPEAKER_00: The lessons are not necessarily generated by AI.
• 5:26
  SPEAKER_00: What's generated by AI,
• 5:29
  SPEAKER_00: as you made the observation before we started,
• 5:32
  SPEAKER_00: is that there is some source curriculum
• 5:38
  SPEAKER_00: What the AI is doing, providing feedback on performance.
• 5:40
  So the AI piece is more on the feedback side than it is on the
• 5:44
  teaching side,
• 5:44
  SPEAKER_00: but the teaching is still for those nuggets that are competency-based,
• 5:49
  SPEAKER_01: are happening digitally.
• 5:53
  And the other thing that I figured out is that they are not
• 5:56
  just looking at student outcomes.
• 5:59
  SPEAKER_01: AI also records eye movement and other things to analyze how students are
• 6:05
  SPEAKER_01: performing on those lessons.
• 6:07
  SPEAKER_00: So and that's challenging.
• 6:09
  That's challenging to me personally,
• 6:12
  because it feels very surveillance oriented kind of things.
• 6:17
  SPEAKER_00: And that's legitimate if you are
• 6:20
  SPEAKER_00: Really generating a lot of high quality data based on that,
• 6:23
  SPEAKER_00: that allows you to say what's going on.
• 6:26
  We have yet to see the evidence for the richness of that data
• 6:30
  really informing instruction.
• 6:32
  SPEAKER_00: And there have been some publications.
• 6:34
  These are anecdotal publications by parents and by people who
• 6:40
  have visited and all
• 6:40
  SPEAKER_00: of that,
• 6:41
  SPEAKER_00: that make different claims.
• 6:44
  SPEAKER_00: You read one piece, so talk a little bit.
• 6:47
  SPEAKER_01: So, let me see.
• 6:49
  SPEAKER_01: The thing that I...
• 6:52
  SPEAKER_01: I'm still reading the thing.
• 6:53
  SPEAKER_01: That's an interesting, I mean, read.
• 6:55
  SPEAKER_01: But I have not finished that.
• 6:57
  SPEAKER_01: They started about how they started
• 7:00
  Sending their kids to a school and how they made a decision to
• 7:05
  send their kids to Alpha School.
• 7:07
  SPEAKER_01: They were mostly excited about the feedback that they got from AI.
• 7:13
  So the AI feedback also provides them with some links to the
• 7:18
  moments of learning.
• 7:20
  SPEAKER_01: For example, when kids made a huge progress.
• 7:23
  SPEAKER_01: They video recorded the kids.
• 7:25
  SPEAKER_01: You can go to the link and even watch that moment.
• 7:29
  SPEAKER_01: And it also gives a lot of good advice in terms of how to,
• 7:35
  SPEAKER_01: I don't know,
• 7:35
  SPEAKER_01: how to learn better,
• 7:38
  SPEAKER_01: what are the mistakes,
• 7:39
  SPEAKER_01: how you can make more progress.
• 7:41
  So on the flip side,
• 7:44
  especially in Wired magazine,
• 7:45
  there was a long story.
• 7:47
  And they've talked to a number of parents who were unhappy,
• 7:51
  even very unhappy.
• 7:53
  SPEAKER_00: So they were really excited by the promise.
• 7:55
  SPEAKER_00: They were excited by the fact that
• 7:58
  SPEAKER_00: There was this idea that the kids would do much better than they are.
• 8:03
  These are parents who were not happy with what was happening in
• 8:06
  the public schools
• 8:07
  SPEAKER_00: where their kids were going.
• 8:08
  SPEAKER_00: So they were preconditioned to think about what do my kids need?
• 8:19
  And obviously the promise,
• 8:20
  because this is very expensive,
• 8:21
  so the promise was important.
• 8:24
  SPEAKER_00: What they've reported back are kind of two effects.
• 8:28
  One is that they haven't seen the growth that they expected to
• 8:31
  see in their kids.
• 8:33
  SPEAKER_00: And that may be, again, this is anecdotal evidence.
• 8:35
  SPEAKER_00: But the reason I'm talking,
• 8:37
  I think we're talking about anecdotal evidence is that alpha
• 8:40
  schools have their
• 8:40
  SPEAKER_00: internal evaluations,
• 8:42
  SPEAKER_00: but they don't have any kind of evidence that would stand and say,
• 8:48
  This is a proven effective practice and our curriculum
• 8:53
  materials work and kids make progress.
• 8:56
  And the reason we're suspicious is obviously there's a business
• 8:59
  model behind it.
• 9:01
  SPEAKER_00: And that business model is charging parents a lot of money.
• 9:04
  SPEAKER_00: So obviously they have an interest to show results.
• 9:08
  And the only evaluation results they have released into the
• 9:12
  public,
• 9:13
  to the public,
• 9:14
  are outcomes.
• 9:17
  SPEAKER_00: And the problem with outcomes is one is there's no comparison group,
• 9:20
  SPEAKER_00: definitely not equal comparison group.
• 9:22
  SPEAKER_00: And there's a definitely selection bias.
• 9:24
  SPEAKER_00: That is,
• 9:25
  these are parents that are paying tens of thousands of dollars
• 9:28
  a year to send their
• 9:29
  SPEAKER_00: kids into this kind of an environment.
• 9:32
  SPEAKER_00: And I know that the vast majority of parents can't do that.
• 9:35
  SPEAKER_00: So there's an inherent selection bias of who is it working for.
• 9:39
  SPEAKER_00: And even if it is working.
• 9:42
  SPEAKER_00: So even if they had evidence that it was working for their kids,
• 9:45
  SPEAKER_00: which is partial,
• 9:46
  SPEAKER_00: they have some evaluations again,
• 9:49
  SPEAKER_00: but it is only working with them.
• 9:51
  SPEAKER_00: So there's no comparable group.
• 9:53
  SPEAKER_00: And we know that from the literature about private schools,
• 9:55
  this is a problem that goes all the way to the dawn of
• 9:59
  evaluation of what public
• 10:00
  schools are doing and what private schools are doing is that
• 10:04
  private schools seem
• 10:05
  SPEAKER_00: to be doing better until you
• 10:09
  SPEAKER_00: You divide the data and you stratify it by the background of the kids.
• 10:13
  SPEAKER_00: And suddenly all of those benefits are going away.
• 10:16
  What it turns out is private schools outside of really some
• 10:21
  religious schools tend
• 10:22
  to cater to a very well to do segment of the population who
• 10:26
  tends to also do better
• 10:27
  SPEAKER_00: at school and for a variety of reasons.
• 10:30
  SPEAKER_00: But we know that for a fact.
• 10:31
  SPEAKER_00: So when parents talked about
• 10:37
  SPEAKER_00: The school, they talked about two things.
• 10:39
  SPEAKER_00: One is, as I said, kids do not make progress.
• 10:41
  SPEAKER_00: The second thing is when people looked at the actual quality of material,
• 10:47
  it looked like things that were ripped off from other digital
• 10:51
  sources without
• 10:52
  SPEAKER_00: permission.
• 10:53
  SPEAKER_00: Guy Trainin
• 10:53
  guytrainin5226193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193193
• 11:21
  SPEAKER_00: Then it's going to be rough.
• 11:24
  SPEAKER_00: And so that's the question that I have about alpha schools is we need an
• 11:29
  SPEAKER_00: independent evaluation of quality.
• 11:31
  SPEAKER_00: And I think that especially as a business,
• 11:36
  SPEAKER_00: we cannot trust the business because the business,
• 11:38
  SPEAKER_00: if they have failed to produce something,
• 11:41
  SPEAKER_00: they are unlikely to share.
• 11:44
  SPEAKER_00: And we know that after they've been approved in a few states,
• 11:47
  there are also a few states that said you don't have enough
• 11:49
  evidence of efficacy,
• 11:50
  SPEAKER_00: so we do not let you operate.
• 11:52
  SPEAKER_00: And I think Pennsylvania was one of those states.
• 11:56
  So that's the Alpha School and the good news is that we still
• 12:01
  need teachers.
• 12:02
  SPEAKER_00: Even in the Alpha School that has a lot of AI,
• 12:05
  SPEAKER_00: they're still using teachers as guides,
• 12:07
  SPEAKER_00: but still teachers.
• 12:08
  We need still adults in that mix and we are a long way from
• 12:15
  figuring out how to use
• 12:16
  SPEAKER_00: AI and integrate it so it works.
• 12:21
  SPEAKER_01: The other thing that we are going to talk about is
• 12:24
  SPEAKER_00: Kanmigo, our best friend.
• 12:26
  SPEAKER_00: He's our amigo.
• 12:29
  SPEAKER_00: I mean, we all want Salman Khan to be our personal friend.
• 12:33
  SPEAKER_01: Very true.
• 12:34
  And they were very excited to introduce that chatbot and
• 12:38
  incorporate it into their
• 12:41
  SPEAKER_01: platform Khan Academy.
• 12:43
  SPEAKER_01: But after a while,
• 12:44
  SPEAKER_01: I mean,
• 12:44
  SPEAKER_01: after a few years,
• 12:47
  SPEAKER_01: Sal Khan said that for a lot of students,
• 12:49
  SPEAKER_01: it was a non-event.
• 12:51
  SPEAKER_00: Yeah, and this is a high quality product.
• 12:55
  SPEAKER_00: So Sal Khan has done a really,
• 12:58
  an excellent job in producing videos and really thinking
• 13:03
  through what do these
• 13:05
  SPEAKER_00: courses need?
• 13:07
  SPEAKER_00: What are the markers of achievement?
• 13:10
  SPEAKER_00: And the impact on learners was always partial.
• 13:13
  They've tried it in a number of districts throughout the years
• 13:17
  with small to medium
• 13:19
  SPEAKER_00: effects at most.
• 13:21
  SPEAKER_00: And the idea was that this tutor would be able to do so much more.
• 13:26
  SPEAKER_01: Yeah.
• 13:26
  SPEAKER_01: And it seems that they changed the design a couple of times.
• 13:30
  SPEAKER_01: So at first the chatbot was silent,
• 13:33
  SPEAKER_01: but then it popped up and it suggested interaction with the students,
• 13:38
  SPEAKER_01: but it still didn't work.
• 13:41
  SPEAKER_00: So, yes.
• 13:44
  SPEAKER_00: And what I do appreciate is that they came out and said,
• 13:48
  SPEAKER_00: as far as we know right now,
• 13:51
  SPEAKER_00: this is not a huge jump forward.
• 13:53
  SPEAKER_00: It can help.
• 13:54
  SPEAKER_00: It can help some students.
• 13:55
  SPEAKER_00: And I think that's where their major claim is.
• 13:58
  SPEAKER_00: Some kids seem to be able to benefit from this.
• 14:01
  SPEAKER_00: But on average, it doesn't justify
• 14:05
  The extra work and investment that something like that would do
• 14:11
  long term.
• 14:12
  SPEAKER_00: I think that it also depends on conditions on the ground.
• 14:14
  I think that it depends on how good the initial educational
• 14:19
  system is before you
• 14:21
  SPEAKER_00: make a judgment.
• 14:22
  SPEAKER_00: So what is the baseline you're judging against is important.
• 14:25
  It could be that in rural environment,
• 14:27
  and I always think about what happens in environment that
• 14:31
  SPEAKER_00: Don't have quality education for whatever reason.
• 14:35
  SPEAKER_00: Is this going to be better than something that is very little?
• 14:39
  SPEAKER_00: And we're experiencing this across the world, really.
• 14:43
  We're experiencing partial teacher shortages in specific areas
• 14:51
  like science and math especially.
• 14:53
  SPEAKER_00: And can this compensate in those conditions?
• 14:56
  SPEAKER_00: Maybe.
• 14:58
  SPEAKER_00: But as a system, it didn't work quite as they anticipated.
• 15:03
  And I think that partially the judgment here is that they are
• 15:08
  intellectually
• 15:09
  SPEAKER_00: honest,
• 15:09
  SPEAKER_00: which I appreciate.
• 15:10
  But the flip side of this is that they have had longer to work
• 15:14
  with AI than anybody else because
• 15:17
  SPEAKER_00: Sal Khan,
• 15:19
  because of his position and because of his centrality and
• 15:22
  talking about all these
• 15:23
  SPEAKER_00: issues around education,
• 15:24
  SPEAKER_00: actually got access to chat GPT months before anybody else has.
• 15:30
  So their engineers and designers had longer than anybody else
• 15:34
  to work on this problem.
• 15:36
  SPEAKER_00: And they sit in Silicon Valley.
• 15:37
  SPEAKER_00: So I'm thinking if they can solve this,
• 15:40
  SPEAKER_00: not that it's unsolvable,
• 15:41
  SPEAKER_00: but it's definitely a much harder problem than we thought.
• 15:46
  SPEAKER_00: And that the AIs as they are, are not doing a phenomenal job.
• 15:53
  SPEAKER_00: Or they maybe are.
• 15:55
  It's just we as humans are not reacting well to it,
• 15:59
  which is totally,
• 16:00
  totally possible.
• 16:01
  SPEAKER_01: Yeah.
• 16:02
  SPEAKER_01: And people, I mean, they have different ideas.
• 16:08
  SPEAKER_01: Yes.
• 16:09
  SPEAKER_01: Khan Academy,
• 16:10
  part of them blame teachers for not incorporating the Conmigo
• 16:15
  very well into their
• 16:16
  SPEAKER_01: teaching and part of them blame students for not asking good questions.
• 16:23
  But part of the problem could be about the way that Conmigo
• 16:27
  interacted with the students.
• 16:29
  SPEAKER_01: Some people believe that it didn't know how to initiate the interaction,
• 16:33
  SPEAKER_01: how to respond to students,
• 16:36
  SPEAKER_01: how to emotionally engage students.
• 16:39
  SPEAKER_00: And tutoring is mostly that, right?
• 16:42
  SPEAKER_00: A really good tutor.
• 16:44
  And I have spent a couple of years of my life living mostly off
• 16:48
  of tutoring.
• 16:49
  SPEAKER_00: I created a tutoring for reading here on campus.
• 16:52
  So I'm aware of that kind of the difficulties and the fine
• 16:56
  dance that you have to do.
• 16:57
  But one of the most important things you have to do as a tutor
• 17:00
  is learn how to
• 17:03
  SPEAKER_00: To encourage your student and to support your student.
• 17:06
  SPEAKER_00: And if you're talking about why is learning happening so much,
• 17:09
  SPEAKER_00: it's not just about the quality of feedback,
• 17:11
  SPEAKER_00: although quality of feedback is enormous,
• 17:13
  SPEAKER_00: but it's the ability to motivate.
• 17:16
  SPEAKER_00: It's the ability to know how much to push and when to push.
• 17:19
  SPEAKER_00: And that is a hard problem and it may be.
• 17:22
  SPEAKER_00: And here's my claim.
• 17:24
  I'm going to make a claim because we talked before this and I
• 17:28
  have opinions.
• 17:30
  SPEAKER_00: And so I really think that AI is,
• 17:35
  SPEAKER_00: in its current iteration,
• 17:37
  SPEAKER_00: is far from providing the kind of solution we would need in education.
• 17:43
  SPEAKER_01: Could you explain more?
• 17:44
  SPEAKER_00: Yes.
• 17:45
  SPEAKER_00: So I think that teachers do not have to worry.
• 17:48
  Really,
• 17:48
  the bottom line is,
• 17:49
  if you know how to teach,
• 17:51
  whether tutor or teach a larger group,
• 17:54
  You are going to be fine because it does not seem as much as AI
• 17:59
  seems to be coming
• 18:00
  SPEAKER_00: for what are quote unquote white collar jobs.
• 18:05
  SPEAKER_00: Teaching does not seem to be one of them.
• 18:07
  SPEAKER_00: It does not seem to be able to do that well.
• 18:11
  It seems like kids are not responding well to that interaction
• 18:16
  in a real way that
• 18:17
  SPEAKER_00: supports their learning.
• 18:18
  SPEAKER_00: They don't care about it.
• 18:20
  SPEAKER_00: Ais much.
• 18:21
  SPEAKER_00: They don't seem to respond necessarily positively.
• 18:24
  SPEAKER_00: And the truth is that for me,
• 18:27
  that digital entity right now does not seem to be able to pick
• 18:31
  up on those cues.
• 18:33
  SPEAKER_00: And some of those cues are bodily cues, right?
• 18:35
  SPEAKER_00: There's a whole,
• 18:37
  the body language of the learner that AIs are not at this point
• 18:42
  able to read and
• 18:43
  SPEAKER_00: process in real time.
• 18:46
  And that creates a distance where the learner is pushing the
• 18:50
  machine away more and
• 18:51
  more and more in a way that they can't when there's a live
• 18:54
  person sitting next to
• 18:55
  SPEAKER_00: them.
• 18:56
  SPEAKER_01: Yeah, especially if you are talking about young learners.
• 18:59
  SPEAKER_00: Yes.
• 19:00
  SPEAKER_00: Yes, I think that.
• 19:01
  And we're seeing this in a lot of the studies about what is the
• 19:04
  effect size in
• 19:05
  SPEAKER_00: using AI in education.
• 19:06
  The older the students,
• 19:07
  the more they have individual goals,
• 19:10
  the better the outcomes are.
• 19:11
  SPEAKER_00: But when they are very young,
• 19:13
  SPEAKER_00: And they're like being sent to school whether they want to or not.
• 19:16
  And they're being asked to do things that they're not sure how
• 19:19
  it matters.
• 19:21
  SPEAKER_00: That's when it gets difficult.
• 19:22
  SPEAKER_01: Yeah.
• 19:23
  SPEAKER_01: They are not that much goal oriented.
• 19:26
  SPEAKER_01: Many of those interactions are not purposeful.
• 19:29
  So they get frustrated when they don't get the response that
• 19:33
  they expect from the chatbot.
• 19:35
  SPEAKER_00: And we are social beings.
• 19:39
  SPEAKER_00: And we want to interact.
• 19:41
  SPEAKER_00: And especially younger students,
• 19:42
  SPEAKER_00: as we both said,
• 19:44
  SPEAKER_00: younger students react a lot better to a person right next to them.
• 19:48
  SPEAKER_00: It's that physical presence.
• 19:50
  SPEAKER_00: And actually I remember as a tutor,
• 19:52
  SPEAKER_00: I sometimes had to be right there.
• 19:55
  SPEAKER_00: And sometimes when I wanted to know how they do without me,
• 19:58
  SPEAKER_00: I had to actually get up and move away from sitting next to them.
• 20:02
  SPEAKER_00: Because A, they were attending to my body language.
• 20:06
  SPEAKER_00: Because you give cues when they do something that isn't right.
• 20:09
  SPEAKER_00: Even if you're not saying anything, you're a whole, at least in my case.
• 20:12
  SPEAKER_00: And I know I'm very expressive, so I'm aware.
• 20:16
  But I needed to remove myself from that situation and not even
• 20:20
  see what they're
• 20:21
  writing to make sure that it's independent and they're actually
• 20:23
  performing.
• 20:24
  SPEAKER_00: And these are the things that machines really can't do.
• 20:27
  It means that my physical existence next to them mattered a lot
• 20:30
  more than just my
• 20:31
  SPEAKER_00: words or whatever I could do from a knowledge and pedagogy perspective.
• 20:39
  SPEAKER_00: And that's where AI
• 20:42
  SPEAKER_00: Cannot yet do this, especially with young kids.
• 20:46
  SPEAKER_00: So all I'm saying is to all my future,
• 20:49
  SPEAKER_00: past,
• 20:49
  SPEAKER_00: future and present elementary education majors is you're fine.
• 20:56
  SPEAKER_00: AI is not coming for your job anytime soon.
• 21:00
  SPEAKER_00: And
• 21:01
  It's probably not even one of their goals because they're
• 21:04
  replacing so many other
• 21:05
  SPEAKER_00: things that generate a lot of money.
• 21:07
  This is probably not where they want to be,
• 21:09
  but that's aside from all this.
• 21:11
  I think we need to continue and put an emphasis on trying to
• 21:16
  discover how this can work.
• 21:18
  SPEAKER_00: But I think that the heart of the design work is not designing the bot,
• 21:25
  SPEAKER_00: but going back to what you said was the feedback.
• 21:28
  SPEAKER_00: You said the feedback from Canmigo was that teachers did not know how to
• 21:34
  SPEAKER_00: incorporate it or were not using it effectively or whatever.
• 21:37
  The bottom line is that the design work has to be at the point
• 21:43
  that students and
• 21:45
  SPEAKER_00: teachers are meeting the Canmigo and leveraging it.
• 21:49
  SPEAKER_00: Or, I mean, I'm using Canmigo as an example, but figuring out
• 21:56
  SPEAKER_00: What is the optimal way to integrate is where the magic is.
• 22:00
  It's not just about figuring out how you make a bot that
• 22:04
  responds and corresponds
• 22:06
  SPEAKER_00: and all that,
• 22:06
  SPEAKER_00: but how does that work in a legitimate environment?
• 22:12
  SPEAKER_00: And that is a design step that they seem to have skipped over.
• 22:17
  SPEAKER_00: So they designed the thing
• 22:19
  SPEAKER_00: And then they sent it to lots of classrooms.
• 22:22
  SPEAKER_00: And the design work in the mill, how do you optimize this learning?
• 22:26
  SPEAKER_00: What are the conditions?
• 22:28
  SPEAKER_00: And what are the supports?
• 22:30
  SPEAKER_00: It's really where the magic is, theoretically at least, to maximize.
• 22:34
  SPEAKER_00: Even then I'm not sure it works, right?
• 22:37
  But without that work,
• 22:40
  just sending it to classrooms,
• 22:41
  you're going to get this uneven work.
• 22:45
  SPEAKER_00: And especially
• 22:46
  SPEAKER_00: And this is a conversation I had actually later today with somebody,
• 22:50
  SPEAKER_00: saying,
• 22:50
  SPEAKER_00: if you just throw in extra AI,
• 22:53
  SPEAKER_00: as good as it may be,
• 22:55
  into an existing situation with people who have been doing the
• 22:57
  work for a long
• 22:58
  SPEAKER_00: time,
• 22:59
  SPEAKER_00: you're asking everybody in the system to do more work.
• 23:03
  SPEAKER_01: Exactly, yeah.
• 23:05
  So they see it as a burden,
• 23:07
  not as a...
• 23:07
  Yeah,
• 23:08
  while they're still doing what they used to do.
• 23:10
  So if you did everything you used to do,
• 23:12
  and now you have more stuff on top of it,
• 23:16
  SPEAKER_00: You're like, why do I need this extra stuff?
• 23:18
  That's why many teachers push back adding AI assistance for
• 23:27
  their instruction.
• 23:28
  SPEAKER_01: They feel that you're already doing all of those things,
• 23:31
  SPEAKER_01: assignments and quizzes,
• 23:32
  SPEAKER_01: and now we have to deal with an extra thing.
• 23:36
  SPEAKER_00: Exactly.
• 23:37
  SPEAKER_00: And it can't be, it has to be internally integrated in an optimal way.
• 23:41
  SPEAKER_00: And that requires us doing a lot of work to discovering how that happens.
• 23:45
  SPEAKER_00: This is why I did the work in my class this past semester,
• 23:49
  SPEAKER_00: because I'm like,
• 23:49
  SPEAKER_00: I can talk to everybody about how you do this.
• 23:52
  We need exemplars of how do you grapple with this in the real
• 23:57
  world and go from that.
• 23:58
  SPEAKER_01: Yeah, yeah, true.
• 24:00
  SPEAKER_01: And Salcon says,
• 24:02
  SPEAKER_01: AI is going to help,
• 24:03
  SPEAKER_01: but I think our biggest lever is really investing in the human systems.
• 24:08
  SPEAKER_00: Yes, and this is again why I appreciate him.
• 24:11
  SPEAKER_00: He tried,
• 24:12
  SPEAKER_00: they did a very high quality evaluation and they were not afraid to say,
• 24:18
  SPEAKER_01: we have failed.
• 24:19
  SPEAKER_00: This may be
• 24:21
  A dead end or this may just require a much better system and
• 24:25
  he's saying it needs
• 24:26
  to be more in the people and that goes back to how does it
• 24:29
  interact instead of
• 24:31
  SPEAKER_00: replace is probably where the magic is at least for now.
• 24:35
  SPEAKER_00: Systems will develop we will have things but
• 24:38
  SPEAKER_00: You know,
• 24:39
  there was that moment where the first lady stepped out with the
• 24:42
  robot and saying,
• 24:43
  SPEAKER_00: this is the future of education.
• 24:44
  SPEAKER_00: Well, turns out she was wrong.
• 24:46
  I know you're surprised because she's well known for her
• 24:49
  quality of education.
• 24:51
  SPEAKER_00: But yes, yes, I did say that out loud.
• 24:59
  I may have to deal with some issues,
• 25:01
  but the bottom line is that I think we're safe.
• 25:04
  SPEAKER_00: So what I say to many of my colleagues is make sure you teach well,
• 25:10
  SPEAKER_00: because it might replace us in research,
• 25:12
  SPEAKER_00: but it seems like it will not replace us anytime soon in teaching.
• 25:17
  SPEAKER_00: And so job security.
• 25:21
  SPEAKER_01: Job security.
• 25:22
  SPEAKER_00: All right.
• 25:24
  SPEAKER_01: Thank you so much for being with us,
• 25:27
  SPEAKER_01: for listening to us,
• 25:29
  SPEAKER_01: and we cannot wait to record another episode.
• 25:32
  SPEAKER_00: Yes, we'll see you next week and Guy.
• 25:34
  SPEAKER_01: On AI.
• 25:35
  And next week we will have Nikki Schaefer talking about a,
• 25:39
  you guessed it,
• 25:39
  AI.
• 25:41
  SPEAKER_00: And if she finished the dissertation,
• 25:43
  SPEAKER_00: she's talking about some interesting things,
• 25:44
  SPEAKER_00: so tune in next week.
• 25:46
  SPEAKER_01: Hi.