Latest / IB Unplugged / IB Unplugged Live @ The IBGC: The Future of IB Assessment w/ Mathew Glanville

Transcript

IB Unplugged Live @ The IBGC: The Future of IB Assessment w/ Mathew Glanville · IB Unplugged

0:00
Welcome to IB Unplugged, a Kognity podcast for IBDP educators,
0:05
that tackles your submitted challenges and questions by connecting you with
0:08
experts and peers who share their ideas and stories from real classrooms.
0:13
Whether you're a seasoned educator or new to the IB world, each bi weekly episode
0:18
and guest aims to equip you with practical tips and positive inspiration,
0:22
helping you teach with more confidence, creativity, and joy.
0:26
So whether you're walking your dog or commuting to school, hit play and join
0:29
a community of educators ready to learn, laugh, and thrive in your IB journey.
0:35
Hi everybody, and welcome to a special edition of IB Unplugged.
0:38
This one was coming to you from the IB Global Conference in The Hague.
0:43
Yeah, we were super excited to actually do a live version
0:46
of our podcast, and this was taking place as one of the conference sessions.
0:50
So some of you were able to join us.
0:52
We had a really cool breakout room and a whole space together.
0:56
And we got to talk with Matt Glanville, talk about the future of assessments,
1:00
specifically digital assessments,
1:02
kind of breaking some myths apart and talking about the advantages
1:05
of getting to know what people know when it comes to, the assessments themselves.
1:10
And that strategy with the IB.
1:12
Yeah. It was so great to have Matt there.
1:15
I think one of my takeaways at the conference is how popular that man is.
1:19
He is the global conference.
1:21
He couldn't go from like the coffee bar
1:24
to a table without at least four people wanting to talk to him.
1:27
I've never seen anything like that.
1:29
It was it was impressive.
1:31
There was somebody who came by the Kognity booth
1:33
and looked at the we had like the, the poster promoting it.
1:37
Right? And they're like, oh, you got that rock star.
1:39
And he's like, ooh, I've reached rock star status.
1:43
All right, who's John Payton?
1:44
I'm talking about Matt Glanville.
1:46
It's like, oh yeah. Fair enough. Exactly.
1:48
And so we tried to make it, look like kind of our comfortable podcast studio.
1:53
We stole furniture from the hallway.
1:57
We set everything up.
1:59
It was nice to do it live.
2:00
It was nice to see, all the faces, some people have been
2:03
watching the podcast in the past.
2:05
So thank you to those of you who acknowledged that we appreciate it.
2:10
And, yeah, hopefully we have some new people here too.
2:13
Yeah, definitely.
2:14
So, you know, this episode,
2:15
while it was a live recording at the Global conference,
2:18
we're really excited to be able to share it with you here
2:20
as something that you can come back to and revisit time and again.
2:23
And for those of you who weren't with us at the Global conference,
2:26
you're going to hear some really great stuff with regard
2:28
to what the IB is doing to innovate assessment,
2:30
to get a better idea of what students know and can do and why
2:33
digital assessments is something
2:35
that we really shouldn't be too scared of, but actually pretty excited about.
2:38
So we're really excited for you to be here.
2:40
Thanks again for always joining us here at IB Unplugged.
2:42
And we hope that you're able to, take away some great stuff from this episode
2:45
here, talking with Matt Glanville and the future of assessment in the IB.
2:56
All right.
2:56
Well, let's dive in.
2:58
And for those of you here with us live.
3:00
And for those of you listening later on, welcome to IB Unplugged.
3:03
We're super excited to have you here.
3:05
And we are super excited to have Matt Glanville back with us on the show,
3:08
this time, talking a little bit more about the future of assessment,
3:11
specifically digital innovation, AI and the evolving role of educators.
3:15
Welcome to the show, Matt.
3:16
Thank you very much for inviting me back.
3:18
Absolutely. Yes.
3:19
So let's kind of kick things off right away.
3:22
And let's just start with this first segment.
3:23
We want to talk about the transition to digital first assessments.
3:26
We know that's coming down the pipeline in the next few years.
3:29
And I wonder if you could just let's just start with
3:31
what are the common misconceptions when it comes to digital assessment?
3:35
So I think the most important one to tracked is to say
3:38
it's not just about multiple choice.
3:41
It's not just about short answers.
3:43
A lot of the digital assessments people have seen in the past
3:46
have been those sort of easy autogenerated, assessments
3:51
where the IB wants to go with this is actually to have
3:54
really meaningful assessments,
3:55
not just as good as we've got now, but pushing the boundaries beyond that.
3:59
So it's first of all, paper first.
4:03
And then to really think about how can we use
4:05
digital to be interactive, how can we make sure it works well.
4:09
So yeah.
4:10
Misconception one it's all multiple choice.
4:12
Misconception two you do it at home.
4:18
Misconception three is probably that digital assessments
4:23
are less less meaningful and they can be more meaningful.
4:26
Yeah, I think there's so much potential and there's a lot of exciting things
4:29
to change.
4:29
But I'm wondering from your opinion, what are the
4:32
what are the pieces that you want to hold on to?
4:34
What is the
4:34
we don't want to throw the baby out with the bathwater kind of thing.
4:38
Absolutely.
4:39
So again, it's really about that opportunity
4:43
to express what you want in an assessment.
4:49
Either your favorite bit,
4:50
your least favorite bit of those long answer questions where you really get
4:53
to show what you understand in the way you understand it.
4:56
And that's the bit
4:57
that we really need to make sure we keep.
5:01
I'm sure we'll get into it, but you just mentioned
5:04
that you can get a little bit
5:05
more meaning from the assessment, not just the traditional sort of form.
5:08
Can you give us just a teaser in terms of what that means?
5:10
What are you talking about when you say we can get more
5:12
from a digital assessment than the typical assessments now?
5:14
So here's a piece of paper.
5:17
It's got the words on it.
5:18
It's not going to say anything to me once I have a digital assessment.
5:22
If I pick up my phone I can start getting it to ask me things back.
5:26
You can have optionality, you can have video,
5:28
all these things that I can't do with just a blunt piece of paper.
5:31
So there's there's a quicker tease of what we can do.
5:34
You're kind of hinting at adaptive assessment kind of stuff possibly?
5:37
Interactive first we'll come to adaptive assessment definitely later.
5:41
But even if it's not adaptive, that interactive ability
5:44
to be able to say something and fit to ask you that was interesting.
5:48
What more do you think about that?
5:50
And we talked about it being in an exam hall.
5:53
From your perspective, what kinds of support,
5:56
what kinds of infrastructure need to be in place for digital assessment
6:00
to be successful in all different types of IB schools?
6:04
So again, the first thing is don't be afraid, be excited.
6:09
You all use laptops
6:11
frequently and they don't explode very often.
6:14
So a lot of where
6:15
schools are frightened is around some of the logistic elements of it.
6:19
And actually we need to make sure that we've got the backups.
6:24
We can cope if the internet drops out, we've got battery life for laptops,
6:29
which is longer than nearly all IB exams.
6:33
Those are the kind of things that we can do.
6:35
It can be bring your own device.
6:37
So all of this stuff is true.
6:40
Also need to think about that sort of lockdown facility
6:43
to make sure that the computers aren't being used elsewhere.
6:47
So you can't just go and surf the internet.
6:50
We've got to make sure the computers are secure in that sense.
6:52
And we do that through the assessment.
6:55
That's the I mean, that the really the big issues here is logistics.
6:58
But we've run MYP exams for many years.
7:02
It works.
7:03
Schools can adapt, but it does feel like a big change.
7:07
We we spoke about this briefly before.
7:09
But there's also the massive advantage,
7:11
especially for DP coordinator where I don't have to be walking around
7:14
like keeping the exams under lock and key and then like, carefully
7:17
bring them into the room,
7:18
make sure I bring them back because it's all unlocked
7:20
digitally and then, of course, locked down when they're done.
7:21
Right? And you know, the exams are going to be
7:24
they're not waiting by the post, not sending it.
7:26
And also, you know, that they jump back the other way.
7:28
You don't need to worry about posting it and getting lost in the post.
7:32
So there are huge, huge benefits to the logistical side of things.
7:37
Also, just some of the
7:38
invigilator tools that potentially you can use
7:41
to be able to support your students and see their problems coming up
7:45
if it's all connected up.
7:46
So there are there are big logical, logistical advantages,
7:50
although that's not the real reason.
7:51
I'm excited by exams.
7:53
Well, I mean, it is pretty clear that you are excited nonetheless
7:56
about this, this move on where we're going.
7:58
And I guess it kind of fundamentally comes down to this idea of like,
8:01
what are we doing when we're talking about assessment?
8:03
What's the purpose of assessment itself?
8:05
I know that obviously, this is the role you've had at the IB for a while.
8:08
I believe in what we get to do here, and I know there's some people out here.
8:11
I've been able to have conversations with already where we're kind of thinking,
8:14
you know, is AI coming in kind of breaking assessment?
8:17
What are we doing with this thing? What's the purpose of it at all?
8:20
So I wonder if we could just start like do you believe the purpose of assessment
8:23
the IB has shifted a little bit when we go to digital,
8:27
assessments or what's, what's kind of the main role of assessment still.
8:30
So, the fundamental role of assessment
8:34
is to understand what the student can do.
8:38
And in the formative space in your classrooms
8:41
that so you can understand how to teach your student better.
8:44
And then the summative space, which is where I tend to say,
8:47
it’s about being the least bad way of differentiating between students.
8:52
It's a great line for your resumé.
8:55
I mean, I do honestly recognize the challenges
8:58
and the flaws with a sort of summative assessment system.
9:01
But also, I know that if you're going to identify
9:04
which are the best students to progress, it's a lot
9:07
better than all the other methods that have inherent biases.
9:11
I spend many nights worrying about fairness and comparability
9:15
and how to remove barriers.
9:16
And I know that interviews obviously don't spend the same length of time
9:21
to make it happen.
9:22
So it's a really strong thing.
9:25
But, one of the limitations of our paper
9:29
exams is what we can assess.
9:32
And so with the digital can we assess more of what we value?
9:38
As an IB, I always try to focus on assessing
9:41
what's important, not what's easy to assess.
9:44
But, I can do an even better job of that with digital assessment.
9:48
Can you tell us more about what you mean
9:49
when you say what's important, what we value?
9:52
So if we think about what,
9:55
why are we educating our young people?
9:58
What's the what's the end goal?
9:59
How do we want them to move?
10:02
Is it to be able to regurgitate loads of facts?
10:05
Yes.
10:09
I respectfully disagree with you as the principal about that.
10:13
Is it about being able to understand and evaluate things and again,
10:17
regurgitating facts
10:19
I couldn't assess very easily using multiple choice in the short answer,
10:22
their ability to form a coherent argument, to evaluate a situation,
10:28
to express different viewpoint, to the very sort of TOK kind of way.
10:32
That's where I think we truly trying to achieve with an IB education.
10:37
And that's where
10:39
I think that having that interactivity is really good.
10:43
The best IB lessons are those where the teacher and the student are talking
10:48
both to teacher and to each other and exploring things together.
10:52
That's well, I really see sort of buzz out of the classrooms,
10:54
and I'd love to see a little bit of that buzz coming into the exam space as well.
10:59
Yeah.
11:00
We have a question from somebody in the audience and I know we've previously
11:04
talked about the MYP, e-assessments, and they wanted to kind of know
11:08
what is the
11:09
what are the similarities,
11:10
what are the differences between what you hope the DP exams,
11:13
digital assessments to be compared to what the MYP ones are now.
11:17
So the biggest difference is that
11:20
the MYP e-assessments are optional.
11:24
So we were able to create something that was really forward
11:28
looking and exciting and using that as many of the opportunities we had.
11:33
Whereas with the DP, we're going to need to run some paper along a side digital.
11:38
So during these early years of the transition, I've got to be
11:43
I've got to be comparable between the paper and the digital. So
11:48
unlike
11:48
the MYP, I could start with a big bang of really exciting stuff.
11:51
I'm going to have to be careful as I run through.
11:56
When we eventually reach the point of how, all of DP exams being on screen,
12:01
whether in the subject or across the board,
12:04
then that's when I can start to really build on the MYP experience.
12:08
I mean, the MYPs, they've knocked off ten years old now, so technology
12:12
and opportunity has moved on. But that's a good blueprint
12:15
for the kind of thing I'd love to be able to do.
12:17
That's kind of the idea within starting with more of the language subjects first,
12:21
then rolling in a little bit more to the the complicated maths and sciences stuff.
12:24
Right?
12:27
Absolutely.
12:28
Most students use word processing for many of their tasks in literature,
12:33
so moving that on screen isn't a challenge.
12:36
Whereas once you get to see the science and the maths, that's
12:39
where we need to manage some of the more complicated transitions.
12:44
And that's also where we need to give schools the most opportunity
12:47
to upskill their students that they're used to working in that way.
12:52
And exams shouldn't be a different experience
12:54
in your student's classroom.
12:55
Sure, sure.
12:57
I mean, another question from the audience that actually fits right now
12:59
is to what extent
13:01
should we be concerned about maybe a handwriting decline or a loss
13:04
because we're kind of moving everything digital, especially given
13:07
considerable research showing the link between handwriting cognition.
13:10
What are kind of your thoughts in that space?
13:13
This is a really interesting element for me.
13:17
I know throughout my business life
13:20
I've never submitted a handwritten piece.
13:23
So I report to my chief exac.
13:27
We just saw a big gesture from the audience, by the way,
13:29
that was like, that's what I've been saying.
13:34
I should be looking that way more than the good part of you.
13:36
Yes, absolutely.
13:38
So I think now there are there are advantages, as you say cognitively,
13:41
to be connecting the handwriting to the the learning process.
13:45
I don’t want to get rid of that.
13:46
But we do need to prepare students for the future.
13:49
And I really don't think it's fair to make kids write for three hours in the
13:52
history exam. That needs to be a thing of the past.
13:55
Yeah.
13:55
No, I mean, that's something that we've we've also talked about of this idea of
14:00
the assessment shouldn't be this alternate universe
14:03
that is this weird exam hall where suddenly everybody has their,
14:08
exam books and their hands are all crammed thing where,
14:11
a lot of you said that you use Kognity, you're using a platform.
14:14
And if you've used our exam style questions,
14:16
it's like, okay, now everyone stop using the platform and get your paper out.
14:20
And we're going to separate everything.
14:22
That ecosystem you're talking about is so powerful.
14:24
If you're learning and you're assessment and everything is all happening
14:29
in the same way, and then it's stopping the cognitive,
14:32
I would say barriers of I'm used to doing everything on my computer.
14:36
I'm used to having an either adaptive or interactive experience.
14:40
And now I just need to sit here with a plain piece of paper and start writing.
14:44
Absolutely.
14:45
I don't want anybody to feel scared that suddenly their classroom
14:49
has got to change completely overnight, because I know there's a big change there.
14:53
But also we know that young children,
14:56
young children, children, full stop, come in the home.
15:00
They use digital technology.
15:02
That's their life now.
15:04
And then they come into schools and sometimes it feels like
15:07
it's sort of I'll politely say, 20 years out of date.
15:11
I could probably say more than that.
15:13
So we we want our education to reflect the real world,
15:17
and we need our assessments to reflect how education works.
15:21
Well, and to that end, then as we kind of keep thinking about rethinking this,
15:24
the purpose of assessment itself, how do you feel like assessment
15:27
can better reflect learning process, not just the outcomes?
15:32
So, there's a I mean, that's part of where
15:36
the gap between, formative and summative come in.
15:41
If I'm trying to measure somebody's achievements
15:44
at the end of their course, then that's what I'm interested in.
15:49
To give the sort of,
15:51
if I'm measured on how much improvement I've made on learning the flute,
15:56
then that's not really fair to give me the same grades
15:58
as somebody who can actually play the flute, even though for me,
16:02
the ability to make a sound output is great progress.
16:05
Having said that, with that assessment
16:08
as learning and as part of the experience, we really should be focusing
16:13
on asking students to reflect during their assessment process.
16:18
It becomes part of that thinking.
16:19
And again, digital can really help with that.
16:23
It's that little bit more adaptability, can offer feedback.
16:26
Students can ask similar questions when the students need it.
16:30
Again, it can support that learning process, and it also can test
16:34
that learning processes in the way that paper can't.
16:38
One of the things we try and do is one of our
16:41
pilots is saying, okay, here's a situation.
16:44
Now we're going to change that situation.
16:47
How do you adapt it?
16:48
Which, of course is exactly how the real world works.
16:52
You think you've got it sorted, then somebody throws you a curveball
16:55
and suddenly you've got to react to that.
16:56
There's no teacher here who's ever had that experience.
17:01
Are you an IB educator wanting
17:03
to improve your practice and needing a just a bit more support getting there?
17:06
Have you ever wanted to learn more about a specific aspect of the DP program, or
17:10
ask a question to an educational theorist about a specific IB aspect?
17:14
IB Unplugged is the place for you.
17:16
Simply visit kognity.com/ibunplugged to submit your question to us,
17:22
and have your question explored in more detail in an upcoming episode.
17:25
And now, back to the show.
17:28
You kind of hinted at it there with regard to
17:31
how do I evaluate this student kind of comparatively against this other student?
17:34
And I think it really brings about this tension
17:36
of how do we reconcile high stakes external assessment
17:40
with the goal of meaningful formative learning, school culture, IB mission.
17:45
You know, we're kind of in this balance between the two where I feel like
17:48
throughout the DP program or really all IB programs where mission driven,
17:52
we're doing all this cool stuff to build an inquiry.
17:54
And then we kind of hit almost a ceiling of it's high stakes,
17:57
and we've got to get into universities and that sort of stuff.
18:00
How are we navigating that now, and how do you see digital assessments
18:04
in that space? I mean, you're absolutely right.
18:06
And one of the challenges I've got is what gets assessed is what gets taught.
18:11
So one of my duties is to try and make sure what I'm asking
18:16
when I assess you is actually what we value.
18:20
Again, coming back to that problem solving
18:23
testing as opposed to knowledge recall.
18:26
So if you've spent your entire sort of revision learning how to problem
18:30
solve, that's probably what I wanted from the IB education anyway.
18:34
That is harder to assess reliably, and where I can use
18:40
perhaps digital assessment is to be able to give people comparable situations,
18:45
but also adjust to their own situations a little bit more.
18:50
One of the I mean, there's a whole range of access and inclusion
18:53
where maybe the IA can change the reading age to meet the student,
18:57
if that's not what's important without it being the one thing
19:01
as well as the so lighted obvious things like font size and font color.
19:04
And you know what I can make things larger.
19:07
Now I'm getting a little bit older.
19:11
Don't get me started on that.
19:14
So I think that's where we've got a great opportunity
19:16
to start measuring those things that are really important.
19:20
The C word collaborations.
19:22
Another really good challenge, because currently
19:27
I can't ensure that every student has the same
19:30
collaborative experience to try and build on that.
19:33
Maybe in a digital world with a touch of
19:36
AI thrown in, I can give them the same group to work with.
19:39
Therefore, I can accurately measure how
19:42
well they've coped with that group and be comparable across students.
19:47
I love that because I think that in a lot of sessions,
19:50
other podcast episodes, there's been a lot of talk about
19:53
assessment versus skills and skills based learning
19:57
and all of the things that the teachers, I'm sorry, actually want to do the teaching.
20:03
And of course, it feels like there's a lot of sentiment of the assessment
20:07
blocking them from going deeper into those kind of things,
20:12
like collaboration, because that hasn't been so.
20:14
It seems like that pulling those skills, pulling in the ETL, really actually
20:20
connecting it all is one of the goals of having this digital assessment.
20:24
Absolutely.
20:25
I think that's a really, really important point,
20:27
that it means that we can start to assess
20:29
some of the things we've never been able to do,
20:32
because it requires that that interaction that you can't get from a piece of paper.
20:37
Although also I need to say that
20:40
not everything that's important should be measured.
20:43
Caring and empathy is incredibly important.
20:46
I do think we should have a grade it.
20:48
Just moment ago, you mentioned, with a touch of AI,
20:51
and I kind of felt
20:53
in the room, people go, oh, what do you mean when you say touch of AI?
20:57
Can we can we talk about opportunities and risks and governance?
21:02
And, you know, I think we've gone
21:03
to a few sessions, throughout the conference this far
21:06
where AI seems quite promising, but then I know I was part of one yesterday,
21:11
there's been a couple of sessions where it's been mentioned,
21:14
and so I guess, yeah, I want to step into it
21:18
almost with a little bit of trepidation, if you will.
21:20
But talk to us about that. What are the opportunities?
21:22
What are the risks when we're talking about AI and assessment?
21:24
Okay.
21:24
So I think, first of all, a lot of the conversation about AI
21:29
in assessment has been, oh, students are cheating with it.
21:33
And throughout all my presentations I've tried to say, actually, no, let's
21:36
go beyond that.
21:38
Let's recognize AI exists out there and see what it does.
21:42
But AI to support learning is great.
21:45
AI to replace learning is is not so great. In the assessment space
21:51
we can also use it to be more interactive and more responsive.
21:56
This is a collaborative thing.
21:58
This is the collaborative thing.
22:00
This could be if it's explain a concept
22:03
to a fellow student, a great exercise in the classroom.
22:07
You can't do it in a paper exam.
22:09
But with AI in a digital world, you can do that.
22:13
You can cite the student.
22:14
That's very interesting. What more can you tell me?
22:16
You can prompt the student to give their full opportunity.
22:20
And ultimately getting to maybe something around that,
22:26
language acquisition.
22:27
You're actually interacting with an avatar
22:29
rather than having a very sort of stifled.
22:32
So there's there's one way to allow a student to be responded to
22:37
and the other way you mentioned was adaptive assessment.
22:42
So at the moment, because my assessments have to cover everything,
22:47
for the average student, the first third is just an opportunity
22:49
for them to make a mistake and lose a mark.
22:51
The middle is where they're really showing what they can do,
22:54
and the last half is that they cry because they can't do it.
22:57
If they can come out of the exam feeling dreadful. How much better,
23:01
if we could just focus on that middle part.
23:04
And there are ways we can do this at the moment.
23:06
We could ask the teacher to guess in advance
23:08
which bits of the question they could do, but so much better
23:11
if we can adapt the assessment as the student goes along.
23:16
Now, there's a lot of work to be done with the communities
23:19
to make people confident that that's really giving everybody
23:23
a fair experience, because at the moment, face validity.
23:26
Everyone gets the same exam.
23:28
That's fair to the extent.
23:29
Well,
23:29
and from that same exam, you've got to be able
23:31
to distinguish your sixes from fives and so on.
23:33
And sometimes it comes down to just one small bit, right?
23:37
Absolutely. I mean, it can be, yes.
23:39
We need to make sure
23:40
that all the questions are doing what they should be doing.
23:43
And that's a challenging thing to set up in advance.
23:47
But we could do it so much better.
23:51
So that's, that's the real student facing.
23:53
And there's a whole pile of stuff behind the scenes using AI to support
23:57
authors mark better as I create better exam papers,
24:01
as inspiration.
24:03
Excuse me.
24:04
Using AI to better quality check papers.
24:07
Using AI to support and quality check marking.
24:11
I'm not suggesting
24:12
we get rid of human markers at all.
24:14
Yeah, what's the examiner in this space then?
24:16
Yes. So,
24:18
we're doing a lot of research at the moment around.
24:21
And it's not just that IB, across the sector around how effective
24:24
AI is at marking student work.
24:29
The initial responses are looking pretty positive,
24:32
but there's no way I think, that society
24:35
is ready for us to have pure AI marking yet.
24:38
So certainly my vision is we have human markers market once
24:43
we then have AI marking, and then the IB can say,
24:47
well yeah, that that agrees, that agrees or that one disagrees.
24:50
So can we give that to a third and do the human marker to decide
24:54
whether it was the AI or the human being that was wrong.
24:56
So it's kind of a bridge between a principal moderator.
24:59
If you're a principal examiner in the meaning to anyone.
25:02
Well, it's about identifying
25:04
those rare occasions when the human examiner makes a mistake
25:08
and they do very occasionally, but we can get them.
25:11
Yeah, I think that's been a theme in some of the sessions
25:13
I've been sitting in this weekend about the partnership element of it's
25:17
not all going to AI, it's not all staying human,
25:20
but how do we make it a really great partnership?
25:22
And I think that, yeah,
25:23
there's been a lot of positive studies on being able to get really good,
25:28
results from AI marking, but I think that
25:31
the examiners, I mean, some of them have been examining for 15, 20 years.
25:35
There's, that element I think doesn't go away either.
25:39
Absolutely.
25:39
I mean, it's
25:40
sometimes called human in the loop, but yes, we absolutely need to keep that.
25:44
And again, moving slightly sideways, I think that's also really important
25:49
in that teaching environment as well.
25:52
AI can support teachers, but they mustn't replace teachers.
25:57
Exactly.
25:58
It can be be great for dealing with that class of 30 where you've got 1 or 2.
26:02
You need to just say pat on the head, but you still need
26:05
to do some of that marking to gain the insight for that student.
26:09
You were talking about, sort of the student facing AI bit,
26:12
but there is a whole bunch of stuff going on, you know, on the back end of it,
26:15
I guess for behind the scenes, if you will.
26:17
And we got a question from our audience
26:18
here, which is, you know, what approaches are you using to green assess
26:22
your use of AI to have minimum use but maximum impact?
26:26
I mean, that's an amazingly great question.
26:29
My principle here is that we only use AI
26:33
when we need to use AI, and I hear a lot of people blurring it.
26:38
So multiple choice marking.
26:40
We don't need AI. We can do that automatically.
26:43
Actually looking at school results and
26:47
or examiner marking to make sure that it's looking okay.
26:51
I don't need to use AI for that.
26:53
I can use standard statistical processes to do that.
26:57
Marking an extended essay.
26:59
Well, actually, that's where I really think the AI is needed.
27:03
Providing that interaction
27:05
for the student, that's what I think
27:08
artificial intelligence is really needed.
27:10
So that's that's my approach at the moment.
27:13
It's about sort of using it when it's going to really add value
27:16
to the assessment process or the student experience.
27:20
Do you mind expanding on that a little bit with regard to the extended essay?
27:23
And the AI is sort of examining there?
27:25
You know, one of the things we want to look into is how can assessment
27:28
design itself evolve to be more AI resilient in certain spaces?
27:33
And then at the same time,
27:34
as we spoke about already use AI to help bring out more from the student as well.
27:38
So I wonder, as you mentioned, what does that mean?
27:44
So, the context I actually meant it was that
27:48
I can't create a simple program to mark, extended essay.
27:53
You've got to look at the whole argument together,
27:57
the whole essay together to understand how all the bits link up.
28:01
And that's something which a human being does well.
28:05
AI can do well.
28:07
And so the more linear marking processes can't do well.
28:11
But I think giving voice would invite me to think about
28:14
how does the extended essay still exist in a world where..
28:18
Well, fundamentally,
28:20
we still need our young people to be able to be interested
28:25
in the topic, work with that topic, examining that topic.
28:30
This is where the role of the
28:31
teacher or supervisor comes in because, yes,
28:35
a student could just ask ChatGPT to write
28:39
their extended essay for them, but to what end?
28:43
They've learned nothing.
28:44
That's using AI to replace.
28:46
So we tell students how you can use it.
28:49
Yeah, it's great for investigating.
28:51
Thinking about the different questions.
28:54
Great. Choosing it. No.
28:56
Getting a sense of what the sources look at.
28:59
Yep. That's a good use.
29:01
Summarizing it and telling you what they say?
29:03
No, you've got to go and read it for yourself.
29:06
And good teachers will know they'll be working alongside the students.
29:10
So I see that also they'll be able to have that conversation
29:14
with the student at the end.
29:16
And if a student really understands what they've written in the whys
29:19
and they can answer questions about it, then that's that indicates
29:24
what they've learned.
29:26
A number of people have talked to me about what
29:28
should we have an oral exam at the end of extended essay?
29:32
And that's a really interesting way of thinking about this going forward.
29:36
But I know oral exams puts a huge burden on schools.
29:41
So at the moment I'm saying that's a great tool
29:44
for the school to use, if they've got a doubt.
29:47
But maybe in the future we could use AI to run those oral exams for us.
29:51
So it's no longer a great burden.
29:54
That was such a change in tone of voice.
29:58
Sounded worried.
29:59
Yeah, yeah.
30:03
I'm Alex, I'm
30:04
an MYP coordinator at Middle School
30:07
and one small change that I made in my teaching
30:13
that had an immediate big impact was a focus on documenting
30:17
the learning process in my units in different ways
30:21
so that students could monitor how they were doing and reflect
30:25
and see their progress, and so that I could see it constantly
30:29
as well.
30:32
No, I think that's really interesting.
30:33
And I think that that's a really great way of explaining
30:37
how we can kind of move past the and we shouldn't leave academic integrity,
30:41
which we've talked about also, but it's a way of saying
30:44
this conversation goes beyond just are people are cheating or not.
30:48
It's how are we actually doing that?
30:49
And then I think that ties into assessment literacy as a bigger conversation.
30:55
How do we talk about assessment literacy beyond just exam prep?
30:59
And yeah, I want to hear kind of your thoughts on
31:02
what some of the most powerful things that, coordinators and school leaders
31:06
can bring back to their schools to help in that culture of assessment literacy.
31:11
So, I mean, the most important thing is to actually talk
31:14
to people about it and talk to the students about it.
31:19
We all know that getting students to evaluate each other's work
31:24
is a really powerful way of helping them learn,
31:28
but you can also use that as an opportunity
31:31
to talk about what a good assessment means.
31:35
It's always great fun to ask students to come up with
31:37
really stupid ways of assessing important things.
31:41
And that be a good, fun, entertaining
31:44
20 minutes of a lesson and you can sort of print. Yes.
31:47
How would we assess?
31:49
I don't know whether something is an elephant or not.
31:51
Oh, would use a multiple choice test for the elephant.
31:54
And if he couldn't answer it, it probably isn't.
31:56
But there's opportunities like that.
32:00
And then it wouldn't be me if I didn't say.
32:03
Also talk about the validity chain.
32:06
Which, again, apologies.
32:08
I think he said every single one of my presentations,
32:10
but think about the elements that make a a valid assessment
32:15
the reliability, the fairness, the comparability, the manageability.
32:20
All of these elements have to come together
32:24
and you can get some really good conversations going.
32:27
And you can also then start applying that to your own school experiences.
32:33
There's no point in having a formative assessment that is strongly biased.
32:38
So if I if I stick by the instructions on what to do six feet above the,
32:43
the desks or the tall kids will be able to do the work, but all the short kids
32:48
will have to cheat and get somebody else to read it for them.
32:51
See the
32:51
exercises that just, again, entertain the kids,
32:54
but also get the parents and the teachers thinking about it.
32:58
I like that a lot because it helps students
33:00
and for that matter, as you said, parents really understand
33:02
a little bit more of the purpose of why we are trying to assess and why that does
33:06
need to be a space for it.
33:09
A more nuanced, I guess, appreciation of how students are learning, right?
33:13
And to really understand what they're doing.
33:14
And so far, I mean, within the conversations
33:16
we've been having, I can just tell so much of this.
33:18
And the forward thinking movements we're having in the next few years
33:21
are all about trying to just understand what the students know a lot better.
33:26
For me, that seems like a pretty encouraging thing,
33:28
especially as we're in where where students could maybe falsify
33:32
what they know, or they could use other ways of showing that,
33:34
but instead, it seems like we're moving in a direction
33:37
where we really know what students actually know.
33:39
And it sounds like that's kind of the main point of what's going on. Right?
33:42
I'm going to be naughty to say it's what students understand rather than what
33:46
students know, or it's what students can do, but not what they know.
33:49
Oh, right. Okay.
33:53
Yeah, yeah.
33:55
So we've kind of covered that, I guess, with regard to
33:58
AI supporting teaching and learning with assessment.
34:01
But I guess, is there anything else that you feel like we can use
34:05
AI meaningfully to support teaching and learning in the context of assessment
34:09
and not distort the learning in the context of assessment?
34:12
So again, the first thing to say is that,
34:16
teaching's a really hard job
34:19
and you spend hours doing lots of different things.
34:22
Let's use AI to remove
34:25
some of those tasks so you could spend more time
34:28
doing the things that you enjoy and that make you a great teacher.
34:32
And that's not a one size fits all.
34:34
If you're a teacher who loves doing lesson prep,
34:37
do your lesson prep.
34:38
If you're a teacher who loves writing reports, you're very strange.
34:44
But use it in the way that you need to.
34:48
But also, I think the real big thing here is
34:51
we've got to help students understand
34:55
the ethical and best uses of AI.
34:59
It's not going away.
35:01
It's going to be part of their lives.
35:03
For a long time.
35:05
We already see situations where students are using it
35:08
in ways that they are amazed. their best buddies,
35:11
a sort of apps, and you see them spiraling down.
35:15
Let's explain to them why that's not the best idea.
35:18
Let's help them understand that knowing something because the AI told you
35:23
is based on, AI knowledge is sort of a mob
35:28
view of sort of what is the majority of people think.
35:31
But you can also go and ask somebody who's an absolute expert in something
35:34
or somebody who has a particular viewpoint,
35:37
and, it’s fairly obvious, other people with different views
35:41
can also be right.
35:42
So go and learn about it, listen to people.
35:45
And I think that's where we can really help students
35:49
use AI responsibly and beneficially in the future.
35:53
And also teachers and schools and assessment organizations.
35:58
Yeah, yeah.
35:59
I think that ties in really nicely to another question that we were asked
36:02
from the audience in when you use AI in assessment,
36:06
there's an inevitable bias that exists within AI.
36:10
How do you, especially in an international school context, how do you address
36:14
that bias and kind of address that conversation with students as well?
36:18
One of the huge challenges of being an international organization
36:23
is that, cultural,
36:27
experiences in different parts of the world.
36:31
My, my nice, clean version of this is if you write about a greenhouse,
36:36
will you get everybody recognizing that you mean a house that's painted green,
36:41
or will some people think you mean the house made out of glass?
36:45
And that's true across the world.
36:47
So it we've got to be careful.
36:49
We set up our exam teams in order to make sure
36:53
that we don't have those cultural blind spots.
36:57
And that's where using AI in that space we have aware of it will watch
37:03
it will spot it, will consider it, will think where where's the gaps here.
37:08
Yeah.
37:09
And then ditto when we're using it for marking.
37:11
We will make sure that the way we train
37:14
it is based on an awareness of the bias.
37:19
AI itself isn't biased.
37:21
The evidence and information is learned from is biased.
37:24
Right.
37:25
And so but that does mean that if you teach the AI
37:29
based on a better cohort information, for example, a pile of history exam papers
37:35
that we've put together for it, you can be careful of that bias.
37:40
Because I guess you've already had to, in a way, teach the examiners about it,
37:44
because every examiner is going to have their own bias as well.
37:47
It's going to be easier to mark, examples that are coming from
37:51
experiences that are coming from you as well.
37:53
So it's a similar notion.
37:55
Absolutely. Yes.
37:57
We we've had to work with our examiners to help them understand.
38:00
Now in a lot of our examiners are IB teachers.
38:02
So that's a that was a very easy conversation.
38:05
But being able to reflect and recognize yourself
38:09
when the context feels familiar and when the context feels unfamiliar
38:13
is a really important element of it, both as examining badges for assessments,
38:18
but also, again, as a as a student learning about it.
38:23
One of the positive elements of
38:26
AI bias
38:27
is it's a really good starting point for getting young people to realize that
38:31
not everything they read from ChatGPT is true.
38:35
And then maybe not everything they hear on social media is true.
38:38
Maybe they need to be that critical listener, that critical receive
38:41
information wherever they get it from.
38:45
Well, so I mean, obviously we're kind of coming to the close
38:47
of of the episode right now, I guess, as it would be.
38:49
And what I'm taking away is that there's
38:51
lots of great possibilities here in the digital assessment space.
38:55
It sounds
38:55
like we're really finding a good way to know what the student understands.
39:00
There we go. Yes.
39:01
And, and it sounds like we're we're working with AI in ways that are going
39:06
to help enhance what this thing knows and help what we know about them as well.
39:10
Without all the possible negative downsides to it as well.
39:13
And I'm sure there's a much larger conversation
39:15
we could have in that space and maybe come back for another one.
39:17
But as we, as we come to a close, I guess, of, of this episode,
39:21
if you had one bold change
39:24
to how assessment works in the IB over in the next five years,
39:28
is it just digital assessment or is there something more? Digital assessment
39:32
is the opportunity to make that change.
39:36
Because what I really want is I want assessments that are interactive
39:42
for the student, and I can't do that with a piece of paper.
39:46
I will eventually be able to do that with digital assessment.
39:50
Matt, thank you so much for joining us for the show.
39:52
Thank you everybody here. Please help us and thank Matt.
39:55
Thank you all.
39:59
IB Unplugged is a Kognity podcast
40:01
hosted by John Patton and Courtney Marline and produced by Katilina Gaffney.
40:05
Be sure to subscribe or follow and share with your fellow IB educators.
40:09
If you'd like to submit a topic, request or a question, please do
40:12
so on kognity.com/ibunplugged.
40:15
Let's keep the conversation going
40:16
and support each other in teaching with confidence, creativity, and joy.
40:20
See you next time.