Latest / IB Unplugged / What’s Next for Assessment? Inside the IB’s Digital Strategy w/ Matthew Glanville

Transcript

What’s Next for Assessment? Inside the IB’s Digital Strategy w/ Matthew Glanville · IB Unplugged

0:00
Welcome to IB Unplugged, a Kognity podcast for IBDP educators,
0:05
thattackles your submitted challenges and questions by connecting you with
0:08
experts and peers who share their ideas and stories from real classrooms.
0:13
Whether you're a seasoned educator or new to the IB world, each bi weekly episode
0:18
and guest aims to equip you with practical tips and positive inspiration,
0:22
helping you teach with more confidence, creativity, and joy.
0:26
So whether you're walking your dog or commuting to school, hit play and join
0:29
a community of educators ready to learn, laugh, and thrive in your IB journey.
0:35
Hi everyone!
0:35
Welcome to another episode of IB Unplugged.
0:38
This week we get to talk with Matthew Glanville,
0:40
who's the Director of Assessment at the IB.
0:43
He's been there for over a decade
0:44
and he is leading the charge with everything from the practice,
0:48
the policy and the philosophy of assessment and specifically
0:51
right now moving from paper to digital assessments.
0:54
We spoke quite a bit about that roll out, the timeline, all that sort of stuff.
0:58
In this episode, some really exciting things for you as well.
1:01
And Courtney, I know from your side as a edtech provider for Kognity,
1:04
there are some really exciting elements from this conversation with Matt as well.
1:07
Yeah, definitely has a lot of, you know, eventually
1:11
the DP is going to be rolling out into digital assessments.
1:15
And this is something
1:16
that we're really excited about because we've seen the advantage,
1:20
tenfold of really having that intelligence and that data behind
1:23
formative assessment and being able to then
1:27
move summative assessments into the digital space.
1:29
It just gives so much potential and exciting ways
1:33
of working with your students and really being able to bridge that gap
1:37
between what you're doing in your classroom,
1:39
what the teaching and learning looks like, and then putting summative assessment
1:43
into that same basket so we can really get better insights
1:47
on the students on their work, better ways of, teaching and testing.
1:52
And yeah, I'm really excited about a lot of the,
1:55
the things that Matt's been talking about in this episode.
1:57
He'll be discussing not just about the logistics,
2:00
about how that transitional work, but then what he's excited to see progress
2:05
and really evolve as we move towards this digital assessment space.
2:10
Yeah.
2:11
It was very obvious that we're talking about yes assessment of learning.
2:14
But also we were seeing the very many possibilities of assessment
2:17
for learning and how we can use that to drive, our student learning
2:20
throughout the entire curriculum, not just at the end of it as well.
2:24
And as a DP coordinator and, you know, somebody who's spoken with other,
2:27
educators in the IB,
2:30
I know there's some fears and some concerns
2:32
about transitioning to digital assessments,
2:33
but this conversation left me pretty excited
2:35
about the move over to digital assessments.
2:37
And so I hope that it will for any of our listeners right now.
2:40
We've got some really exciting things
2:41
coming up in this episode, so we hope you enjoy this week's
2:44
IB Unplugged episode, talking with Matt Glanville about digital assessments.
2:54
Hi Matt, welcome to IB Unplugged.
2:56
Thanks for joining us today.
2:58
Thank you very much for inviting me.
2:59
We're really happy to have you here today talking about digital assessments.
3:03
And we want to just kick off first with getting a little bit of your background.
3:07
So could you tell us why you've picked a career in education?
3:12
Fundamentally, I love seeing people learn
3:15
that sort of moment of recognition and to, oh, that's how it all gets together.
3:20
That for me, has always been the absolute delight.
3:24
There's something about being the person that helps guide and mentor, particularly
3:29
young people, but also members of staff within the IB into that realization.
3:33
That is just great.
3:35
I also think with the world as it is now, it's really important
3:38
we have more young people who question and think about different perspectives.
3:43
And as part of education, that's the one thing we can really do
3:46
to really help embed that curiosity across all of our young people.
3:51
Yeah, I love that.
3:52
Now, you've obviously
3:53
you've been at the IB for over a decade now working with an assessment there.
3:57
I wonder if you could talk to us.
3:58
What have been some of the highlights of your career with the IB?
4:02
I have a huge highlight twice every year
4:05
when we get the NYP and DP results out, and students are happy with it.
4:09
So I mean, obviously within that assessment, well, that's the
4:13
the element that we are building up to because we understand how important it is.
4:18
Beyond that, over the past decade I've really been able to see
4:22
that we can get our organization to understand and articulate
4:27
what makes meaningful, valid, exciting assessments
4:31
and not just sort of this is what we've always done, but this is why,
4:35
and this is what should look better going forward.
4:38
And obviously digital assessment is a key part of that.
4:41
And then I do just want to say I've been really proud and privileged
4:44
to be able to sort of
4:45
help IB take that forward looking view on artificial intelligence.
4:50
I felt that we came out and said, it's here, let's embrace it.
4:54
Let's teach ethical use at a time when maybe some of our competitors
4:57
were trying to stick their head in the sand
4:59
and say, no, no, it doesn't exist.
5:01
Let's just pretend it isn't there. Yeah.
5:04
I especially like that part of meaningful, valid and exciting assessments.
5:07
And I think we can all agree with the first two.
5:09
The third one might take some convincing,
5:11
for some of our students especially.
5:12
These can be exciting ways to demonstrate your learning.
5:15
Opportunities of digital assessment.
5:16
Again, if we start looking
5:18
at some of those NYP assessments as opposed to the digital
5:20
DP ones, I have had students come back from that say, you know what?
5:24
These were quite good.
5:25
These were actually interesting, maybe less exciting, but certainly interesting.
5:29
Good. Well, we can always aim for that as well.
5:32
That's okay.
5:33
Question though
5:34
I think, it would be great if you could give a bit of, like,
5:38
a background of what your role at the IBO is. Like,
5:41
what is it generally look like in your team and with you?
5:45
Okay, so I'm director of assessment, which fundamentally means I'm responsible
5:50
for making sure that all of the assessment sessions happen,
5:53
that our approaches, our policies to assessment are absolutely best practice
5:59
around the world, that we're offering students the right level of access,
6:02
ability and inclusion policies.
6:06
It picks up all of the sort of paper
6:09
creation, pay for editing, paper marking,
6:13
all of that element, as well as the academic integrity.
6:17
So what is acceptable?
6:18
What is not?
6:19
Where do we draw those lines?
6:23
There's a lot of sort of maths behind a lot of that.
6:26
There's a lot of thinking around
6:27
how we set ground grade boundaries, what a fair exam means.
6:31
But one of the most fun things
6:33
about being the International Baccalaureate is it is international.
6:36
So we have to consider what it looks like beyond so
6:39
a key question the past year was I think about is
6:42
and if we translate a question from English into French,
6:46
does the meaning of that question actually change?
6:49
If we then have the same question
6:52
in northern Europe or southern Africa, is the context
6:56
so very different that it allows or creates barriers for students
7:01
and that sort of cultural and linguistic interdependencies
7:05
really quite a challenge that the IB faces that maybe national systems don't.
7:10
Yeah, I can imagine that's an incredible challenge.
7:12
I mean, we have students at our school
7:14
who are, you know, from different backgrounds, different languages,
7:17
and sometimes you'll see them come out of an assessment.
7:19
They're just asking about one, one word.
7:21
What is that word?
7:22
And I
7:22
tried to do the direct translation of it, but it didn't seem to fit in this thing.
7:26
And, I can imagine that's got to be an absolute challenge,
7:30
when it comes from your side. Also between the same language.
7:33
So again, English people will obviously joke
7:35
between the American English and British English,
7:39
but the same is true of European Spanish and South American Spanish,
7:44
and other combination of languages around the world.
7:47
It's such an interesting area to look at. Yeah, separated
7:50
by a common language. Absolutely.
7:54
Exactly. Yeah.
7:54
Well, that's
7:55
that's interesting because it's not just the philosophy of assessment,
7:58
it's the policies of assessment, it's the practices of assessment,
8:00
sort of all three that you really, really pushing through.
8:03
So you wear a big hat, that's for sure.
8:05
Yeah.
8:06
What do you see the future of education looking like?
8:10
I guess not just education, but the future of teaching as well.
8:13
How's our role going to evolve in the coming years?
8:16
Yes, I think about this quite a lot.
8:19
The thing that I really want to make sure of is whatever changes
8:23
happen, it's teachers and educators who are at the forefront of it.
8:29
We don't end up having technology or other aspects
8:34
force us into a direction that impacts negatively on learning.
8:37
So one of the things I do think we'll get
8:40
is a lot more information into the big data about the student.
8:45
I think different ways of measuring students, different ways of recording it,
8:50
but I really want to make sure that teachers are then
8:53
in control of how that information gets turned from data
8:57
into intelligence and information. That is called.. We understand..
9:02
I was a teacher myself many decades ago now.
9:07
Teachers understand what
9:09
what it means for a student to learn in a way that maybe
9:13
some of the adtech companies don't always have that insight.
9:18
I do think the role of the teacher
9:20
will become more specialized over years.
9:24
At the moment,
9:26
there's a whole multitude of different tasks.
9:29
You have to stand up and be the entertainer.
9:31
You have to be the mentor.
9:33
You have to be the sort of counselor.
9:34
There's all these different roles you have to jump between multiple times.
9:38
I think perhaps as we make better use of technology,
9:42
some of the things we've learned from Covid about how you can use,
9:47
podcasts to explain.
9:48
So teacher bits, we'll be able to have teachers
9:51
specializing and focusing more on other elements,
9:55
but we certainly need that sort of mentor teacher
9:58
I say with physically with the student to be able to support their learning,
10:02
that's never going to go away.
10:06
All right.
10:07
Let's jump in.
10:08
So I would love for you to talk about your
10:11
and I guess the IBO's philosophy on the assessment of learning.
10:15
Like why are we doing it?
10:17
What are we assessing? And,
10:19
yeah, just kind of starting there broad before we dive in.
10:24
Fundamentally, for me,
10:26
moving to digital assessment is for the student.
10:29
How many of our young people don't engage
10:33
with technology in their everyday lives?
10:36
How many of us as adults don't use computers in their everyday lives?
10:41
I can't remember the last time I sat down for three hours and wrote an essay
10:45
or a report out longhand, so
10:49
we need to move from the old fashioned
10:53
that approaches to examining students with pen and paper, to doing it
10:57
in a digital environment, which is the way they're going to engage with the world.
11:02
Then it
11:03
really allows us to do that more authentic element of things.
11:07
At the moment, I'm stuck with a piece of paper, so that means
11:10
if I want to talk to you about a historical event,
11:15
the best I can give you is photos and cartoons.
11:18
Where's the interactive media?
11:21
Where's the video footage?
11:23
Where's the audio?
11:24
Where's that element of it?
11:26
It's so inauthentic at the moment.
11:29
Thinking more about how we get student responses.
11:31
Again, flat piece of paper doesn't respond at all.
11:35
As teachers we know that the best approach is that you interact with the student
11:40
to get them, not to just give their first answer,
11:42
but to be able to give you multiple iterations of that answer to improve it.
11:46
And again, digital allows us to do that.
11:49
So that's really why the move to digital.
11:53
it's the authentic way of students interacting with the world.
11:56
It's also the opportunities
11:58
to get much more realistic and meaningful ways of questioning and talking.
12:02
Having said all that, at its heart, we still value
12:06
what we've always valued at the IB.
12:08
It's just that I can't always test that in
12:10
the paper setting and I think I can test it better with that interactive
12:14
digital experience.
12:15
I love this, so it sounds like the value of assessment is to really fully
12:20
understand the student, what they know and what they can do,
12:24
and that sometimes there's limitations from a paper that maybe
12:27
a digital assessment might expand that and might allow us to understand
12:30
more about the student. Is that right? Absolutely.
12:33
Yes. That's exactly where I think we're going.
12:35
Okay.
12:35
Now, I do want to say we need to take our schools with us on this journey.
12:41
So there could be some schools who are listening.
12:43
They say, well, that sounds really exciting, but I'm terrified of doing it
12:46
next week.
12:47
So we do need to do this in a gradual process,
12:50
which is why we started off with just offering our existing paper
12:55
assessments in a digital format, sort of paper on screen.
13:00
But over time, as we move to being wholly digital,
13:02
that's where I'm really going to be able to start having that
13:06
more authentic, more meaningful experience for the students.
13:09
Okay.
13:10
Because right now with NYP, you could do either digital or paper.
13:14
No, no.
13:15
With NYP, we went from no exams at all, just a straight digital.
13:21
So we were able to do the jump immediately.
13:23
And that's why our NYP exams, which have been running
13:27
for the last ten years or so, are that interactive.
13:30
It's much more designed around that.
13:34
With DP,
13:35
we realized there's a transition from the paper world to the digital world.
13:39
It's a very similar approach, but it's got to be slightly
13:42
more nuanced to bring all of our schools with us.
13:46
So I want to pick up on that because you mentioned almost like it's
13:49
kind of starting as if the paper assessment just on a screen, basically.
13:53
Yeah.
13:54
But you use hinted at this might create something else.
13:57
There might be something more here.
13:58
I mean, to what extent are we talking about like adaptive testing here?
14:01
To what extent are we talking about,
14:02
you know, different ways that students can demonstrate their learning.
14:05
Wouldn't we get to the early 2030s, which is when we've got our aim to be
14:11
introducing fully, then we're absolutely moving into that space.
14:16
I hope we come back to talking about,
14:19
adaptive, assessment a little bit later on,
14:23
because again, that's a great opportunity that is possible with the digital world.
14:28
That isn't possible in the paper world and will help our students by only
14:31
asking them the questions that are meaningful to them.
14:35
Again, I think once
14:37
we are all digital, we've got that chance to do
14:40
the more interactive tasks the video footage,
14:44
the communication between the student so the student to something,
14:48
the computer responds to it, the student that reacts to that, all of this.
14:53
But in the short term, whilst we running both paper exams and digital exams,
14:58
we are going to keep them as close as we can
15:01
to keep that confidence that it doesn't matter which route you've taken.
15:05
But what I would say even in that paper on screen
15:08
world, the opportunities to student is quite large.
15:11
Suddenly they can control the colors that the screen is, the font is.
15:15
They could control the font size.
15:17
They can start using some of those
15:20
cut and paste tools, which they currently can't in the exam.
15:24
So from an access inclusion point of view,
15:27
it's a real door open there for our young people.
15:30
Yeah, I mean, you could definitely see that,
15:32
especially in like the essay writing kind of components.
15:34
I always think about this when a students writing it.
15:36
I'm like so often when I am typing
15:38
something up, I'm like, oh, that doesn't go there.
15:40
That should go later on here. I need to move this up here.
15:42
And you can't really do that when you're writing.
15:44
You just have to have that so well planned out.
15:46
And you can see where that becomes almost an advantage
15:49
for some of the students who maybe have access to a word processor. Yes.
15:52
And that model effect is going to be one of the things we
15:56
operate and observe very carefully.
15:58
So as you say, whilst it's the same essay question,
16:02
the ability to move stuff around may make a difference.
16:06
So that's why we're going to set grade boundaries
16:09
differently for the digital and paper exams.
16:13
My expectation and based on the research is actually that modal effect
16:17
won't be as big as people might think it is.
16:20
But if it is there, you'll be able to see that small change
16:25
in what our expectations are.
16:26
Because again, great descriptors are based very much on this holistic.
16:31
This is what they students been able to communicate.
16:34
And we don't want that to be impacted by the constructive, relevant element
16:38
of having to cross out and rewrite as opposed to being able to move around.
16:42
I'm really interested, I guess, in what you see happening
16:46
maybe once we've gone fully digital.
16:48
But I think what are some of the parts of the MYP digital assessments
16:51
that you're most excited to bring to the DP?
16:55
Oh, all of it, I guess, is to a certain degree.
17:01
I mean,
17:02
having done the MYP now for ten years,
17:05
the biggest message I would say is
17:08
our NYP schools are no longer worried about the logistic element of it.
17:13
They know what happens if the computer explodes.
17:15
They've gotten past that.
17:18
I know that's going to be one of the fears that many,
17:21
teachers have about this as the introduction of digital assessment.
17:24
So it does work.
17:26
We do get beyond that. It can be done.
17:30
Where the MYP has allowed us to do things is around
17:35
just giving that agency to the students, allowing them to do some of the modeling.
17:39
So again, a classic science experiment in the paper world.
17:43
We give you a table of data and you can see that we've controlled this variable.
17:47
We've varied this and it's all given to you.
17:51
And actually we've taken away one of the key elements of it scientific
17:55
experimentation, which is understanding what is a control variable.
17:59
Not within a digital computer, will be able to say, here it is.
18:03
You vary things and click the button to get a set of readings,
18:06
and you've created the data which you then can analyze.
18:10
It's much more than being controlled, it's much more then being able to
18:14
implement that.
18:16
So that's one example of what we've done in the MYP space
18:19
that we could absolutely bring once we get to that full digital area.
18:24
Sounds like there's
18:24
lots of great potentials here and some really good possibilities for,
18:27
you know, truly understanding what the student might know.
18:30
Yes, exactly.
18:33
What are some of the tensions or some of the challenges that you've
18:37
I guess one from the MYP roll out, but what do you maybe are getting for feedback
18:41
right now from the DP side of things besides exploding computers?
18:44
Well, I say besides that, although in many cases
18:47
that is the biggest fear, which is the how do I get an exam room
18:51
full of 50 students to all have a computer
18:55
that works, and what are all the things that go wrong?
18:57
And I understand that.
19:00
One of the other
19:01
points that is often brought up is that sort of digital divide.
19:05
Are we going to have students being disadvantaged
19:09
if they don't have access to sort of computers as much in their home
19:13
lives, or the resources of the schools isn't as strong?
19:16
And I think that is something that is a real concern to me.
19:21
But I also want to recognize
19:22
for a second that students who don't have a home environment
19:26
where there are a large number of books are currently disadvantaged.
19:30
So I think maybe that digital divide will be a reflection
19:36
of the existing cultural divides and the social economic divide.
19:39
So that's not so big.
19:43
But we are making sure that we set the
19:45
expectations for the tech of the computer very low.
19:49
So we aren't doing as much as we would perhaps love to do in the exciting space,
19:53
because we want the processing power to be within the reach
19:57
and scope of this very basic computers.
20:01
So we've got that element of it.
20:04
Obviously teachers are really keen to know
20:07
what these new assessments are going to look like.
20:09
And that confidence they've got
20:12
a few years old, maybe for some teachers,
20:14
many years of recognizing what a paper exam looks like.
20:17
But digital is something new to them.
20:20
And so we are in the process of soon to be publishing,
20:26
the sort of digital assessment, elements on the,
20:30
PRC, the resource Center, which where we put all that guides
20:33
and everything else so that they'll be able to see
20:36
not only familiarization tools, which are sort of content free
20:41
exams where you can play around with all the functions,
20:43
but also mock exams that can be used so that they can
20:48
have that chance to understand what the student is going to do.
20:52
I think that's a really important part.
20:54
We need to help our teachers prepare our students
20:58
what these new assessments are going to look like.
21:00
So it's not a surprise. We're still testing the same thing.
21:03
It's still early is going to be very much the same questions,
21:07
but we don't want to frighten students or surprise students
21:10
because it's not a familiar environment to them.
21:12
So that's where the familiarization really comes in.
21:15
Important.
21:17
Are you and IB educator wanting
21:18
to improve your practice and needing just a bit more support getting there?
21:22
Ever wanted to learn more about a specific aspect of the diploma program?
21:26
Or just ask a question to an educational theorist about a specific IB aspect?
21:31
IB Unplugged is the place for you.
21:33
Simply visit kognity.com/ibunplugged to submit your question to us,
21:38
and have your question explored in more detail in an upcoming episode.
21:41
And now, back to the show.
21:44
Just as you say, like whether you are a seasoned educator
21:47
or if you're just sort of new to the IB, you've got access to years
21:51
and years of paper assessments and
21:54
it kind of lends itself back to the philosophy of assessment itself.
21:57
And kind of does assessment drive curriculum and, drive what we're doing.
22:01
But I don't necessarily want to dive into that. Yeah.
22:03
Just the idea of like
22:04
when you go in as a teacher, you kind of know, like,
22:05
here's the end goal and this is what I'm working toward.
22:08
And now if you go into a digital space where, as you say, there's
22:11
some good adaptations here, there's some good ways we can really understand
22:14
what a student knows that can be a bit foreign.
22:18
And so I definitely appreciate that you're putting things up on the PRC,
22:21
and people are going to have access
22:22
to that beforehand so they can see what's going on.
22:24
But you also brought up this idea of, you know, the mock exams.
22:27
And that's a huge benefit for many students and for many schools is
22:31
let's run this now.
22:32
I think most schools do this in like a February or March sort of time.
22:36
Let's do some mocks.
22:36
Let's get used to what this is going to look like.
22:38
How do you see that kind of happening in a digital space,
22:41
so that everybody can kind of get a good practice out first?
22:44
It's absolutely our intention before the first wave
22:48
of digital assessments come out, to have digital exams available
22:53
for those subjects, and then we'll continue doing that.
22:56
And then one of the important features we need to understand is
23:00
how do we allow schools to repeat past exams in that mock kind of context?
23:06
Because I know that standard practice
23:09
within current schools, we need to be able to continue that.
23:15
Usually it's a really good way of preparing the students
23:18
for the exams.
23:20
And so we need to recreate that in the way in which we do
23:24
the digital assessments.
23:26
Yeah, it kind of lends itself to that assessment for learning
23:28
before they get into that assessment of their learning.
23:31
Right? Exactly.
23:32
That's the goal. Yes.
23:34
Although if I can briefly detour,
23:38
I am really excited about the opportunities that digital,
23:42
assessments and digital classroom practices
23:44
can get to improving the quality of that assessment for learning.
23:48
So maybe not quite in the way we do things summatively, but it gives us
23:52
more opportunity to spot how the student has approached a question
23:55
in that digital space,
23:57
rather than just having to sit there and watch them answer the maths question.
24:00
It can record and spot the key moments.
24:03
Give us that information.
24:05
So that's part of the big data opportunity
24:08
I said earlier on. Let's follow that tangent for a little bit.
24:11
Tell us a little bit more.
24:12
What else is kind of exciting in that space for you
24:14
that you see the digital space really enhancing the assessment for learning.
24:18
So one of the really important things is this isn't just digital assessment.
24:24
We need to use this as a opportunity to bring more digital thinking
24:30
and more digital opportunities into all of our classrooms.
24:33
Not only will that mean that we have students familiar
24:38
with the way of approaching things before the exam, it'll also give teachers
24:42
that opportunity to collect more data and understand more data about things.
24:47
We've all had really great lessons
24:50
where we've gathered them in the little group and we've done this
24:53
and done that, and you can't do that every single lesson that often.
24:57
You need to group a little group together, then look after the rest of the class.
25:01
With that digital opportunity, it's much easier to be able to create
25:05
those little sort of communities of work where they can talk to each other.
25:09
It's much easier to capture that.
25:11
It's much easier to have that interactive experience.
25:14
This sort of good quality gaming ification of learning, where the students learn
25:20
how to do it in a more playful environment than the sort of very traditional
25:26
“Here's a question. Answer it.”
25:27
I think there are opportunities for how we could improve education going forward.
25:32
And then, of course, also with the introduction of artificial intelligence,
25:36
we can spend more time,
25:40
where we need to.
25:41
So, my common example: class of 30,
25:45
as soon as I look out for one student, there were five kids there,
25:48
and I know three of them just need a pat on the head.
25:50
Say, yes, you got it right. Get on with it.
25:53
I've been waiting five minutes.
25:54
I can do that.
25:56
And let me spend my time
25:57
with the other two students who really need that human intervention.
26:01
So these are the kinds of opportunities I can see by getting
26:05
more good use of digital in the classroom
26:08
as opposed to bad use, which goes back to let's make sure
26:12
teachers are at the forefront of what is good digital assessment.
26:16
What is good digital practice.
26:18
No, I love that.
26:19
And I think it's something that at Kognity,
26:20
we obviously think so much about.
26:22
And it's been in the past with exams and it's we're really excited
26:26
about the digital assessments also because for us it's there's so much
26:30
with formative assessment and building up that data.
26:33
And it's all of this great stuff and then all of the sudden you hit
26:35
like the exam style questions and it's like, and now
26:38
everyone take out their pen and papers and we're going to write these down again.
26:42
And it's like a, it's a block as opposed to it being part of this
26:45
collective, intelligence, as you said, it's not just data.
26:49
And then it becomes something where you're not doing learning
26:53
over here and assessment over there and practice over there.
26:58
It's actually all being able
27:00
to come together where suddenly then the formative assessment pieces
27:04
and the data that you're collecting over time is actually also exam prep.
27:10
And it's not this separate thing, this alternate universe where suddenly
27:14
we all turn off our computers and we write our essays.
27:18
So, no, we're super excited about that.
27:21
Just to that point,
27:21
it does seem a little bit foreign, where I think most classrooms now
27:25
have a very tight integration between a digital space and
27:29
an authentic space.
27:31
I guess I could say that, and then suddenly when we go to assessments, it's,
27:34
yeah, shift it and go back to paper and pen kind of thing.
27:38
And, you know, I was tell my students, as they prepare for exams, I'm like,
27:41
make sure that most of your preparation is done writing.
27:43
You need to train your hand how to write again,
27:45
how to write for that long, you know, just get used to that.
27:48
It can, for a lot of students, be a real big change
27:52
from having done a lot of word processing and to
27:55
to suddenly having to deal with that hand cramp. Yes.
27:58
Yeah. That is certainly something else
28:00
I've heard.
28:01
Again, the challenging bit I'm going to raise, of course, is also
28:04
we just need to be cognizant that there is a still
28:07
a space for handwriting around the world as well.
28:11
It's not just that we're trying to get rid of
28:13
that as a concept, but it needs appropriate place
28:17
not being a barrier to students showing what they can do in an exam.
28:20
Okay. That's good.
28:21
So you've landed self quite a bit here to the use of AI
28:25
and using AI to kind of assess how students are doing as well.
28:28
I wonder if you could talk a little bit about on the back end of assessment
28:31
now with regard to getting feedback from the students
28:34
and that sort of possibility of adaptive testing.
28:37
I'm also an examiner, so part of my head is in that examiner space of like,
28:41
what does this look like there?
28:43
I wonder if you could kind of talk about the
28:44
the opposite side of this. So once the students
28:46
have done the assessment, what kind of is going on there? Okay.
28:49
I mean obviously adaptive assessments is front end, not a back end, but
28:53
if we think about how we use artificial intelligence
28:57
and sort of assessment, in this space, the first thing to say is that
29:02
digital assessment is still being marked by human beings.
29:07
We're not changing the approach where the IB currently has
29:11
MCQ’s. We’ll use MCQ engineers to mark that, but the essays are still coming
29:17
to human examiners through the internet so that they can mark it correctly.
29:22
So now to be a lot easier to read the responses is what you're telling us.
29:25
Certainly it will remove that handwriting aspect of things.
29:30
Absolutely.
29:31
It might even allow us to be able to remove some of those blank pages
29:36
that as an examiner you get and either is absolute
29:40
joy or absolute frustration, depending on how you feel about stamping
29:45
“seen” on a piece of paper that you've marked it.
29:51
I don't think that,
29:54
the AI tools we've got at the moment is good enough
29:57
to do high stakes marking.
30:00
We're doing a number of pieces of work with various partners
30:04
to understand how good it is, but I don't think it's there yet.
30:09
I can imagine in the next 5
30:11
to 10 years we can use it as a quality check.
30:15
So, we get a human marker to mark the piece of work.
30:20
We then see what's the AI makes of it.
30:23
And if there is a big discrepancy between the human marker and the AI marker
30:27
that rings an alarm bells, and we give it to another
30:30
senior human marker to market, and that to me seems to be a better
30:35
sort of at risk mechanism than the at risk mechanisms we do at the moment,
30:40
which is about sort of..
30:44
Well, I think the seeding, the checking that the examiner is marking to write
30:49
standard will still need to remain because I need to be confident
30:52
that you are marking correctly before I let you loose on live marks.
30:56
a live student outcomes.
30:58
But again, it's just that extra check to pick
31:02
the one where maybe was the last one that somebody marked in the evening.
31:07
Or maybe it was that really weird one
31:11
where it was somebody who's from such a different cultural background
31:14
that you didn't really get the points they were trying to make.
31:18
Or yeah, maybe just had a bit of a headache that day.
31:21
Yeah.
31:23
So okay.
31:24
We're certainly using that.
31:26
Also, we could then use the of digital technology
31:33
to get started on the marking process
31:36
much easier, much quicker as well.
31:38
At the moment we send papers to schools.
31:43
Carbon cost.
31:44
Right. They then send it back to us.
31:47
Carbon cost, but also delay of up to a week or so.
31:52
And so we can't start marking the exams until after they've arrived
31:58
at the scanning center being scanned with the digital world, not only do
32:02
we take out that sort of carbon cost of sending it around,
32:06
but we could start sort of 20 minutes after the exam finishes,
32:10
and that's a real opportunity. Also,
32:14
of course, it helps us with that exam security element as well, because,
32:18
we don't have to ask schools to have exams in a locked room for
32:25
a week or the half before the exams are sat.
32:27
We know that they're secure with us until such time as the exam
32:31
gets started, at which point it's then available to the students.
32:34
So there's some of the other elements of the digital assessment,
32:39
which are really helpful, a positive impact on the
32:44
both us and then teachers and examiners not having those delays as well.
32:48
Yeah, I definitely see these as real positives.
32:50
I mean, there's the DP coordinator and sort of exams administrator at my school.
32:53
It's like every morning you go to the safe, open the safe,
32:57
bring them out, cut open the papers in front of the students,
33:00
collect them back, put them in a DHL, send them over
33:03
and even just that, the financial cost of like shipping stuff back.
33:07
Yeah.
33:07
You know, you spoke a little bit before about like accessibility
33:10
and having access to word processors and that sort of thing.
33:12
I think if schools kind of take a note
33:15
to think about how much money they're spending on shipping papers,
33:18
that amount of money could be used in investing computers.
33:21
And now, you know, you get digital assessments a little bit easier.
33:24
Definitely
33:24
from the whole, you know, coordinator role of like,
33:27
making sure their security of the paper that's actually a big weight off
33:31
my shoulder is to recognize that
33:32
yeah, you're taking care of that security aspect.
33:35
Because it's a pretty high stress sort of thing.
33:37
Yeah, I could appreciate it is. Yes.
33:39
Yeah, that's a really good point, John,
33:40
because also, like, I would imagine that the entire exam process
33:44
could actually be a barrier to entry for a school becoming an IB school.
33:49
Right.
33:50
Like buying the it all of the purchasing and dealing like logistics of paper
33:55
exams could actually be something that makes it less accessible globally.
34:01
I think, in fairness, the IB
34:03
approach is quite similar to a lot of other national systems
34:07
because it's the again, we need to get you to exams well in advance.
34:11
You need to store them securely.
34:13
So we're not a complete outlier compared to some of the others.
34:18
But for all,
34:20
all of these systems, it does reduce that.
34:23
That's challenge.
34:25
I guess the other element of that is things like,
34:29
if we allow certain materials
34:32
in the exam, so a formula booklet
34:36
that's there as part of the digital assessment,
34:38
you don't need to print all those formula booklets out and then distribute them.
34:43
You don't need to then either ditch those or check them to see if anyone
34:47
scribbles anything inappropriate on them.
34:50
All of this it gives the students a much better experience
34:53
and then all those access elements are there ready for the student.
34:58
You don't need to have a separate paper
35:00
for the student who wants a slightly bigger or slightly different font type,
35:05
and they can control that during the exam as well.
35:08
I do think that we already I do think
35:11
that the digital gives much better accessibility options.
35:15
We already offer a laptop for quite a lot of students
35:19
who have these issues and things like,
35:23
screen readers and the like.
35:25
Suddenly it becomes normal not asking them to be different to anybody else,
35:29
that it's just there and it can be accessed as needed.
35:34
So sounds like a lot of positives here.
35:36
And to kind of calm any of those fears we're not immediately going to
35:40
AI marking the work.
35:42
It's still a human examiner. I'm afraid
35:44
you’ll still have to mark for us.
35:49
All right.
35:50
So it sounds like, quite a few positives here from your view,
35:53
what do you see as like
35:54
the professional development needs that are going to be there for teachers
35:57
to make sure that they can transition this quite well?
35:58
I mean, we've touched on it a bit with regard to having access to,
36:01
you know, past papers and mock exams, but is there more to it, from your side?
36:06
I think really the first bit of that professional development is just
36:11
letting the teacher see what the digital assessments look like
36:16
and that sort of reassurance that this isn't a complete change
36:20
to what we've had in the past.
36:23
It's a the evolution, of it.
36:26
Now, once we move to that more purely digital element,
36:31
I think that's when we're going to be able to share existing good practice.
36:37
Because what's really important to me is I don't see,
36:41
digital assessment as changing teacher practice.
36:44
I see being able to look to see what those really good teacher practices look like,
36:48
and then being able to now replicate that
36:52
in my high stakes summative assessments.
36:55
But I do appreciate we'll need to do a lot more to sort of share
36:58
those kind of opportunities, with teachers.
37:02
For most of our IB teachers, it'll be very familiar,
37:04
we're not changing the principles of what we value with just being able to do it,
37:09
but it will mean exam practice
37:12
is much, much closer.
37:15
One of our learnings from the MYP
37:18
is that there is less benefits to sort of really hard,
37:23
exam practice leading up to the exams
37:26
because they are unfamiliar in a good way,
37:30
and they require the students to understand what's going on,
37:34
not remember what the answers to the questions are.
37:38
And again, we tried to do that in the DP space,
37:41
but with that interactivity, we'll be able to do that even more.
37:46
So there's another opportunity.
37:48
So yeah,
37:50
we have a clear plan for how we support
37:54
teachers, understand what the assessments look like, mentioned
37:58
again, the familiarization tools and the mock exams.
38:02
And then we've also, as we go forward, get to be sharing those good practices.
38:07
And some of the things we are thinking of going forward
38:11
for that post 2030 exciting world.
38:24
Laura said try assigning tasks as formative assessments
38:27
or homework to extend learning beyond the classroom.
38:30
Independent practice helps reinforce understanding and encourages
38:34
students to research and explore concepts on their own,
38:38
strengthening both knowledge retention and confidence.
38:41
Thanks, Laura.
38:44
I'm thinking push back because I think about with Kognity,
38:47
for example, the teachers that I think there's the biggest maybe barrier
38:52
to entry with digital assessment would be usually we find with maths subjects
38:57
and you're shaking your head for those who can't see.
39:01
The look you just gave Matt was hilarious.
39:03
You are absolutely right.
39:07
The biggest challenge we have in that digital space is the
39:11
is the writing of extended mathematical proofs.
39:15
Yeah.
39:17
My educational background was as a maths teacher.
39:20
So this is a matter of very close to my heart.
39:24
We are still working to understand how we address that.
39:29
Now in some areas
39:32
formula engines are very good.
39:36
And in the MYP space where we just ask them to write one equation
39:39
down, it's perfect.
39:42
That's not going to work for DP maths.
39:46
So we are still examining what's the best way to do it.
39:49
Is it that we use
39:52
some kind of keyboard shortcuts
39:56
against for the mathematicians out there,
39:59
LaTeX is quite a good language to be able to write and produce
40:03
your equations rapidly, but that would require us to include that
40:08
explicitly as part of the teaching and learning process.
40:12
Is it to that,
40:14
that we find some different approach.
40:16
We've experimented with some of the,
40:19
touch pads to see whether we can get
40:22
handwriting to be reflected straight on the screen.
40:26
That offers us a sort of technical barrier, maybe for some schools.
40:32
So, you've hit on the one
40:34
that I don't know how we're going to fix yet,
40:38
which is why the maths and the physics
40:41
of the other sciences are the absolute
40:44
talent of what we're introducing, we're introducing it for the,
40:48
languages and sort of the essay
40:50
subjects first, because that's the most familiar.
40:53
I have ideas.
40:54
We don't have a good solution yet.
40:56
Fair enough.
40:57
We, it's something
40:58
yeah, we always are iterating on and figuring out the best ways to get
41:02
yeah, maths, especially online.
41:05
But then just logistically for those, listeners
41:09
who aren't exactly sure what is the brief
41:12
overview of the kind of rollout of this?
41:16
Excellent.
41:16
So in May 26th, we've got a small group of,
41:22
I think it is 38 schools spread around the world
41:25
and selected to be both ready and also reflective
41:30
of our kind of communities, who will be starting off doing digital
41:34
assessments in maths, language and, sorry, English language and literature.
41:40
Spanish language and literature and English B.
41:44
So that'll be the first three subjects.
41:46
And obviously they are some of the ones which are easiest to adapt through.
41:52
Then we're going to increase year on year.
41:55
Additional subjects,
41:57
more languages
41:58
followed by some of the group three subjects which are getting out low
42:02
and then towards the end of the process, the science and the maths.
42:07
Throughout that period, we're looking at encouraging schools
42:11
to transition from paper to digital, but we are running books in parallel
42:16
for, let's say, the next 5 to 7 years
42:19
so that schools can move across at their own rate.
42:23
Although I'm hopeful they will be enthusiastic
42:25
about moving across because that then opens up the door
42:28
to all the exciting opportunities we've spoken about.
42:31
But I'm no longer restricted
42:32
to being able to sort of reproduce the assessments on paper and on screen.
42:36
Yeah, let's talk about that.
42:38
So let's talk sort of future after the rollout, the sort of exciting
42:42
opportunities to AI and the possibility, I guess, of adaptive testing.
42:45
You mentioned you wanted to talk a little bit more about that.
42:47
Where do you see that kind of having a place?
42:50
So at the moment, every exam paper I produce has to account
42:56
for the whole range of students who are going to take it.
43:01
So I have to have questions in there for the students who are going
43:03
to be hoping to get that grade three, and that's their target.
43:07
And the students that are hoping to get a grade seven,
43:10
and it's far too easy for the student
43:14
who's looking at the top end to make a few silly, trivial mistakes and lose marks.
43:18
And for the weaker
43:19
student to be just depressed by all these questions, they can't answer.
43:23
Right.
43:23
So the idea behind adaptive assessment is that as
43:27
the student goes along, the answers they give allow us to direct
43:33
the sort of level of question to being appropriate for that student.
43:37
So the majority of questions they answer are at their ability level.
43:42
Now the current I mean adaptive assessment has existed.
43:46
But the moment you've got to be able to mark it as you go along
43:51
and that effectively means multiple choice or very, very short answer,
43:56
which is not the right tool for for asking these things.
44:00
So here's some other area where we might use
44:04
AI to do a first run of
44:06
given that answer, where do we think the student should go,
44:10
and then we would get a human marker to look across the whole
44:14
spectrum of what they'd written to give them their actual mark.
44:18
So that's how adaptive assessment could really make
44:22
the educational experience so much better for the student.
44:26
Because the test is their test.
44:29
It's not the test that's designed to fit everybody.
44:33
I won't go into then all of
44:34
my psychometric challenges to how I make sure it all works.
44:38
But there is a science behind that which gives me confidence
44:42
that we'll be able to explain that when it comes.
44:45
Another opportunity,
44:48
might be, for the computer
44:51
to ask follow on questions to what they've put.
44:56
So if a student gives a very short, narrow answer to a six mark question,
45:01
maybe the computer could then prompt and prompt again
45:04
to try and get them to show everything they know.
45:08
And removing that constructive relevance of
45:11
being good at answering exam questions
45:13
as opposed to showing what you actually know.
45:17
I can see,
45:19
opportunities for artificial intelligence to be able
45:24
to be an interactive conversation with somebody.
45:28
So the question could be explain to this person, this concept,
45:32
or how would you explain this to somebody else?
45:37
And we can do that at the moment.
45:39
It's one of the really great tasks you get in that classroom,
45:43
isn't it, where you've got that as a 1 to 1 relationship,
45:46
but you can't do that with a class of 30 students.
45:50
in the exam setting.
45:52
Maybe with AI we can.
45:53
Maybe we can start getting those different types
45:56
of skills, different types of opportunities being done.
46:02
And then I get finally the bit that comes to
46:05
my mind is we know the world is full of complex problems.
46:09
When you talk on this bit, it has this consequence over here.
46:13
You took on this bit, has there. Now,
46:16
if we have digital assessment,
46:19
we can allow students to try and make a change
46:22
and then react to the consequences of those changes.
46:27
That I think getting to the heart of what
46:29
complex systems and complex problem solving really needs to look like.
46:33
I can't do that on a paper based exam, because all they need to do
46:37
is turn back and change their decision to the previous thing.
46:41
Yeah.
46:42
I'm allowed to respond to what the student has given me.
46:47
Again, we can do that in a
46:48
sort of storyboard fashion now with digital, tools,
46:53
but maybe with AI in the future,
46:55
it can be much less storyboarded, much less choose your own adventure.
46:59
It can really be a sort of reacting
47:02
in the same way as a human being, which a teacher would react to.
47:05
Okay, you did that. I didn't expect that.
47:07
But I understand what the consequences will be, now deal with that.
47:11
So these are the chances, opportunities.
47:14
I was thinking of that quite a bit when you gave that example
47:16
of kind of the experiment, almost simulation of like,
47:19
okay, pick your variables and this sort of thing.
47:21
I was like, oh, that lends itself
47:22
to so many opportunities of like, what else would you control?
47:25
Like little prompts here
47:27
or what else could be, you know, manipulated in this particular experiment.
47:30
And that seems like such a better way to really understand what the student knows.
47:35
And, you know, just to your point earlier about trying to make a test
47:38
that's accessible for everybody, the challenge that must be to
47:42
have questions that are accessible for that student,
47:44
who's you know, going to hit the twos or the threes or something
47:46
like this, and the students who's going to hit the sixes and sevens.
47:49
You can see where this sort of not fully adaptive
47:52
but initially kind of adaptive testing gives that better chance to know
47:57
is this student really a kind of level five, level six, something like that.
48:00
Because we know
48:01
that sometimes that difference between the two can be really challenging.
48:04
And especially
48:04
when maybe there's only a couple questions in the paper that really tease that out.
48:08
Right?
48:09
Indeed.
48:09
And remove those constructive, relevant mistakes
48:13
early on where you see your astute to clearly understand it.
48:17
But just, you know, added three numbers up incorrectly.
48:21
It suddenly is just lost that 1 or 2 marks that'll impact them later on.
48:26
Right. Yeah.
48:28
So you know, in some of our other episodes we've spoken
48:31
quite a bit about like the importance of content knowledge. Yes.
48:34
But also really a move towards skill development.
48:37
Competency based learning, the ATL’s, these sort of things
48:40
that we have within the IBDP.
48:42
To what extent do you see this
48:43
move to digital assessments, allowing more of an assessment of those skills,
48:47
of those competencies, as well as the actual content knowledge?
48:51
So I think you used exactly the right word.
48:54
They're allowing.
48:58
Well, there is a
48:58
whole separate movement to let's focus on these
49:03
all those competency elements and how can we get that.
49:06
But digital assessment in part
49:09
through that interaction, in part through that being able
49:13
to set up different scenarios makes a big difference to what we can.
49:18
Collaboration is the one that I so often hear.
49:21
We want to assess, but I know when I was teaching,
49:25
in a school, often we would lose students towards the end of their courses
49:30
because they'd go off and do something else.
49:32
And the issue was, if I gave you a group, your collaborative skill
49:36
was being able to hold the group together as it fell apart.
49:40
That would be very different to a school where perhaps everybody was trying
49:45
to show their leadership skills with the group. And collaboration
49:49
it's always at, if I've had a different group, could I perform differently?
49:53
Well, yeah, we need that variability
49:56
but I don't need it when some of these diversity place depends on it.
50:00
Sure.
50:00
AI could potentially give you all the same group to work with.
50:05
So at least everybody's being compared with the same experience.
50:09
And so we're recognizing that.
50:12
Also that sort of
50:13
interactive nature helps us address some of the sort of communication skills.
50:17
It's not just can you use a good poster, but can you explain, can you communicate
50:22
the concepts to somebody else
50:25
and respond and vary to the way which they think?
50:28
I mean, basically, can you be a good teacher?
50:31
Yeah.
50:32
That is where I think the digital opportunities can help us
50:37
address skills that are much, much harder either
50:42
to address accurately in a paper world or to address
50:48
fairly, given the different locations, different experiences, different context
50:53
with all the students. So yeah, it's possibilities.
50:58
I mean, it's definitely exciting, you know, because as I think now kind of
51:01
with my university careers guidance counselor hat on, it's like
51:04
you want the assessment,
51:06
you want the specific grades and how students doing,
51:08
but then you also need other sort of ways that you can comment on the skills
51:11
and things that students can do.
51:13
It sounds like
51:13
this is kind of creating possibilities where the two can kind of overlap
51:17
and you can show more of their skills, or you can show more their competencies
51:19
as well as their content knowledge through a very good assessment.
51:23
That is where I think we need to be able to change the,
51:28
approach to education and assessment going forward.
51:32
It's so that we can capture more information
51:35
about the whole of the student to really recognize
51:40
all of their strengths, not just their narrow academic strengths.
51:45
But on the flip side also, why are we capturing that?
51:49
For me, it's very much about helping the student
51:53
understand themselves so they can make decisions based on that.
51:57
My nightmare scenario is that we go from having to have a grade
52:02
seven in biology and chemistry to become a doctor,
52:05
to having to have a grade seven biology, chemistry and caring.
52:09
Yeah, right.
52:09
Because I know that that caring just it's not something we should be grading.
52:15
We can assess it
52:17
to gain insight, to help the students.
52:20
But sticking a number on it is going to destroy the very,
52:24
the very nature of what that important competency is.
52:27
I really appreciate that answer that helps to clarify, I guess,
52:31
a little bit more of like even just the philosophy of assessment in general.
52:35
What are we actually assessing? Why are we assessing it?
52:37
Where are the gaps and those gaps,
52:39
some of those gaps should stay in place?
52:41
Yes. We are talking about an assessment that allows us to get more competency
52:45
and more skills.
52:46
Yet there's still a line
52:48
where there's other elements that shouldn't be assessed and shouldn't
52:51
come down to a number or a grade and be that humanistic sort of element.
52:56
Assessment for insight, not assessments for measurement.
53:00
I might just try to summarize then, if that's okay, I want to just sort of
53:02
see if I got most of this.
53:04
It sounds like really there's a lot of excitement
53:06
and a lot of cool opportunities with digital assessment.
53:09
It sounds like the transition from paper to digital is being calculated.
53:14
It's measured out.
53:15
We know how it's going to go.
53:16
We don't need to be too worried about it.
53:18
It sounds like we've already done some of this with MYP.
53:21
And really, there's more to be excited about than there is to be worried about.
53:25
From what we've discussed so far, it sounds like that's
53:28
kind of the key thing that I've taken away from this conversation.
53:31
Thank you very much.
53:31
That's exactly, I think you summed it up really well.
53:34
There is opportunities.
53:36
It's going to be done at a reasonable pace to bring our schools with us, and
53:41
we can see some of the challenges will work
53:43
with our schools to make sure we help bring them online with this.
53:47
Perfect. I'm really excited about this.
53:50
Definitely from the Kognity side, the potential that it can really help
53:54
with learning and really bridge that gap between assessment
53:57
and the learning that's happening in the classrooms.
54:00
So Matt, thank you so much.
54:02
This was really fun.
54:03
I'm glad that we got to have this conversation.
54:06
It was great.
54:07
Thank you very much. Yeah. Thanks a bunch, Matt.
54:09
Always fun.
54:10
IB Unplugged as a Kognity podcast hosted by John Patton
54:14
and Courtney Marline and produced by Katilina Gaffney.
54:17
Be sure to subscribe or follow and share with your fellow IB educators.
54:21
If you'd like to submit a topic, request or a question, please do
54:23
so on kognity.com/ibunplugged.
54:27
Let's keep the conversation going
54:28
and support each other in teaching with confidence, creativity and joy.
54:32
See you next time.