Latest / Integrate This! Ed Tech Podcast / Leon Furze on Assessing AI in Education | A Framework for Teachers

Transcript

Leon Furze on Assessing AI in Education | A Framework for Teachers · Integrate This! Ed Tech Podcast

0:00
Welcome to Integrate This, your go -to EdTech
0:03
podcast where curiosity meets classroom innovation.
0:07
Join your hosts, David Berner and Jeremy Mikla,
0:09
as we explore the rapidly evolving world of educational
0:12
technology. From game -changing tools to insightful
0:15
conversations with today's top EdTech voices,
0:18
we're here to help educators stay informed, inspired,
0:21
and ready to innovate. So tune in, level up,
0:25
and let's integrate this. Welcome to the integrate
1:11
this ed tech podcast today. We have a very special
1:15
guest with us out of Australia It's actually
1:17
a evening for us here in the US and Leon is joining
1:22
us from Australia and Leon Actually, how do you
1:29
pronounce your last name again? Sorry? However,
1:32
you like really take take a punt see if you get
1:35
it right. I I say Leon furs Got it got it in
1:39
one. All right. Wow, that's pretty impressive.
1:42
Yeah Okay, so Leon is like I said joining us
1:46
from Australia and is a consultant best -selling
1:49
author and PhD candidate with over 15 years experience
1:53
in secondary and tertiary education. His PhD
1:56
is focused on the implications of generative
1:59
artificial intelligence for teachers of writing.
2:02
And I wanted to have Leon on the podcast and
2:05
talk to Jeremy about this as well, just because
2:08
there's been a lot of buzz in circles that we're
2:12
in with specifically the assessment model, the
2:16
AIAS model that Leon was part of developing.
2:21
And so we're very excited to have Leon with us
2:24
this evening slash morning. And so welcome to
2:28
the podcast. And if you don't mind, maybe tell
2:30
us a little bit about your experience, background,
2:33
life in Australia and kind of kick things off.
2:37
Yeah, thank you. And thanks for the invite. Yeah,
2:40
so I I've been a secondary teacher for most of
2:43
my career. I'm from the UK originally and moved
2:47
out here in 2009. So I did my teacher training
2:51
in the UK, English, drama, media, and taught
2:55
over there for a little while before moving out
2:57
here, which is where my wife's family is from.
3:01
So 2009, we moved out here and I've worked in
3:04
a regional Catholic school for most of that time.
3:07
So a small -ish regional co -educational Catholic
3:11
school. And I taught mostly senior secondary
3:14
English and a handful of other subjects. And
3:18
then, yeah, I bounced through a few different
3:22
leadership roles, Head of English, Director of
3:24
Teaching and Learning, and eventually decided
3:26
that I would jump out of those roles and into
3:29
a PhD. I chose a PhD in Generative Artificial
3:34
Intelligence, which at the time was just called
3:36
Automated Writing Technologies, because I started
3:40
my PhD just before CHAP GPT was released. And
3:44
then CHAP GPT has just kind of blown everything
3:46
up, as we all know. And so that whole idea of.
3:51
you know, when chat GPT comes onto the scene
3:53
and stuff and the you know, really what we think
3:56
of is then being generative artificial intelligence,
3:59
you know, but it's it's not the first time that
4:02
a new technology has burst onto the sheet the
4:05
scene that is really kind of shaken education.
4:08
You know, if you want to think about things like
4:10
in my time, I remember the internet coming into
4:14
our high school. And I remember teachers kind
4:17
of, you know, like, keep that god -forsaken thing
4:21
out of my classroom or you know um graphing calculators
4:25
or even the calculator you know these these sort
4:28
of technologies that come and really disrupt
4:31
education um you know but it has some kind of
4:35
skeptics but yet you know it it can lead us to
4:38
some other greater things you know like how do
4:40
you see gen ai kind of fitting into some of those
4:43
other disruptive technologies Yeah, it's a super
4:49
interesting conversation, I think, at the moment,
4:51
because like a lot of technologies, there's a
4:55
big commercial push to get this stuff across
4:57
the line and to get it into schools. And we've
5:00
seen that with other things, particularly during
5:03
the COVID years and remote learning, when we
5:05
saw schools en masse picking up platforms like
5:09
Microsoft Teams, 365, Google Classroom, Workspace,
5:13
Zoom, all of those kinds of platforms, which
5:16
I guess some schools were using pre -COVID, but
5:19
then post -COVID were really kind of exploding.
5:23
And I think we're seeing a similar kind of ramp
5:26
up at the moment from the technology companies
5:30
that own these technologies. But I think finding
5:33
that middle ground is really important. Don't
5:35
be too keen to rush in and adopt every single
5:40
thing, but also be open to some of the possibilities.
5:46
But it's a bit rocky sometimes being in the middle
5:49
ground. People want to push you in one way or
5:50
the other. I think from my perspective, I tend
5:54
to be the one that wants to jump in and not even
5:56
consider some of the ramifications of what I'm
5:59
doing. So I always need to take a deep breath
6:01
and kind of remind myself that, hey, just be
6:05
a little bit patient here and try to figure some
6:08
of it out before I kill myself in diving head
6:12
first into the shallow end of the pool. Yeah,
6:17
we usually I know in the past like for us we've
6:19
used the cliff analogy like usually there's a
6:22
person in the middle and then there's one person
6:23
running off the side of the cliff or One trying
6:26
to stay really far away from the edge of the
6:27
cliff, you know, it's a good depiction of like
6:30
where people are at you know with their with
6:33
their AI user how they feel about it, but is
6:36
there a so with the the AI gen AI it's kind of
6:41
a unique situation because You know even in exchanging
6:47
messages with you prior, everything's changing
6:51
so quickly. Can you take us back a little bit
6:55
with your role and given the AIAS, the first
7:00
iteration of that, can you explain for the audience,
7:04
what was the process of coming to terms with,
7:09
we need to develop something here. What was the
7:12
rationale behind the first AI assessment scale?
7:15
and the purpose for it, or what were the conversations
7:18
like when that was going on? I think back in
7:25
February, March, 2023, after that first initial
7:28
rush. And over here in Australia, we have our
7:31
big summer break in December, January. So the
7:35
teachers went away and came back to schools.
7:38
And all of a sudden, ChatGPT was everywhere.
7:40
And I know that your semesters in the northern
7:42
hemisphere are a little different, but you experience
7:44
a similar thing where the chat GPT landed in
7:47
November 2022, and then there were a few months
7:49
of absolute chaos. And in the media, we saw a
7:53
lot of, this is going to be the end of education,
7:56
the end of the essay, the end of high school
7:57
English, my subject, all of these things. And
8:01
there was a reaction from our K to 12 sector
8:05
in the government sector to try to ban the technology.
8:09
And there were similar conversations happening
8:11
in the universities. So our group of eight universities,
8:14
Melbourne, Sydney, the older universities, they
8:19
said, we're just going to temporarily, we're
8:21
going to go back to pen and paper exams, and
8:22
we're going to see what happens. And I was chatting
8:26
to a university up in Perth, Edith Cohen University,
8:30
their teaching and learning team, and we said,
8:31
we've got to have something more than just ban
8:34
it or let them use whatever they want. So I just
8:37
kind of, you know, throughout this scale idea,
8:40
you know, use it a little here, a little here,
8:42
a little bit more here. And then Mike Perkins
8:45
and Jasper Rowe and Jason McVaugh, the other
8:48
authors on the academic publications, they emailed
8:51
me and said, hey, we like this scale idea. Can
8:54
we adapt it for higher education? And we started
8:57
working on it from about mid -2023 with that
9:01
original version. Yeah, and that's kind of I
9:05
think you know as far as this it's been a brief
9:08
timeline But there's still definitely a timeline
9:10
here of the evolution of of generative AI From
9:13
like you said where your teachers all go away
9:16
and they come back and it's just chaos because
9:18
chat GPT for us here I remember it coming out
9:21
like in the in the U .S. like at Thanksgiving
9:24
time and then people float through Christmas
9:27
and stuff like that and then that similar thing
9:30
where you've got that break and then people come
9:32
back and it is just like changed everything.
9:36
But you know I also don't think that you typically
9:39
see Education being on the forefront of some
9:43
of those things all the time when the new technology
9:46
emerges. But I think from its very beginnings
9:49
here, we could definitely see that education
9:52
has definitely played a role. And that's where
9:54
I think this idea of the AI assessment scale
9:58
being so important at that even beginning times
10:03
just as a way to start generating conversations,
10:06
right? Between the teachers and students, between
10:10
admin and faculty, all of those different groups
10:14
kind of coming together. So as part of that,
10:19
when you release this scale, because the technology
10:21
changes so fast, did you find out that you had
10:24
to change what you saw as the assessment scale
10:27
to match some of those changes that were taking
10:29
place in the different groups and partnerships
10:32
involved in that? Oh, yeah, absolutely. So we've
10:36
been really keen all the way through the development
10:40
to get as much opinion as possible from teachers
10:43
and from students and from K -12, higher education,
10:47
adult education, vocational, all the different
10:49
sectors where we've seen it used. And we did
10:53
our own pilot study which Mike conducted at British
10:55
University Vietnam with the students and the
10:58
staff there. So we published that earlier this
11:02
end of last year now in the Australasian Journal
11:06
of Education Technologies. And in that pilot,
11:10
we found that one of the biggest advantages was
11:12
that it started those conversations, which otherwise
11:15
weren't happening. And we have said all along
11:20
that the AI assessment scale isn't going to stop
11:25
anyone from cheating or misusing artificial intelligence.
11:29
This is the big rationale for why we ditched
11:31
the sort of the traffic light colors of the first
11:34
version, because we found that You know, we had
11:39
written about this in the original paper, but,
11:41
you know, people don't necessarily have time
11:42
to read all of the academic literature around
11:45
these things. We just kind of grabbed the stuff
11:47
off the shelf and run with it. But we had said,
11:50
if you are in any situation other than, you know,
11:55
a no AI supervised probably in -person task.
11:59
then students have access to the technology and
12:01
they'll probably use it, and they'll use it in
12:03
many ways, some of which might be considered
12:06
academic misconduct, some of which will be fine,
12:09
some of which will be kind of weird and wonderful,
12:11
and some of which will be totally inappropriate.
12:14
But we can't necessarily control that, but we
12:17
can have conversations about it, you know, and
12:19
we can help each other to understand what that
12:22
looks like. And you bring up the stoplight here,
12:25
so just to... Clarify that we're on the same
12:28
page here, right? In the stoplight format, it
12:31
would be red means no, right? And sometimes I
12:36
think red might carry a negative connotation
12:38
with it. And then you've got the green means
12:42
full go, use AI for everything, which might be
12:46
good, but in some cases could be bad. And then
12:48
yellow is kind of that area in between where
12:51
it maybe is a little. used in a more limited
12:53
scale? Or in certain instances, yes, and in certain
12:57
instances, no, right? And so what you're saying
13:00
is that you've taken that model and kind of turned
13:04
that around a little bit, right? Maybe taken
13:06
away some of the good, bad connotations with
13:09
it and more just come up with levels of use within
13:13
there. Would that be a fair thing to say in what
13:16
instances? I think we've sort of softened the
13:21
approach from something that seemed very didactic
13:25
and very much about the teacher saying to the
13:27
student, stop, or go, go, go, to something which
13:32
is much more about. OK, so here are the types
13:35
of assessments that you already do. these are
13:38
the kinds of things that fit many different subject
13:41
areas. So lots of subject disciplines have a
13:44
stage where they're doing planning or research
13:46
or the students are taking notes and developing
13:48
ideas. Lots of assessments have written outcomes
13:51
where students might be using AI now to assist
13:55
with the editing. And we say, OK, so if we're
13:58
doing a task at a particular level, then let's
14:02
separate that from the whole assessment. So we
14:05
don't want to see And we see this a lot in education
14:08
in general, but big, huge criteria sheets where
14:12
every single component of a project is awarded
14:15
a certain number of grades. And at the end of
14:18
the unit, the student gets assessed five marks
14:20
for research, five marks for development, 10
14:23
marks for the essay, five marks for the evaluation.
14:26
And now we're saying, OK, so how about each one
14:28
of those is its own separate task? And each one
14:32
of those, we discuss the appropriate level of
14:36
AI use. And in some situations, that might be
14:39
no AI. But increasingly, that means in person.
14:43
It means bring students into us. It means move
14:48
the technology out of the classroom, out of the
14:49
way. And then for other things like the research
14:52
stage, well, does it really matter if they get
14:54
their research from Google Scholar, a library
14:57
search, or perplexity? And if that's the case,
15:01
then we need to teach them those skills as well.
15:05
Right. I was having dinner with a composition
15:08
professor from a university that my daughter
15:11
actually attended. And he was very much like
15:16
he has colleagues that were going to go back
15:19
to the old paper pencil. We called them blue
15:22
book tests back when I was in college, you know,
15:26
where they just had had enough and don't want
15:30
to see kids involve any sort of AI in anything
15:33
that they're doing. But at the same time, he
15:36
also teaches poetry and realizes that, you know,
15:40
AI can be that first tool that you might go to,
15:44
to get some good constructive quality feedback
15:47
on a new poem that you're working on and maybe
15:51
find ways to make it way much better than it.
15:54
way much better. Make it good English there.
15:58
That was poetic. Yeah, right. Obviously poetry
16:02
was not my first first class that I took in college
16:06
but but you know like so I could see that the
16:10
ideas of what you're saying were in some instances
16:12
yes and in some instances no but then there's
16:14
those areas in between where it is very practical.
16:18
Yeah and and we you know we're seeing examples
16:21
of things like that coming from writers, coming
16:25
from educators, coming from students themselves.
16:27
I saw an article recently from Punya Mishra,
16:31
I'm sure you know, he's the TPAC guy out at ASU
16:35
and he's been experimenting with creating poetry
16:42
with a large language model and he's written
16:45
poetry and he's written fiction and non -fiction
16:48
and all kinds of weird and wonderful things over
16:51
his career. So it was really interesting to see
16:54
him experimenting with the technology as a way
16:57
of playing with poetry. And just this morning,
17:02
I saw a post from Monash University here in Australia,
17:06
where some of their researchers had been looking
17:09
at where AI fits in the writing process, which
17:11
is obviously something I'm very interested in,
17:13
and just kind of breaking down different tasks.
17:16
Like, OK, so when you are developing your ideas,
17:19
Here's a few examples of ways that you could
17:21
go, some of which use AI, some of which don't.
17:24
Experiment, play around. So Leon, just thinking
17:28
about, like, listening to you and Jeremy both,
17:30
I think one thing that's very clear and it's
17:32
actually been fascinating to watch is all the
17:34
different responses, right, from, you know, college
17:39
level, high school, individual school districts
17:44
and how they have responded to some of the AI
17:47
tools that have been released and the conversations
17:49
around it from like the outright banning to you
17:52
know, those that are are way more open to integrating
17:56
them and Now if it may not be the case that this
18:01
is all across the board, but you know, it seems
18:03
like we're moving more towards Everyone is open
18:06
to it. They're just trying to figure out to what
18:08
extent in all these bare variety of areas, right?
18:13
Is there when you were when you were had the
18:17
paper from BVU, that specific situation or that
18:23
case study, that was with the first iteration
18:28
of the AIS, right? AIS. And when you came out
18:34
of that one or the data that you got from that,
18:36
was there certain things that you found most
18:40
fascinating or you knew were gonna have to be,
18:43
okay, now we need to look at changing the model
18:46
slightly because of this data that we got. Do
18:49
you recall what things stood out to you at all?
18:53
Yeah, so we definitely had conversations following
18:56
that on the lead up to our second version, the
18:59
current version of the scale, which were around,
19:02
first of all, some ambiguity around those middle
19:04
levels, again, which was always the fairly ambiguous
19:08
part, the yellow and the traffic lights or the
19:11
three levels in the middle. One thing we knew
19:15
was that we can't, we the authors of the scale,
19:20
we can't sort of arbitrarily tell people how
19:22
to use AI or not use AI because we're not experts
19:24
in every single subject. I can talk to English
19:27
teachers, literature teachers, writing competition
19:30
teachers, because that's my language, but I can't
19:32
turn around to a physics teacher and say, and
19:34
this is what you should use AI for in your course.
19:37
So one thing that came out of those studies because
19:40
we heard from a lot of different people in different
19:43
faculties was that there are there are tensions
19:48
with trying to make institutional policies where
19:51
if you try to be too kind of you know top down
19:56
people will resist that and they'll say well
19:57
that doesn't work in my faculty because so one
20:00
thing we had to to do was adjust the language
20:04
maybe open up the language a little and one thing
20:06
that I've been working on for the last couple
20:09
of years now certainly is is really getting close
20:12
and working with individual faculties to find
20:15
out, you know, what does AI mean for the maths
20:17
teacher? What does AI mean for the English teacher?
20:20
What does it mean for the science teacher? Because
20:21
it's a different technology in every context.
20:25
And that really stressed the importance of disciplinary
20:28
expertise, which I think can get lost in these
20:32
conversations about AI because people want to
20:35
say, OK, just how do we use the technology? I'd
20:38
say I don't know. What subject do you teach?
20:41
You tell me. Yeah, could you could you see it
20:45
ever get to a point where we look at things at
20:48
the subject level, like almost where would it
20:52
ever be something where you see, oh, we have
20:55
an AIS specifically for math and we have an AIS
20:58
specifically for English language arts. And I
21:02
mean, would it ever get to that point? Or do
21:04
you think that it can stay broad enough where
21:07
the teachers or they would almost just make tweaks
21:12
within the framework of their class? Or do you
21:15
ever see it getting to the point where it'd be
21:17
subject -specific models for those? I think people
21:22
have got to come to their own terms with the
21:25
technology. And one thing that I've been saying
21:29
recently about the assessment scale is it's just
21:31
an idea. It's not a hard and fast set of rules.
21:36
It has to be flexible. It has to be something
21:39
which we, you know, we offer to people and say,
21:41
here you go, do what you want, which is why since
21:43
day one, we've always made sure everything is
21:46
published in open access journals and released
21:48
under a really permissive license. So that the
21:50
first thing people see is, you know, you can
21:52
change and adapt and do whatever you like with
21:55
this scale. Please take it and do what you need.
21:59
Because I don't think there's ever going to be
22:02
one model that fits. this technology in particular,
22:07
because the technology is so broad. I see a lot
22:10
of conversations at the moment around AI and
22:13
assessment, which is still really, to me, they
22:16
seem predicated on two ideas. One is that writing
22:19
is the best proxy for learning. And the second
22:22
is that generative artificial intelligence is
22:24
a writing technology. And I would argue that
22:27
neither of those are true. So we need flexible
22:31
models which account for both the breadth of
22:34
the technology and also the fact that writing
22:37
isn't always the best way to assess learning.
22:40
Oh, I'm sorry, David, go ahead. No, I was going
22:43
to say even just in that little short time span
22:46
of two years or so, we saw it go from just the
22:50
language model to the multimodal where it's got
22:53
audio and video and everything. So then it's
22:56
completely changed the game again to some effect.
23:01
Yeah, I don't see, and you know, multimodal discourse
23:04
and multimodal analysis is one of my, I guess
23:08
it's a hobby because it's not really a huge part
23:10
of my PhD thesis, and I should probably be concentrating
23:13
on the things that are in my PhD. But I love
23:16
multimodal, multimodality and multimodal literacy
23:20
theories. And I just don't see a lot of people
23:23
talking about the fact that, yeah, okay, you
23:25
know, chat GPT can write essays, sort of ho -hum,
23:29
but ChatGBT already, just that one application,
23:33
is multimodal across speech -to -speech, text
23:37
-to -image, image -to -text with image recognition,
23:41
and all variations on those different modes.
23:45
So once they fold Sora into that platform, which
23:50
they will inevitably, and once Sora generates
23:52
audio like Google's VO3 already does now, then
23:56
we've got a an almost fully multimodal platform.
24:00
The only sort of modes that it's not really working
24:02
in is, I guess, spatial, gestural, physical modes.
24:07
But how do I maybe chuck a robot onto the top
24:09
of GBT and see what happens? And it exists in
24:14
the sci -fi movies that we all grew up watching,
24:16
at least. Maybe this is the thing that Johnny
24:19
Ives and Sam Altman are working on. Yeah, maybe
24:21
that is. That's what their combination is really
24:23
for. Who knows? Probably not. It's interesting
24:28
to hear you talk, I think, when I've listened
24:32
to you talk about that writing isn't always the
24:36
best technology and that AI isn't always the
24:40
best technology either. To me, it sounds like
24:44
you're viewing GenAI as a tool. And that's how
24:48
I like to think about it in that, you know, tools
24:51
in education. sometimes work in certain settings
24:55
because you have the right teacher involved,
24:58
you have the right students, you have the right
25:00
content, and all of these different factors need
25:02
to be in place to use that tool. But to me, to
25:06
make that tool work then, you still need the
25:08
human at the center and you need that teacher
25:11
to decide if it's the best right... if it's the
25:15
best time to use that tool at that moment. And
25:18
I think that's what you're saying with some of
25:20
the flexibility within the assessment scale allows
25:23
for the teacher to be the decider there when
25:26
that is appropriate. I think I've written about
25:31
this. I wrote an article. I don't know. I write
25:33
too many articles. I can't remember. Maybe 2023
25:36
end of or in 2024 where the only issue I have
25:41
with the tool idea is that it can be a little
25:44
bit narrow, it can be a little bit limiting and
25:47
it can distance from some of the systemic concerns
25:53
around the technology, the ethical ideas and
25:55
things like that. But I do like to really focus
25:58
people's attention on the fact that it is a technology
26:01
and it's a system level technology. I think like
26:05
the internet and electricity, this technology
26:08
is already being shoved into all kinds of other
26:11
technologies. But the way that we use it, and
26:14
I think what you're alluding to there, Jeremy,
26:16
is that we use it in a specific instance to get
26:22
a specific job done. And where we are falling
26:26
into traps, I think, at the moment is when we
26:28
say, well, it's just a cheating thing. It's just
26:32
a tool for cheating. It's just a tool that students
26:35
are using to write essays. And actually, I mean,
26:37
the way that I use AI most is probably I guess
26:41
as like an interface layer with other technologies.
26:46
So I use chat GPT a lot for writing code, for
26:49
fiddling about with things on my website, writing
26:52
little scripts. I'll use it to generate Word
26:55
documents and Excel files as templates because
26:59
it can use Python to create those files. So,
27:02
you know, I'm using it as a technology. I'm using
27:04
it as software because that's what it is. I guess
27:09
this is a long -winded way to answer your question,
27:12
but I don't want to just talk about it as like
27:16
a tool that does a thing, but I also don't want
27:19
to talk about it as like this almost godlike
27:23
semi -conscious entity that the technologies
27:27
want us to believe. I want to talk about it as
27:30
if it's software, which is sometimes useful for
27:33
doing a bunch of stuff. Sure, that makes sense
27:36
to me. So just to get a little bit into the framework
27:43
and the ethical integration part, why in your
27:51
view do you think that having just saying that
27:56
AI -generated writing as it is a violation of
28:00
academic integrity is an unsustainable way to
28:04
look at things? you know, tie into the AIS in
28:11
the sense that, like, if you talk about writing
28:15
a paper, you know, maybe a teacher says it's
28:18
okay to generate, you know, an outline for your
28:21
paper, but it's not okay to, you know, construct
28:24
the paper, things like that. Why is that just
28:28
saying that any use of it is an outright violation
28:31
of academic integrity? Why in your view do you
28:33
think that that doesn't really work? I think
28:38
it's problematic on a couple of counts. One is
28:42
that it's just totally unrealistic. If we think
28:46
about how ubiquitous the technology already is,
28:49
and like I said at the beginning of the show,
28:51
the commercial push behind this technology is
28:54
immense. And the technology is already part and
28:57
parcel of most of the digital writing platforms
29:01
that we're using. So if we think about just writing
29:03
as in typing words, it's Already Copilot is in
29:06
Word, Gemini is in Google Docs. Those are probably
29:09
the two major platforms that most schools use
29:12
for digital writing. Even if they broaden out
29:15
away from those tools, many of the other applications
29:18
have generative AI built into them now. It's
29:22
built into email platforms. There's really no
29:25
way to avoid it in a digital context. And so
29:27
the only way to say, well, it's automatically
29:30
cheating and therefore we're going to ban it.
29:33
is to say we are banning digital writing, which
29:36
is totally unpragmatic, impractical, unrealistic.
29:42
And then the second thing is that it doesn't
29:45
really do a service to students learning to write,
29:50
because digital writing is already a huge part
29:53
of their world, all of our worlds, I think. you
29:57
know, past a certain age. And I'm old enough,
30:00
Jeremy, as you said, to remember getting the
30:03
internet for the first time. I'm old enough to
30:05
remember the first time I plugged in a modem
30:07
and heard that dial -up noise. And I've just
30:10
written a post on the weekend about, you know,
30:12
I've had a blog for 15 years. I've had my own
30:15
website for, oh God, I don't know, 25 years or
30:19
thereabouts now. I've been making websites. But...
30:24
The internet continues to evolve and artificial
30:27
intelligence is part of that now. Yeah, I just
30:33
to date myself here again to I remember hearing
30:36
that sound when you're trying to dial up and
30:39
then you need to call your parents because you're.
30:43
you know if you're in college and you need a
30:44
little bit of money and you want your parents
30:46
to send you money and then you try to pick up
30:48
the phone and you hear that and then you have
30:52
to yell at your roommate to get off the phone
30:53
because you need to call mom to send you money
30:55
yeah to get off the internet so that you can
30:58
use the phone you know because that dial -up
31:00
was it's amazing My dad used to, he's a plumber,
31:08
so he used to work from home and his business
31:10
phone was the home phone. I used to have to say,
31:12
look, I'm going to try and download this picture.
31:14
All right. So just give me like 40 minutes where
31:17
you're not on the phone. One image. Yeah. You
31:23
had to print everything you could get off the
31:25
internet too, because you didn't know if you'd
31:26
ever find it again. Right. I remember my dad
31:30
telling me I would be on the internet. He said,
31:33
you got to get off the internet. I'm waiting
31:34
for a call, you know, so. No, I'm busy. All right.
31:44
Jeremy, next section there. Yep. So. In the development
31:50
in the in the iterate iterative process of the
31:53
the assessment scale, which I think is, you know,
31:55
like when you're working with AI and you're working
31:58
on prompting AI, I always try to remind people
32:00
it's an iterative process, right? And so I think
32:04
the assessment scale seems to be in that iterative
32:07
mode as well. You've talked about how you're
32:09
already on your second version of it. And I think
32:13
one of the major changes that I noticed and I'd
32:17
like you to touch on is why it kind of went from
32:20
being a tiered leveled approach in, you know.
32:25
at this level, it's no AI at this level, you
32:28
know, the gray areas you talked about, and then
32:31
it's no AI, and it's now much more circular.
32:34
What led to that approach and what do you think
32:37
that signifies? Yeah, like the traffic lights
32:43
changes, we've hit issues along the way with
32:48
that kind of hierarchical structure. of the scale,
32:52
even the language that we use, you know, level
32:54
one, level two, level three, level four, which
32:56
suggests that, you know, you're kind of inherently
33:00
moving up through the levels. And we really wanted
33:02
to emphasize that no level is inherently better
33:07
or worse than any other. I guess you know retrospectively
33:13
and these technologies have developed so fast
33:15
that there's not really much time for retrospective
33:17
but you know this is this is more like an assessment
33:21
design framework than an assessment um you know
33:26
an integrity tool and so there is no real like
33:30
you must do this or you must do this or you must
33:33
do this at various levels it's much more amorphous
33:36
than that and so we try to suggest that you know
33:40
we should be iterating through different types
33:42
of assessments. I think one thing that's come
33:46
out really clearly is that in both higher education
33:48
and K to 12, but particularly that kind of secondary
33:52
college age group, we rely a lot often on single
33:57
moment in time, very high stakes assessments
34:00
in a unit of work. So, you know, we will go for
34:04
eight, 10, 12 weeks teaching a unit and then
34:07
just have one major assessment at the end. that
34:10
tries to do everything. And you could slap that
34:14
at level one, no AI, and kind of go, OK, we'll
34:17
do it as a blue book or an exam or whatever.
34:21
But then you've missed so many opportunities
34:24
along the way to check the student's learning.
34:29
So we really want to suggest, OK, assessment
34:31
is a process. Assessment happens from from before
34:35
the course even starts right the way through
34:37
to after the course finishes. It's not just something
34:40
that you go okay we've got this one essay and
34:42
we're going to do that here. Yeah and there so
34:48
one of the things obviously we've talked about
34:50
this a little bit going back and forth and we've
34:54
referenced like the stoplight model a couple
34:56
times and I don't know if you had read like there
35:00
was a recent paper that I read along with looking
35:02
through those that were put out by you and Dr.
35:07
Perkins and Jasper. The recent article I read,
35:12
which was by Corbin Dawson and Lou, that one
35:16
was called Talk is Cheap and they kind of made
35:20
the argument that there's like two ways to look
35:23
at it. They had talked about the stoplight model
35:26
and then they also put the AIAS in that same
35:29
category and I believe they added one other one
35:33
in there. that I can't recall but they talked
35:35
about those as discursive models and basically
35:38
that they one of the ineffective pieces of it
35:44
is that the way that they're laid out is that
35:47
they just tell students what you can't do or
35:51
can do but there's no real enforcement mechanism.
35:55
Do you have any have you I don't know if you've
35:57
read that but do you have any thoughts on that?
36:00
concept or do you how do you because some of
36:03
the conversation that you've mentioned in our
36:06
interview here is like you obviously realize
36:09
the importance for the fluidity of of something
36:12
like the AIS model and that it is kind of up
36:15
to different teachers in different disciplines
36:17
to figure out how it works best for them. So
36:19
what are your thoughts on like having a very
36:22
clear enforcement mechanism for some of these
36:25
or they talk a lot about structural changes and
36:30
allowing for, I guess, a lot of movement within
36:33
a single assignment that's given to a student,
36:37
and AI being worked in in a variety of ways.
36:40
What are your thoughts on all that, like discursive
36:43
versus structural, and then the whole enforcement
36:46
mechanism concept? Yeah, no, it's a really interesting
36:50
paper. lots of conversations over the past couple
36:55
of years with Tom and Phil and Danny, the authors.
37:01
And particularly with Phil Dawson's work on assessment
37:05
validity is fantastic, really, really helpful
37:08
in these conversations. And that work on assessment
37:11
validity extends long before chat GPT was a thing,
37:15
but has been really relevant since. sort of structural
37:23
discursive conversation I find interesting. I
37:25
think that we do need to have both, and I think
37:28
that everything is sort of necessarily a hybrid.
37:31
And they sort of suggest that in the paper as
37:32
well. They say, you know, structural changes
37:35
by necessity have a discursive element because
37:38
you can't just change the structure of assessment
37:39
and then not tell students about it, which I
37:42
think is fair. I think, you know, one thing is
37:46
that I would say the assessment, our AI assessment
37:48
scale is not just discursive. And we make that
37:52
pretty clear in the second version, because we
37:54
do say quite explicitly, if you're going to do
37:57
a no AI assessment, that means we're getting
38:00
rid of technology, we're probably doing it in
38:02
person. And there are serious implications of
38:04
that in terms of things like accessibility and
38:06
equity, which I think we need to consider. I
38:10
think the models that it talks about, the traffic
38:13
light model, the assessment scale and declarative
38:17
statements is the third one where they say, you
38:19
know, just telling a student what they can and
38:20
can't do or getting a student to sign a piece
38:23
of paper. A lot of those approaches, they won't
38:26
work in isolation. If you just show the assessment
38:30
scale or show, you know, a traffic light scale
38:34
or get a student to sign a piece of paper that
38:36
says, I promise I won't use a high. then there's
38:39
no rigor to that at all. So there do need to
38:44
be structural changes. One thing I have been
38:47
thinking about a little lately is what the difference
38:49
is between those necessary structural changes
38:52
in K to 12 versus in higher education. Because
38:55
one really important distinction, I think, is
38:58
that we often feel like we talk the same language
39:01
because we talk education. We have very, very
39:04
different contexts. And K to 12 education, largely,
39:09
is a social enterprise. It's a non -profit model
39:12
which is designed from many, many years of various
39:18
philosophies. But it's a different model to the
39:21
education model in universities, which is much
39:24
more reliant on import -export. If we think about
39:27
in Australia, for example, higher education is
39:30
a huge export business with international students.
39:33
The economics are different. The business model
39:35
is different. the financial risks and rewards
39:38
are different, and the risks and rewards from
39:41
assessments are different. Because in K to 12,
39:44
if a student cheats with AI, they might, I guess,
39:49
ultimately cheat their way into a university
39:50
course. But in university, if a student cheats
39:53
with AI, they might cheat their way into being
39:55
a lawyer or a doctor or a teacher or a mechanic
39:58
or an engineer. And so I think the conversations
40:03
about assessment security can be a bit more nuanced
40:06
between K -12 and higher education. So yeah,
40:10
there's a lot to dig into in those kinds of structural
40:16
and discursive changes and those conversations
40:18
about assessment security. Yeah, and some of
40:21
it too, you can kind of let me know if I'm hearing
40:24
you right, but the AIAS does provide, if you
40:28
look at the no AI use, it kind of assumes that
40:33
there's more guidance there, right? There's where
40:36
if I was teaching students that particular for
40:40
that maybe part of a lesson where we're doing
40:43
some research or something, the no AI part would
40:47
lend itself to we're working on some things in
40:49
class. We're looking at research books, maybe
40:53
from the library or what have you. And then there
40:55
may be an element later on in that same project
40:58
where uh we might go up to like a level three
41:01
or four just depending on what what we're trying
41:03
to achieve right would that be accurate yeah
41:06
absolutely so what's what's really important
41:11
is that the the educator has the disciplinary
41:14
expertise to make some judgments about where
41:17
ai is helpful and where it's harmful and that
41:20
comes from subject expertise and pedagogical
41:24
expertise and we have to value the teachers judgment
41:29
in saying, OK, sometimes AI is going to be useful.
41:32
Sometimes it's going to get in the way. And then
41:34
we have that conversation with the students.
41:36
And we say, look, in this particular instance
41:38
here, we're going to do this tech -free. We're
41:41
going to do this in class. We've got to be clear
41:43
that does not mean under exam conditions or with
41:47
pen and paper. That can be a conversation. It
41:49
can be a debate. It can be hand -drawn mind maps,
41:52
annotations, draft work, whatever. But the reason
41:56
that we are doing that is because I can't make
41:58
a valid judgment of your capabilities if you
42:01
have access to this technology right now. Later
42:04
on, you know, let's say we've done some initial
42:06
brainstorming. I've listened to what the students
42:08
have to say. I've made some initial informal
42:11
judgments as the teacher. Later on, if we then
42:14
take that material and students go to AI and
42:16
they expand on their brainstorming and planning,
42:19
you know, maybe they come up with an outline
42:21
for a project, then that's OK. you know, that
42:24
use of AI is acceptable. And then at a later
42:27
stage, maybe there's another point at which AI
42:29
is going to get in the way. We go back to doing
42:31
it without. And I think, you know, this is pretty
42:35
much speaking the same language as that Corbyn
42:39
Dawson and Lou Paper, which is to say that there
42:42
are certain things now which we cannot reliably
42:46
assure the learning of the student if they have
42:48
access to the technology. It's hugely problematic
42:52
in an online context, hugely problematic, I think,
42:56
for regional schools that may do a lot of assessment
42:58
online for various reasons. So for me, there's
43:01
a bigger picture conversation in the background
43:03
that needs to be had now about, OK, so where
43:06
is K -12 and higher education different? And
43:08
also, what do we do about this huge scale of
43:13
online learning? which is really beneficial for
43:16
a lot of students to work under those situations,
43:18
but almost impossible to guarantee securely on
43:22
the same scale as face -to -face assessments.
43:27
Yeah, and I think you bring in the variable of
43:30
all of the different levels of students that
43:32
you're talking about here too, from the earliest
43:35
young learners through your higher ed students
43:37
here too. And you touched on some of those differences
43:41
between those different levels and how they would
43:44
would use that. Do you think there's any universal
43:48
standards or universal concepts that you think
43:53
should be consistent regardless of what grade
43:56
level or what age level of student you're talking
43:58
about? Yeah, I mean, if we start with some mutual
44:03
respect, that'd be awesome. If we look at the
44:07
recent articles in the New York Times, there
44:11
was one article that said all of the students
44:14
are using AI to cheat their way through university.
44:17
And then next week there was an article that
44:18
said all of the professors are using AI to create
44:22
and assess all of their assignments. And it's
44:24
this really antagonistic scenario where everybody's
44:28
out to get everybody and the professors are trying
44:30
to catch the students and the students are trying
44:32
to get away with not doing any work. And I think
44:34
if you actually talk to people in the real world,
44:37
it's not that adversarial. There will be some
44:40
students who are exploiting these technologies
44:44
and exploiting the loopholes that exist in the
44:46
system to kind of get away with not learning.
44:49
And those students have always existed. Maybe
44:52
the volume of those students is going to scale
44:54
up because we've just handed them this technology.
44:57
But we haven't created the problem of cheating
45:00
with chat VPT. This is something we've dealt
45:04
with for a while now. So I think we have to start
45:07
with the assumption that the majority of students
45:10
want to do the right thing and that the majority
45:13
of teachers want to help students. So, yeah,
45:17
mutual respect, I think, is a great starting
45:19
point, a good universal. Yeah, and that fits
45:23
for a lot of different things. Yeah, it's funny
45:27
that you mentioned that because we've talked
45:29
in some of our other interviews and just Jeremy
45:32
and I talking before, too, just about it's always
45:36
funny to watch teachers reactions where you show
45:41
them the cool things that they can do with these
45:43
tools to save themselves sometimes. But then
45:45
when you show what the students can do, you know,
45:47
obviously they have kind of a visceral reaction
45:50
to allowing that to happen, you know, but see
45:53
the benefit for themselves. It's always kind
45:54
of funny to see, but. I mean, that's a really
45:59
important point as well, though, because I've
46:01
said in the past that. These technologies, in
46:04
a lot of ways, they're not great learning technologies
46:06
because of the risk of, I guess, circumventing
46:10
learning is very easy, but also because they're
46:12
not particularly reliable. Even if we're using
46:15
the most advanced model, even if we're using
46:17
like an O3 reasoning model, they will still hallucinate
46:20
and make stuff up, which is terrible for learning
46:22
new stuff. And like I've said before, I use it
46:25
a lot for coding and my HTML chops are very rusty.
46:31
And so I'll use chat GPT to fill in the gaps
46:34
in my knowledge. But whenever something goes
46:36
wrong, if there's a hallucination in there, if
46:39
there's some kind of code that doesn't make sense,
46:41
if I'm using Python and it's importing modules
46:43
that don't actually exist, I don't know, because
46:47
I don't have the required level of expertise
46:49
to address those problems. And so to your point,
46:52
David, I think educators look at this stuff and
46:55
they go, wow, look, I know my stuff. I've been
46:58
through uni. I've done my qualifications. I've
47:00
been teaching. English for a few years. This
47:03
could be really helpful for me in doing this
47:04
job that I'm familiar with and that I want to
47:06
kind of automate a bit of this job. And then
47:09
you throw it at students and you go, OK, great.
47:13
What do we use it for? We can't use it to learn
47:15
new stuff, but we can use it to do the appearance
47:18
of learning. So that's where some of that tension,
47:21
I think, exists. And, you know, myself, Punya
47:25
Mishra has written about this recently and a
47:27
few others. I've called this like the expertise
47:30
problem, where you've got to have a certain amount
47:33
of domain expertise and a certain amount of technological
47:36
expertise before you can really get the most
47:39
out of the technology. And students don't necessarily
47:42
always have the domain expertise, and teachers
47:45
don't have necessarily the technological expertise.
47:48
So we're both kind of weaving around each other.
47:53
I've kind of been thinking about that a lot lately
47:55
as well, and I think Is there a curiosity piece
47:58
that fits in there, too? That you have to have
48:00
the curiosity to be able to explore those things,
48:03
to want to learn more, and to seek out the technologies
48:08
or the resources that you need in order to expand
48:11
the learning in those areas that you're curious
48:14
about. And I know I was a social studies teacher,
48:18
and so I feel like I have great content expertise
48:22
in those areas, right? But I don't necessarily
48:26
have the same experience as some of my students
48:28
do in many other facets, right? The kids that
48:31
were, you know, they would take my class because
48:33
they had to, but they would think the rest of
48:35
their day about going into their building trades
48:39
class because they're working on building a house
48:42
and you know how they're going to fit the drywall
48:45
in this certain area that to me is just be way
48:48
over my head. But you know they have that expertise.
48:51
It's just in different areas, right? And it.
48:54
I think what we need to do is allow for students
48:56
to be able to be curious and to use AI to help
49:00
them be curious. Yeah, I think curiosity is a
49:05
really nice way to frame that idea of intrinsic
49:09
motivation, isn't it? It's a part of intrinsic
49:11
motivation. People will get curious about all
49:15
different kinds of things. I mean, personally,
49:16
I can get curious about anything. I love it.
49:19
And I think that's why I was a teacher for so
49:21
long. I will get curious about something and
49:24
then just focus on that to the exclusion of everything
49:26
else for like weeks at a time and bounce from
49:30
one little curiosity to another. Some people,
49:32
they get curious about one thing and they want
49:34
to focus on that one thing. And one of the big
49:36
tensions we have in the classroom, particularly
49:38
in K to 12, is that students do so many courses
49:41
that they are there because they're compulsory,
49:44
because it's on the timetable. And they, you
49:46
know, frankly, they couldn't care less. Like
49:48
in Australia, in the state that I teach in, Secondary
49:51
English is a compulsory part of the course and
49:55
is weighted so that it's worth about a quarter
49:59
of the final grade, the final score that gets
50:02
into university, for example. So if you're doing
50:05
a scored study program where you want to go to
50:08
university, you have to do English and you have
50:10
to do well because it's worth a quarter of your
50:12
overall mark. Probably 80 percent of the students
50:16
in my English classroom on any given year didn't
50:18
really want to be there. You know, we've got
50:21
kids who really want to do maths and physics
50:22
and science and they want to go into engineering.
50:24
They don't really care about analyzing Shakespeare
50:27
or Jane Austen or whatever I'm telling them is
50:29
great. And like convincing those students to
50:33
not offload everything to AI, that's that's the
50:37
struggle. I would say, again, you know, this
50:40
is one of those different tensions between K
50:42
to 12 and university, because in university,
50:44
you would assume. Most students have selected
50:47
most of their courses of study because they're
50:50
interested in them or because they have a career
50:52
pathway in mind. And that means that, you know,
50:56
the focus, the spotlight is slightly different.
50:59
It's not about how do we make this subject engaging
51:02
because the students don't want to be here. It's
51:05
how do we make sure that the students are here
51:06
for the right reasons. So, you know, again, it's
51:09
part of that really complex conversation. Yeah,
51:13
and it's funny that you mentioned the the curiosity
51:16
aspect too because like this is just a side note
51:18
but I know from for myself like there's certain
51:21
things that I never really probably would have
51:24
given any time to if it wasn't for the creation
51:28
of these gen AI tools because You were mentioning
51:31
coding earlier and I've been messing around with
51:33
a ton of stuff there with like website creation
51:35
and things recently and to me I think about like
51:39
well I didn't really know what an API key was
51:42
or how to make a website authenticate emails
51:46
so that it's safer. I didn't know anything about
51:48
that. And I would always be like, oh, yeah, we
51:50
have cybersecurity guys for that. I don't know
51:52
anything about it. But just out of playing around
51:56
with it, I have come to learn what that is. And
52:00
I think one of the fascinating things is that
52:03
everybody can approach these tools with something
52:06
like that in mind. that they want to explore.
52:09
And that's one aspect of it that I find really
52:12
fascinating. But I wanted to kind of ask you,
52:19
too, with the first version coming out in March,
52:24
so over two years ago now. And then you had,
52:27
in December 2024, the second iteration of the
52:31
AIS model. Do you see, and I'm sure you're looking
52:37
at it all the time, but do you see a new version
52:39
coming out again that adds more modifications
52:43
to it that, I know the rest of us kind of wait
52:46
around and we are figuring out ways to implement
52:48
this into our AI guidelines and things, but is
52:51
there anything you can provide for us there?
52:53
Is a sneak peek or changes that you see happening?
52:57
Obviously Mike and Jasper and I in particular,
53:00
we have a lot of conversations about this. One
53:04
thing we're really conscious of is we don't want
53:06
to just bombard people with new stuff all the
53:08
time because they get enough of that from OpenAI
53:10
and Google. The technology companies are great
53:14
at dropping 16 new features every week, so we
53:17
probably don't have to do that. So what I'm focusing
53:21
on at the moment is really sharpening the language
53:24
around that second version and really clarifying
53:27
how I would use the assessment scale. And even
53:33
that might mean at some point saying, OK, so
53:35
in version two of the assessment scale, we have
53:38
these declarative statements for students where
53:40
we say, you may not use AI or you may use AI
53:43
for this purpose and this purpose. But that is
53:46
the start of a conversation, not the end of it.
53:49
We're not going to then give them that piece
53:50
of paper and make them sign off on that and say,
53:53
I promise I have only used because that won't
53:56
work. And some of that language is not really
53:59
necessarily in the academic publication and it's
54:04
evolved since then. So rather than dropping you
54:07
know AIS version 3 or anything like that, for
54:10
me it's more about saying okay so this is the
54:12
version that we're using at the moment and this
54:14
is how I talk about it to teachers and students
54:17
and let's be really clear about what it is and
54:19
what it isn't and you know pieces like the talk
54:24
is cheap paper that we mentioned earlier are
54:25
really helpful for that because they allow us
54:28
to see what other people think. the assessment
54:31
scale is. And then we look at that and we go,
54:33
oh, really, that's not how we've been talking
54:35
about it for the last six months or so. So maybe
54:37
we need to articulate that more clearly. Because
54:40
obviously, these Thomas and Philip and Danny,
54:45
for example, they're still seeing people using
54:48
the AI assessment scale in the same way as that
54:51
traffic light model as a declarative statement.
54:54
We don't see that, but we're working with educators
54:57
ourselves and we have our own opinions on the
54:59
scale. So yeah, clarity and transparency and
55:02
just sharpening the language up around how we
55:04
use the scale I think is really important. And
55:07
I know that we like we work with many districts
55:11
as I mentioned before and so we're kind of looking
55:13
for ways to use things like the AIS to communicate
55:16
with our schools about Like, what are the models
55:22
that you should be looking at? Because the reality
55:24
is that we have so many schools that they haven't
55:28
taken any steps. There really hasn't been any
55:30
work done. And so we want to help them with that
55:33
process by integrating these models. What would
55:38
be your advice for them as far as what steps
55:42
should they take to begin that process? I mean,
55:46
obviously, we're of support to them and we can
55:49
continue to update them as, like you said, the
55:52
clear language comes out and things like that.
55:54
But what would you say to a district that needs
55:57
to get the process started? What would be your
56:00
first recommendation to them? I would say first
56:04
be sort of playful and open to experiment. And,
56:07
you know, these can be high stakes and quite
56:09
charged conversations because we're dealing with
56:12
students and we're dealing with academic integrity.
56:15
But If we can approach it experimentally, one
56:18
thing that I do as a first step with many schools
56:21
is we do this process I call attack your assessments,
56:25
which sounds a bit aggressive, but we take an
56:28
assessment and we look, first of all, is this
56:31
assessment a valid assessment? Does it assess
56:32
what it means to assess? And does it assess the
56:35
content from the course? Before we even think
56:37
about AI, because sometimes they don't, frankly.
56:40
And then once we've established that baseline,
56:43
we say, OK, so where is this task particularly
56:45
vulnerable to AI misuse? And let's go hard. Let's
56:49
get access to the best AI technologies possible.
56:53
Let's take out the chat GPT plus or the Claude
56:56
4 paid subscription. And let's really hammer
56:59
these assessment tasks as if we are the most
57:02
nefarious students in the world deliberately
57:05
trying to break this thing. And then once we've
57:07
done that, let's say, OK, so. Which parts of
57:10
this task do we really want to hold dear and
57:14
to keep sort of separate things and away from
57:16
AI? And which parts does it actually really not
57:19
matter if the student goes hard and uses AI for
57:21
everything? And then we break down the task into
57:24
different individual moments of assessment. And
57:27
then we say, OK, so maybe this is at level two
57:30
and this is at level one and so on. So that's
57:32
the approach that I would suggest is to have
57:35
faculty groups in particular leading those conversations,
57:38
to have the subject matter experts leading those
57:40
conversations, but to do it that kind of experimental
57:43
fashion. Yeah, so if I were to be planning a
57:51
trip to Australia, am I better off asking AI
57:57
to give me recommendations of places to see or
57:59
should I ask Leon? So for the past couple of
58:04
years, I've visited quite a lot more of Australia
58:07
than I had for the last 10 or so. But I tend
58:13
to stick to the major cities. So I don't know,
58:15
maybe chat GPT can point you to a few of the
58:18
less well trodden areas. You don't need to just
58:21
go and see the Sydney Harbour Bridge and the
58:23
Opera House. There's a lot of lovely places.
58:27
Although my son did a month, he studied in like
58:31
five weeks between New Zealand and Australia
58:33
and was there for Australia Day and was at the
58:39
harbor overlooking the Opera House with the fireworks
58:41
and just he had the most amazing photographs
58:44
though. So I do have to say that was pretty cool.
58:48
It's pretty iconic. Yeah. But you're welcome
58:52
to come out and visit me on the farm as well.
58:56
Perfect. We appreciate it. We obviously need
59:01
to wrap up and let you get back to all your regular
59:06
daily tasks and everything. I just wanted to
59:09
give you a chance to give people information
59:13
on where they could find your social medias,
59:17
the work that you're doing, any upcoming things
59:20
that you'd like to promote as well that we could
59:24
post so people can check out. Yeah, so we just
59:29
published a new website at aiassessmentscale
59:32
.com, which has all of the open access journals,
59:35
all of the CC materials, the assets from the
59:39
scale itself. Everything's all in that one place
59:41
now, which is good, because prior to that, it
59:44
was all on my website, which is at leonfirst
59:47
.com. And that leonfurs .com slash blog is where
59:51
I do all of my writing. And then you'll find
59:55
me on LinkedIn. I spend way too much time on
59:58
LinkedIn. I'm always open to fielding DMs and
1:00:02
answering questions there. David, I think that's
1:00:04
how you got in touch with me originally. As I
1:00:06
say, you and me both, we're probably on the same
1:00:08
amount, so. Yeah. But I will say just on that,
1:00:13
I mean, to the people listening, LinkedIn can
1:00:16
feel a bit cheesy and a little bit sort of like
1:00:20
it's filled with business hacks trying to sell
1:00:22
you marketing deals. But it's been one of the
1:00:24
best communities I've seen online for these conversations
1:00:27
amongst educators. And a lot of people left Twitter
1:00:30
and came to LinkedIn. So there is a really strong
1:00:33
community on LinkedIn of educators talking about
1:00:36
AI. Yeah, it was actually for me, I can just
1:00:39
speak to this. my own experiences, it was quite
1:00:42
eye opening to the amount of information and
1:00:46
people that are on their sharing ideas is incredible.
1:00:50
Particularly in our spheres, like there's just
1:00:53
so much like rich content and things that you
1:00:56
can come across in the clean. I mean, that's
1:00:58
how I ended up reaching out to you is, you know,
1:01:01
coming across some of the different things. And
1:01:03
I talked with Mike and Jasper as well. And yeah,
1:01:06
it's great for that. So We'll be sure to include
1:01:09
the links in our description for the YouTube
1:01:12
video and the podcast for anybody that wants
1:01:15
to navigate over to Leon's blog. Definitely check
1:01:18
out the AI assessment scale. I'd highly recommend
1:01:21
reading the papers and everything as well and
1:01:24
just following their work. It's really incredible
1:01:26
and eye -opening. Leon, we just want to thank
1:01:30
you again for taking the time out of your busy
1:01:33
schedule to join us. and having you on the Integrate
1:01:36
This podcast. So thanks again. Thanks for listening
1:01:44
to Integrate This. Want to take your learning
1:01:50
further? Head over to our YouTube channel for
1:01:53
quick tutorials, deep dives, and more ed tech
1:01:56
tips you can use right away. Don't forget to
1:01:59
subscribe and catch new podcast episodes every
1:02:02
Thursday on Apple. Spotify, or wherever you get
1:02:06
your podcasts. Until next time, keep exploring,
1:02:10
keep innovating and keep integrating this.