Latest / Integrate This! Ed Tech Podcast / Can AI Truly Redefine Learning? Becky Keene Suggests AI Optimism in Education
Transcript
- 0:00Welcome to Integrate This, your go -to EdTech
- 0:03podcast where curiosity meets classroom innovation.
- 0:07Join your hosts, David Berner and Jeremy Mikla,
- 0:09as we explore the rapidly evolving world of educational
- 0:12technology. From game -changing tools to insightful
- 0:15conversations with today's top EdTech voices,
- 0:18we're here to help educators stay informed, inspired,
- 0:21and ready to innovate. So tune in, level up,
- 0:25and let's integrate this. Welcome to the Integrate
- 0:31This EdTech podcast. Today we're very excited
- 0:34to be joined by AI optimism author, Becky Keene.
- 0:40Very excited to have you with us today. Becky
- 0:43Keene is an educator, author, and speaker focused
- 0:46on innovative teaching and learning. She specializes
- 0:50in instructional coaching, game -based learning,
- 0:52and integrating AI into education to empower
- 0:55students as creators. She has developed eSports
- 0:58programs for schools and explores immersive learning
- 1:01through games. Becky speaks globally on AI and
- 1:05education and has spent over 20 years designing
- 1:07professional learning experiences for teachers.
- 1:10A national board certified teacher, ISTE certified
- 1:13educator and certified instructional coach. She
- 1:17has spent 15 years teaching, coaching and leading
- 1:19programs in public schools. She's the author
- 1:22of the book AI Optimism and holds a master's
- 1:26in ed in early literacy. So welcome, Becky. Very
- 1:30happy to have you here. Join the integrate this
- 1:32podcast. Can you give the audience a little bit
- 1:35more insight into your background and how the
- 1:38last 20 years here have gone for you? And now
- 1:42you've got the book release as well. So. Yeah,
- 1:46thanks. Thanks. First of all, thanks for having
- 1:47me and. Happy to share a little background. Sometimes
- 1:51people ask about my like career journey. I've
- 1:55spent 25. This is my 25th year in public supporting
- 1:59public education, which is nuts to me that it's
- 2:04been that long. But I taught elementary school
- 2:07and then I taught middle school and I was really
- 2:10fortunate to work with people, leaders, school
- 2:14leaders, system leaders who allowed me to do
- 2:18things I was excited about, who didn't kind of
- 2:21keep me in a box or force me to follow some curriculum.
- 2:25A lot of the mantra early on in the 2000s was
- 2:28if your test scores are good enough, you can
- 2:33do whatever you want. And so that was really
- 2:35helpful for me. And I got to do a lot of really
- 2:38innovative and creative things, which really
- 2:40set the stage for me understanding that there's
- 2:43a lot possible in education. if we move some
- 2:47of those challenges out of the way. And so that
- 2:50was really fun. And I got to be a part of a lot
- 2:52of different proof of concept programs and pilot
- 2:54programs because I said yes. I just kept saying,
- 2:57sure, I'll do that with my students. Yeah, I'd
- 2:59love to try that out. invited at some point to
- 3:05work at a system level in my school district
- 3:08south of Seattle, Washington to help support
- 3:11and technology district wide and really scale
- 3:14the programs that I had been piloting in my own
- 3:18classroom with some other teachers as well. And
- 3:20so we worked on that for about 10 years and then
- 3:24I had the opportunity to join my current. company,
- 3:28I2E, and help schools and systems, primarily
- 3:32North America, but we work with companies and
- 3:35organizations worldwide to help them implement
- 3:37technology in new and innovative ways. In the
- 3:42business world, we focus on productivity and
- 3:44creativity. And then in the education space,
- 3:46obviously, we focus on best teaching and innovation,
- 3:49which is a ton of fun. So that's a little bit
- 3:53about me. And somewhere along the way, I became
- 3:54a LinkedIn learning instructor. which is fun
- 3:58and I have two books now and a blog and and a
- 4:02newsletter and I do all sorts of oh and TikTok
- 4:04walks. I do all sorts of little things but it's
- 4:07it's been really fun to get out there and get
- 4:10connected with educators like you from all over.
- 4:13Yeah and that's that's really awesome. I've seen
- 4:15a couple of your TikTok walks that you've done
- 4:18here in the last couple weeks or so. But it was
- 4:21really cool for me to be able to connect with
- 4:24you at ISTE this year. I know I came up, you
- 4:27were doing a like a tabletop presentation and
- 4:31I came by there and it was nice to be able to
- 4:35connect with you there. But overall, what would
- 4:36you say about an experience like a conference
- 4:39like that? You know, being able to not just kind
- 4:43of get your message out there about your book,
- 4:44but to be able to like find networks of people
- 4:47who kind of like are on the same way. of length
- 4:50and can kind of like get to get to understand
- 4:53kind of what it is that you know it helps make
- 4:57Ed technology kind of supportive of of increasing
- 5:00student learning you know I mean oh absolutely
- 5:04and and I love ISTE it was kind of a dream of
- 5:07mine to go when I was working for my school system
- 5:10and I finally got up the guts to ask and my supervisor
- 5:15said Of course you can go. Why didn't you ask
- 5:17before? My teammate and I were like, oh, I guess
- 5:21we should have asked earlier. I don't know why
- 5:23we thought it was beyond us. So I've been attending
- 5:27ISTE for over 10 years straight, and it's an
- 5:30amazing event. I love the energy. I love connecting
- 5:34with people. It's obviously... one -stop place
- 5:37to be in June for EdTech. However, I will say
- 5:41that in the past five years, I've spent a lot
- 5:44of time in smaller regional EdTech conferences.
- 5:49In my role at I2E and also as a featured speaker,
- 5:52I get to go out and attend some of these other
- 5:54events. And I just want to add that as a word
- 5:57of encouragement for all of the listeners that
- 6:00going to a smaller regional event, I think is
- 6:04often more impactful. because you have more time
- 6:08to really connect with people and network and
- 6:12there are fewer sessions to choose from which
- 6:14actually makes life a little bit easier. I mean
- 6:18ISTE is nuts as far as like at any given moment
- 6:22there are 15 things you could be doing and that's
- 6:26exciting but I do find high value in the regional
- 6:30tech conferences both in connection and time
- 6:33and just the availability to really dig deeper.
- 6:36with individuals and with the vendors who are
- 6:39there. It's just not quite so chaotic. So there's
- 6:42a place for both. It's really funny that you
- 6:45bring that up because last week David and I both
- 6:49hosted regional tech conferences in our different
- 6:52parts of Minnesota, where it was just exactly
- 6:55like you're describing it, fewer sessions, but
- 6:57it's much one. It's much easier for presenters
- 7:00because they're dealing with other teachers and
- 7:03educators that are like from their area and from
- 7:07their schools and deal with similar issues. But
- 7:10at the same time, you know, it's it's also like
- 7:12you said, the number of choices is just it's
- 7:15more manageable and it's easier to find, find
- 7:18a track that that you could go down, you know
- 7:20what I mean? Yep. Yep. Yeah. So I really enjoy
- 7:23those. My regional is, excuse me, called NCCE
- 7:28here in the Seattle area. And I've been attending
- 7:30that since 2004, I think. And I've presented
- 7:35there most of those years. And it's just a great
- 7:39event. It's not huge, but you get a lot done,
- 7:42so. Yeah. And I know what you're saying about
- 7:45ISTE. I think I've literally had FOMO, for another
- 7:49session while sitting in a really good session
- 7:51at ISTE. So that tends to happen. But yeah, and
- 7:55Jeremy and I, like you said, we do a number of
- 7:58trainings too. And one of the things that I saw
- 8:01you posted recently that you start some of your
- 8:04trainings with that I found really interesting.
- 8:07was the AI personalities concept that you posted.
- 8:11I had seen that on LinkedIn, and I thought it
- 8:13was a really great bell ringer, I guess, if you
- 8:17want to call it, for those that are attending
- 8:19the training to identify. Are you like one of
- 8:22these personalities? There's five on there. It's
- 8:24like, are you one of these personalities? Or,
- 8:26you know, are you a combination of them? And
- 8:30it did a nice job of laying out the different
- 8:32feelings that people have around AI. Why do you
- 8:35think that that was like an important exercise
- 8:37to include in your trainings recently? I think
- 8:42it's really important that we validate the emotions
- 8:45attached to generative AI. And as an AI optimist,
- 8:50I don't want to focus on fear or negativity,
- 8:55you know, honestly, there's a lot of negative
- 8:56around generative AI. And I want to recognize
- 9:00the challenges and the issues around privacy
- 9:03and safety and bias and the things that are true
- 9:06and real. But I also don't want to dwell there.
- 9:09And so I've found that giving people an avenue
- 9:12to kind of claim their excitement and their concerns
- 9:16in a way that was The word I guess is inclusive,
- 9:22right? Like, hey, we all have opinions about
- 9:24this and they all add value to the conversation
- 9:26and it's okay to say like, hey, I really care
- 9:29about the human element. There's a flip side
- 9:33to all of those AI personalities where we're
- 9:35concerned, but I think it frames it in a positive
- 9:38light where like this is what you're bringing
- 9:39to this conversation. Last week I was working
- 9:43with a school in Southern California and one
- 9:46of the teachers just kind of had a a moment during
- 9:49one of our breakout sessions, and he was kind
- 9:53of upset. I mean, I could feel his frustration
- 9:57around some of his expectations around AI and
- 10:00how those weren't met and how students are misusing
- 10:02it at times. And I just stopped the conversation
- 10:06and said, thank you for sharing. I think other
- 10:09people in the room have similar concerns, and
- 10:11it's really important we talk about. These things,
- 10:14I said, you sound like a traditionalist and there's
- 10:17room for you. You don't have to feel like you're
- 10:21not a part of this conversation. And he came
- 10:23up to me after the session and thanked me for
- 10:25how I handled it because he didn't feel like
- 10:27he was an outsider. He was a part of the community
- 10:30that we were learning together. So I just think
- 10:33the personalities lend themselves to that. type
- 10:36of culture in the school system. And of course,
- 10:41whenever people feel included and validated,
- 10:43that's a good day. Right. And to be honored is
- 10:47just going to, you know, to know that their opinion
- 10:49matters and that you're honoring their position
- 10:51and where they're at currently is just going
- 10:54to make it easier, I think, for them in the long
- 10:56run to start to maybe come over to that optimist
- 11:00side gradually. Right. But, you know, like since
- 11:05you brought up the you being an AI optimist here
- 11:08and why don't we why don't you spend a minute
- 11:12here and tell us you know what was kind of the
- 11:13inspiration behind the the writing of AI Optimism?
- 11:18Well I have to tell you that the title came last
- 11:21and a lot of people are surprised at that yeah
- 11:24I didn't like have this vision to write about
- 11:26AI Optimism and then write a book. I wrote the
- 11:29book and based on what I felt was missing, honestly,
- 11:36kind of this hole in the space of education AI,
- 11:40where we have some people doing a great job surfacing
- 11:44concerns, and we have some people doing a great
- 11:46job talking about, you know, tools and what those
- 11:52look like in education. And I felt there was
- 11:56this gap of not understanding kind of the trajectory
- 12:01of AI, if you will. And that's why I use the
- 12:04SAMR model in the book because I just, I saw
- 12:09so many educators scrambling to quote, use AI.
- 12:13And that's the wrong approach. And that has always
- 12:16been the wrong approach to me as an instructional
- 12:18coach. We shouldn't be focused on what tool am
- 12:21I using or how much am I using technology. It's
- 12:24really about meeting student needs and developing
- 12:28learning experiences that transform and empower
- 12:31learning and so I felt like people were just
- 12:35you know getting kind of distracted by the the
- 12:38sparkly glitter of AI and all these all these
- 12:42little apps popping up and I was seeing social
- 12:44media posts like I want to use AI with my students
- 12:47where should I start and and those are heartbreaking
- 12:49to me because it's the wrong lens. It's tool
- 12:54centric, it's use centric, you know, instead
- 12:57of thinking about what I want to achieve with
- 12:59my students. And so as I started having that
- 13:02roll around in my head, I felt like the SAMR
- 13:04model was a really good tool to help people understand
- 13:07maybe where they had started with using AI and
- 13:10then what was possible to really redefine. And
- 13:14then after I started looking at all these different
- 13:16use cases, I started thinking about how they
- 13:21fit into these categories. And that's where the
- 13:23AI Optimism Framework started. So this whole
- 13:25thing kind of was designed in a backwards way,
- 13:29right? Like backwards design. Just give them
- 13:32a shout out right now. Because I had the ideas
- 13:35and then I thought, okay, these fit into six
- 13:37categories of how we use, like the things we
- 13:40have to do in education every day. We have to
- 13:42design and create and support and analyze and
- 13:45evaluate and manage. And those are the things
- 13:46we're doing. And so how can I help educators
- 13:49categorize and say, OK, if I want to evaluate
- 13:53student use and student growth in my classroom,
- 13:57what are some tools I can use and how much do
- 14:00I want to be involved? Is it just a substitution
- 14:03level or am I going to really modify what I've
- 14:06been doing or redefine something completely unique
- 14:09and different? And so the And then I had to write
- 14:12the beginning of the book, which is like, OK,
- 14:14if you're going to do all those things, I need
- 14:15you to understand first what I believe about
- 14:19privacy and praxis and prompting. And so really,
- 14:23the whole thing came about backwards. And the
- 14:25very last thing was the title, which was a conversation
- 14:29I had with my editor when he said, OK, but you
- 14:32really need a title because we need to work on
- 14:35a cover and. We have like the book was already
- 14:37through developmental and kind of far along the
- 14:40process when I finally landed on what I wanted
- 14:42to call it. And that's simply because the middle
- 14:47of the book to me was what I wanted to say. And
- 14:50then it took me a while to figure out how to
- 14:52put that into one succinct to two or three words.
- 14:57And the SAMR model has been around for a while.
- 15:01And that was developed by Dr. Ruben Pantadora,
- 15:08who did the forward for the book as well. And
- 15:12so what was that like? I got two questions, I
- 15:16guess. What was it like talking with him about?
- 15:19this process or, you know, what conversations
- 15:22did you have with him being that you were going
- 15:24to use the SAMR model? Ultimately, why did you
- 15:27feel like that would be the most complementary
- 15:29model to what you were trying to convey in the
- 15:32book? Well, I actually started field testing
- 15:37SAMR within AI a year over a year before the
- 15:41book published because I'd had it kind of rumbling
- 15:44around in my head. So. Some of the things I did
- 15:47is I started presenting with the SAMR model infused
- 15:52in my AI workshops. So we just talked about NCC,
- 15:56for example, February, 2024. So a year and a
- 16:00half ago now, I did a couple of workshops for
- 16:02NCC as just a presenter at the conference and
- 16:05I infused SAMR into the AI workshop I was leading.
- 16:11rave reviews like people are coming up to me
- 16:13saying this makes so much sense. I'm so thankful
- 16:16I it makes so much sense now to you know think
- 16:19about my purpose first and then go back and look
- 16:21at how I'm using the tools and just got amazing
- 16:25feedback. I did the same thing at idea con last
- 16:28year. I was there as a featured speaker and I
- 16:29just I just put it in and the feedback I was
- 16:33getting was that it was really helpful. Educators
- 16:35were like this is beautiful. And then Matt Miller's
- 16:39team at Ditch that textbook invited me to write
- 16:41a guest blog. And I said, hey, I'd like to do
- 16:45Sam or AI. Like let's see how that lands and
- 16:47put that out really broadly because he has massive
- 16:50reach. And so I did that guest blog and that
- 16:53went out and he had people later come up to him
- 16:55and say, this could be a book. Like Becky's blog
- 16:59makes so much sense. Have you thought about doing
- 17:01a book? And Matt and I had had a little side
- 17:03conversation and he was like, I think you're
- 17:05going to get something for that. because he was
- 17:09kind of in the loop on the whole book idea earlier.
- 17:12And so I got really good feedback from educators
- 17:16that that was the right path to be going down.
- 17:20There's a lot of different models we can use
- 17:22that all add value. But I think for what I wanted
- 17:25to communicate, it really was a perfect fit.
- 17:29Yeah, and I think the SAMR model is great just
- 17:31in terms of like the depth in which you are integrating
- 17:34technology and has been for a long time. But
- 17:36what I really like is how you've applied AI to
- 17:39that and be particularly because it's new. And
- 17:42a lot of people are kind of like, how do we use
- 17:44this? And to what degree? And, you know, with
- 17:47our students, what does that look like? So and
- 17:49I'm sorry, you had a two part question and I
- 17:52can't let the conversation move on without giving
- 17:54credit to Dr. Quintedora. He He was so great.
- 18:00So I had a mutual colleague connect us and he
- 18:06didn't reply right away and that's totally okay.
- 18:08Like I don't reply instantly to everything that
- 18:11hits my inbox either. Sometimes we're drowning.
- 18:14So I gave it a little time and then I sent him
- 18:16another little ping and said, hey, you know,
- 18:18just checking in, is this something you'd be
- 18:20interested in? And he got right back to me. He
- 18:23was like, yes, I thank you for the second email,
- 18:26which is a great reminder that like it's OK to
- 18:28send a second email sometimes. He's like, thank
- 18:31you so much for getting in touch. Yes, I would
- 18:33love to be a part of this. You know, let me know
- 18:35what you need. And so I just knowing that life
- 18:39is busy for everyone and I'm sure he has a lot
- 18:42on his plate. I didn't I didn't really want to
- 18:44hold my breath on like it would have been OK
- 18:47with me if it came to the time he was ready to
- 18:49write a forward and he was. swamped with something
- 18:51else in life, like that would have been OK. My
- 18:54publisher was super excited. They're like, this
- 18:56is amazing. He agreed to write it. And then he
- 18:59did. We just were in communication from then
- 19:02on. As far as the timing, I would give him updates
- 19:05on, OK, I think I'm going to have a really solid
- 19:08manuscript to you by this time and so on. And
- 19:10he was so sweet. He wrote me back after he read
- 19:14it the first time and was like, he said, I have
- 19:16seven pages of notes. it's can I have a little
- 19:19extra time to write the forward like he was so
- 19:22complimentary and positive about how I reflected
- 19:25his work in an AI sense and that was just really
- 19:29exciting for me as well to have him be a part
- 19:32of the project and lend his voice to something
- 19:34that I think really makes sense. And definitely
- 19:39it does and I I'll admit here that I didn't really,
- 19:44when I first started even presenting on AI, really
- 19:47consider the SAMR model as something that should
- 19:49be integrated through the training of teachers
- 19:54in the different levels and where they're at
- 19:56and how it can even change depending on the type
- 19:59of activity and stuff you have. But I remember
- 20:01presenting on the SAMR model when I was still
- 20:03in the classroom and doing some of the the part
- 20:06-time technology integration stuff. When we were
- 20:10first going one -to -one and trying to get students
- 20:13and teachers to use their devices more purposefully
- 20:16and not just as that substitution tool to, hey,
- 20:21don't just write your notes on your notebook
- 20:23paper anymore. Let's type them on our iPad or
- 20:27like some of those things. I think one of the
- 20:30things that I really appreciated too from the
- 20:32book is how you make that ease of entry into
- 20:37AI use through the SAMR model, I think much more
- 20:41accessible to teachers. But I'm also kind of
- 20:44wondering, do you think there's like, what sort
- 20:46of barriers do you think teachers face when it
- 20:49comes to adopting AI into their practice, even
- 20:52at like some of those basic substitution levels?
- 20:56Oh, so many, so many challenges. I think if I
- 21:01could categorize them into one, Topic it would
- 21:04be lack of trust and that has several different
- 21:08pieces to it. One is not trusting the tool to
- 21:12keep their data safe or their students safe.
- 21:15Another one is not trusting that it's not going
- 21:18to be used to train the model or use their data
- 21:23without their permission long term. So short
- 21:25term and long term. Then there's the lack of
- 21:28trust that it will return great results that
- 21:30are accurate and reliable. I hear a lot from
- 21:34educators how they've tried using a tool, you
- 21:38know, whatever it is, it doesn't pick an LLM.
- 21:41And they've been disappointed with the results.
- 21:43And so they've chosen to walk away from all of
- 21:47AI because of that experience. And we know that
- 21:50in education specifically, having been a trainer
- 21:53in education for 20 years, if educators don't
- 21:57see valid, reliable, trustworthy, responsible
- 22:00use on the first They're not coming back to that
- 22:04tool. Like that's pretty universal. And so it's
- 22:07been really difficult because generative AI is
- 22:10still new in the realm of the universe, right?
- 22:14We're two and a half, three years in. And the
- 22:18tools are changing at lightning speed. And so
- 22:22when you've seen something work or not work last
- 22:25week, it might be completely different this week.
- 22:28And I've been talking to educators about I'm
- 22:31just calling it AI grit and reminding them that
- 22:3525 years ago they had to learn how to search
- 22:37Google, right? We had to learn how to type in
- 22:40queries and like all the hits were wrong and
- 22:43it didn't work the first few times. And we've
- 22:45been teaching students that ever since in digital
- 22:47literacy lessons. And for some reason, I think
- 22:51we've gotten really comfortable with Alexa and
- 22:54Siri and the tools that we've had for a long
- 22:57time. And so this tool, when it didn't respond
- 23:01the way we expected it to, to throw it out, feels
- 23:05a little bit easy, you know, like easy quitter.
- 23:10And we don't, you know, not everyone has time
- 23:12to learn and learn prompting. And I've heard
- 23:14that argument, but I've also just kind of come
- 23:16back and said, hey, you spend the time to learn
- 23:20what's important to you. When you got an air
- 23:22fryer, You know you you got out all these recipes
- 23:25and you figured out how to use it and you made
- 23:27sure your Trader Joe's orange chicken turned
- 23:28out amazing like you put the time in so If you
- 23:33want to use these tools like you're gonna have
- 23:35to put some time in and that's why my third core
- 23:37principle is prompting because the reality is
- 23:40it doesn't always do what we think it's going
- 23:42to do and Throwing it out because it didn't return
- 23:45the experience. We hope is the wrong approach
- 23:49And it was definitely in the classroom teaching
- 23:53kids how to do Google searches. I'm that old,
- 23:55too. Yeah. Oh, same. I remember even like I and
- 23:59I when I would first kind of talk to teachers
- 24:01to try to remind them or to maybe build their
- 24:05trust or to tell them that they feel supported,
- 24:07that if we want to go way back, you know, and
- 24:10talk about even like when I was in school, the
- 24:13librarian in the school library, because we didn't
- 24:16even call the media centers then was showing
- 24:18us how to use the card catalog. and how we would
- 24:21find journal and periodical articles in the Reader's
- 24:25Guide to Periodical Literature. And those have
- 24:28their own strategies that you need to employ
- 24:31to get the best results possible for the information
- 24:34that you're looking for. And I really do feel
- 24:36like with prompting and to get the best results
- 24:40that you're looking for, there are skill sets
- 24:42there that you need to learn also. And so I'm
- 24:45right with you with what you're talking about
- 24:48when it comes to that. Appreciate that. Yeah,
- 24:51and I like I like the AI grit term too because
- 24:54like I feel like as much as I've used AI and
- 24:58these different programs like I feel like failure
- 25:01is Almost essential to the whole process because
- 25:04it's like I can't you know, every single time
- 25:07I go to it I basically think like it's gonna
- 25:09take several iterations of me talking to this
- 25:12and Explaining what it is. I want to actually
- 25:14get what I have in my mind, you know what it
- 25:17is that I want and And just to expand on that
- 25:21briefly, different AI tools respond different
- 25:24ways. And I've heard lots of people say, you
- 25:27know, you can think of them as different friends.
- 25:29We don't want to overhumanize AI, but it is a
- 25:32nice lens to be thinking about. You know, I have
- 25:34one friend I might ask for for advice on making
- 25:38a meal for my family because she cooks really
- 25:41well. I might have another friend that I ask
- 25:43for advice on a fitness plan or another friend
- 25:46that I'm like, hey, what's a good vacation spot?
- 25:48So you know looking at not just using one a lot
- 25:52of us are we have this core desire to find like
- 25:55the one ring to rule them all and we think there's
- 25:57one tool is going to do everything and with generative
- 25:59AI I just I don't see it I see different tools
- 26:03fitting different needs and that takes time to
- 26:05learn as well so it can feel so overwhelming
- 26:08for educators who are already tapped out in you
- 26:11know the amount of things they're supposed to
- 26:13do and manage and learn so that was another hope
- 26:16of my book is that they could use it as a guidebook
- 26:19and just flip right to a strategy they were interested
- 26:22in, see a couple of tool recommendations or prompt
- 26:25recommendations and go from there. Instead of
- 26:27having to feel like they're starting with this
- 26:29empty slate, at least they're getting something
- 26:31to start with. Yeah, and it's been interesting
- 26:35too, like we've had a lot of conversations with
- 26:37our like our special education department on
- 26:40the various things that they can do to save time
- 26:43and work with students on. And one of the things
- 26:47that you bring up in the book is personalized
- 26:49learning. And so you look at that as one of the
- 26:54boons of AI, right? The personalized learning,
- 26:58how can that be something that can really be
- 27:02utilized by any teacher, in your opinion? Well,
- 27:07I so I think that term is being overused a little
- 27:11bit at the moment Like everybody's like and then
- 27:14AI can personalize learning and without a an
- 27:17actual application of what that looks like I
- 27:20think it can be really difficult to understand
- 27:22what that even means I've seen it work a variety
- 27:27of different ways. So even at a substitution
- 27:29level, I just Talked about this with some math
- 27:32teachers last week. I said hey give give like
- 27:37a chat GBT or a copilot or whatever your kids
- 27:40have or a Gemini, give it what you're trying
- 27:43to learn in math and then their interests. And
- 27:47it created these story problems that were maybe
- 27:49more relevant for the students. Now, that's just
- 27:51a strict substitution level. And one of the teachers
- 27:54said to me, I already have a book full of story
- 27:57problems. Like, why would I do this? And I said,
- 28:00hey, I think you missed my point. And that's
- 28:03good feedback for me as the presenter. Like,
- 28:04I think you maybe missed something. And I said
- 28:07the whole point here is that students, which
- 28:10is what I said, and he like missed it, but is
- 28:13that students would be writing the story problems
- 28:17based on them being interviewed by the AI and
- 28:20coming up with story problems that they're actually
- 28:22interested in. And we were talking about graphing
- 28:24linear inequalities. And so with graphing linear
- 28:29inequality, two of them specifically, now there's
- 28:32a decision point. And so they were actually writing
- 28:36Story problems that caused them to question their
- 28:39own value system and think about what they would
- 28:42choose based on a scenario. Like there was so
- 28:44much more here than a story problem. And it was
- 28:46something that resonated with those individual
- 28:48students. So in my sample lesson, the students
- 28:53were then telling a story around, okay, if I
- 28:56have two choices in how to spend my afternoon,
- 28:58either doing homework or. a hobby with friends
- 29:02but I only have a finite amount of time and I
- 29:04have to get my homework done like where's that
- 29:06intersection that's a that's two linear inequalities
- 29:09so I'm like isn't that more interesting for your
- 29:14students to like talk through and think about
- 29:16when it actually connects to their own life than
- 29:19a story problem out of a book and so with all
- 29:24that in mind I think personalized learning has
- 29:26this huge range right it can be as simple as
- 29:28just literally personalizing the content that
- 29:31you're trying to generate, but it can be as powerful
- 29:33as a full personalized learning pathway where
- 29:36students are being taken along a journey of growth
- 29:39from their own starting point through meeting
- 29:45standard at a certain level. And there's specialized
- 29:47apps that do that that are really good. That's
- 29:50what they're designed for. So personalized learning
- 29:52has kind of gotten this overkill at the moment.
- 29:55But the reality is that there are a wide variety
- 29:58of tools and options that fit in that umbrella
- 30:01modality. And AI can definitely, I think, or
- 30:05generative AI, assist you in finding ways to
- 30:09or to help a teacher connect the content to the
- 30:12interests of their students. In fact, we just
- 30:15had an interview the other day with an instructional
- 30:18technology specialist from out east in Massachusetts.
- 30:22Her name is Liz Boese. And she has a project
- 30:26that she works with elementary age teachers in
- 30:30her buildings that she helps to support called
- 30:33the Perfect Book Project. And really, it is a
- 30:40reading lesson, a reading project, and not a
- 30:43writing or an AI project, right? But it's to
- 30:46have students work with AI. to help create books
- 30:51and stories with plots and characters and subjects
- 30:55that they're interested in. And then they help
- 30:59to design outlines for these chapters of the
- 31:03book. The students work to add their own ideas
- 31:07in with ones suggested by AI. And in the end,
- 31:11they produce a manuscript that they turn around
- 31:15and they go to like staples and they have it
- 31:17published. So they get a physical copy. of the
- 31:19book as well. Um, of those books in their o So
- 31:24now you're building a of these different things
- 31:28personalized, right? And ma they're using those
- 31:33to hel better readers, um, which i how you're,
- 31:39you're able concepts and put that all same thing
- 31:43could apply li like you're talking about. So
- 31:51Becky, one of the things that you brought up
- 31:53was prompting. And I know when I do trainings
- 31:55and stuff, I have a prompting strategy that I
- 31:58like to encourage people to follow. And I know
- 32:02there's several acronyms, because we're teachers
- 32:05and educators. We have an acronym for everything.
- 32:08But I'm kind of curious, is there a frame that
- 32:11you kind of think of when you are talking with
- 32:15educators about how to become better prompters?
- 32:18Or is it maybe even, like I'll give you maybe
- 32:21even part of my answer to this too. I'm starting
- 32:24to become a little more of a convert to the conversational
- 32:27method of just having that dialogue developed
- 32:30too. But I don't know if that's the best either.
- 32:33But is there kind of a format that you tend to
- 32:36follow? There is. I do a whole session called
- 32:40The Art of Prompting sometimes when invited.
- 32:44And we dig into just a myriad of ways you can
- 32:47prompt powerfully. In the book, I use Saba Kinway's
- 32:51model Spark because she's just a wonderful advocate
- 32:56for using AI. And she kindly said, yes, I could
- 33:00include that in the book as one of the models.
- 33:01And I wanted to promote her as a woman in AI.
- 33:04But I do a lot of co -pilot training in my day
- 33:09job. There is a model that Microsoft uses that's
- 33:14just goal context expectations and. Oh my gosh,
- 33:21why? Why am I playing on the fourth thing? This
- 33:23is weird. I do it all the time. Source there
- 33:27we go. Thank you. Goal source expectations in
- 33:30context and so that kind of just. Is the simplest
- 33:34to me and it doesn't require anything like fancy
- 33:38or interesting. It's not an acronym. which is
- 33:41probably why I blanked on source, but it really
- 33:44makes sense to me. Like most of us prompt AI
- 33:47with a goal initially, but adding context and
- 33:51expectations is something that we often don't
- 33:54and AI will over deliver at a frustration level
- 33:58because it doesn't know what you want, right?
- 34:00If you're prompting vaguely, it's like just throwing
- 34:02things back at you. And so I do tend to just
- 34:05kind of stop And I close my eyes a lot of times,
- 34:10honestly, and I think, okay, if I was telling
- 34:12a human this direction, like someone who worked
- 34:16for me, let's pretend I have a real assistant,
- 34:18a human assistant, what would I say? And often
- 34:23I find it's a whole lot more details than I was
- 34:25gonna tell AI, and that feels backwards. Like
- 34:28why would I tell AI less? They don't know less,
- 34:31right? And so I feel like I have to, at some
- 34:35point, remember that AI is not, it doesn't know
- 34:41everything I think it's going to know. It can't
- 34:43read my mind. I have to give it more details
- 34:45than I think. And when I do that, I get much
- 34:49better results. And so one tip I'll give that
- 34:52I really appreciate as well that my colleague
- 34:55Nathan Wells says a lot is that he'll at the
- 34:58end of his prompt say, and is there ask me any
- 35:00questions you need, you know, to know about this
- 35:04before you answer me or Is there anything I should
- 35:07have added to this prompt? And the AI will tell
- 35:09you. Claude is actually my favorite model for
- 35:14that method because I think it does a great job
- 35:16of asking questions and prompting the user on
- 35:20what else it needs to do a good job answering
- 35:22your question. It will do that even without you
- 35:25prompting it. It'll just be like, actually, before
- 35:27I answer you, can you help me with these questions?
- 35:30And I love that because it does feel more like
- 35:32a thought partner that way. So I think the biggest
- 35:36advice I can give on prompting is to offer more,
- 35:40not less. And you know, I think along those lines,
- 35:43I think teachers kind of naturally are good explainers
- 35:47in providing background and context to things
- 35:49because we do that every day when we introduce
- 35:52an assignment or an activity or a lesson, we
- 35:54kind of have to fit it within the frame of connecting
- 35:57it to the students prior knowledge and let them
- 35:59know what we're doing so that we know where we're
- 36:02going. And I find keeping that in my head, too,
- 36:06as I'm starting to prompt really helps get better
- 36:09results. But I really am going to take your suggestion
- 36:12of asking the question, what else am I missing
- 36:14that should be included? Because I often overlook
- 36:17that. And because it's a thought partner and
- 36:21highly creative, AI will come up with things
- 36:24that we wouldn't have. And so when I'm designing
- 36:29new workshops or new strategies, I'll sometimes
- 36:32take, and I'm not exaggerating here, an hour.
- 36:36of dialoguing with AI like I'm thinking it's
- 36:40asking me questions and I'm like oh okay yeah
- 36:42how would I approach that what do I believe about
- 36:44that um and we're having this conversation back
- 36:48and forth and at the end of that hour I have
- 36:52really refined my ideas it's incredibly helpful.
- 36:57Have you have you had any conversations with
- 36:59teachers about like um you know I I think with
- 37:02our roles I know that Jeremy and I particularly
- 37:05spend like compared to teachers or we spend a
- 37:07bunch of time in front of our computer with these
- 37:10systems so like like chat gbt or maybe if it's
- 37:13like magic school and you know we we have time
- 37:16to kind of really dig in and do tutorials on
- 37:18them and things like that have you had any conversations
- 37:20with teachers that kind of streamline their process
- 37:24a little bit because as we know like when they're
- 37:26getting back into like back to school week it's
- 37:29crazy They've got all sorts of trainings that
- 37:33are required of them. And so to sink as much
- 37:36time as we do into AI, it's really impossible
- 37:40for them. Have you had any good conversations
- 37:42with teachers about like, how do they start planning
- 37:45and organizing for their classes and provided
- 37:50the time constraints that they have? Yeah, the
- 37:54one advice that I often give when they say like,
- 37:57I just don't feel like I have time to learn it.
- 37:59As I say, don't worry about learning it as a
- 38:04goal. We would say the same thing about Minecraft
- 38:08or Canva. I'm not just going to sit down and
- 38:10do a bunch of tutorials. I always say, find a
- 38:14purpose. When you have a problem to solve is
- 38:18my big recommendation. Let's say that I had this
- 38:24conversation with educators just late last week
- 38:27where I said, What's something that you wish
- 38:31was different about your classroom or like you're
- 38:34trying to come up with a poster or a better way
- 38:36to explain something, like what's a problem that
- 38:38you want solved? And then can you ask AI for
- 38:41help with that? And I tell them, be very specific,
- 38:45like give lots of details on what you want. You
- 38:48know, do that good prompting, but they don't
- 38:50have to like memorize some prompt framework to
- 38:52just give it lots of, you know, lots of description.
- 38:56And so giving those use cases as far as having
- 39:00a problem to solve, I think is really helpful.
- 39:02When I see teachers get frustrated, it's when
- 39:04they feel like they're supposed to just go use
- 39:07it and they don't even know what for. You know,
- 39:10they're like, OK, well, I don't need it to write
- 39:12an email for me. Like I'm good. Thanks. I can
- 39:15answer my email. So what am I using it for? And
- 39:19focusing on that, what is a problem you want
- 39:21help solving is really helpful. Yeah, now let's
- 39:28kind of get into the book here a little bit because
- 39:30I feel we've had great conversations, but I also
- 39:32want to bring up some of the great points in
- 39:37the content that's in the book. You know of course
- 39:39you've mentioned and discussed how it's based
- 39:42on the SAMR model or you apply that model to
- 39:45generative AI use within education, but you know
- 39:49what are some of your kind of things that stick
- 39:51out in your mind about you know picking activity
- 39:54if you wish or a topic and maybe kind of give
- 39:58us an idea of how a teacher might approach it
- 40:00from the different levels within that SAMR model.
- 40:04Oh, goodness. Is that too vague? I can be more
- 40:07specific if you want. Yeah, that's super. There's
- 40:11like a hundred -year -old strategy in the book.
- 40:13A teacher would like, you know, a special ed
- 40:16teacher is, you know, needing some assistance,
- 40:19maybe scaffolding, you know, some directions
- 40:23within their lessons and stuff like that. Like,
- 40:26how could they do that using AI on some of those
- 40:29different levels? Well, OK, that's a little narrower.
- 40:34So what's interesting about what you just said
- 40:36is that we start at the substitution level without
- 40:40even using generative AI. And this is a good
- 40:44time to call that out, is that AI is this big
- 40:48umbrella. And I'm optimistic about all of it.
- 40:52Generative AI is a subset. of artificial intelligence
- 40:55that we've been laser focused on for a long time
- 40:57now because of all the controversy and all the
- 41:00possibilities. So for a special education teacher
- 41:04that's looking to scaffold or support would be
- 41:06the category of the AI optimism framework. We
- 41:10start with just substitution, like can we be
- 41:13giving audio directions? Can we be giving graphic
- 41:16directions? Can we be, do we need to allow for
- 41:19translated directions? for an assignment or for
- 41:22a task, and all of that is done with what we
- 41:25would call more traditional AI resources, right?
- 41:30Dictation, translation, read aloud, that's all
- 41:34just kind of normal, normal AI that's been around
- 41:36for a really long time. But then we could get
- 41:39into visualizing those directions. We now have
- 41:42tools like Notebook LM and Co -Pilot Notebooks
- 41:45that will generate a word online. generate that
- 41:49audio overview kind of podcast style of a set
- 41:52of directions or content information. Google
- 41:57LM just notebook LM for Google just announced
- 41:59video as a possibility. And so giving kids that
- 42:03multimodal input for looking to just kind of
- 42:06raise some understanding and going to a more
- 42:10transformative level. is where we can have students
- 42:13actually engaging. So speaking of personalized
- 42:16learning, scaffolding the directions to an assignment
- 42:19in a way where they also are asking questions
- 42:22and not just absorbing information. And I think
- 42:25that's a big shift for me in the framework. As
- 42:29far as the S and A levels of SAMR being very
- 42:32focused on pushing kids content, right? Giving
- 42:35them things where the M and R levels are more
- 42:38about kids. being engaged in a part of the generative
- 42:43AI, a part of the content. So maybe we have students
- 42:46copy and paste the directions for a task into
- 42:51napkin .ai and that creates a whole little infographic
- 42:55for them on the steps involved that's very visual.
- 43:00And because they initiated that, they weren't
- 43:02just handed a graphic. They're then able to go
- 43:05in and interact and and even evaluate that graphic
- 43:09and be like, is this correct? Are these actually
- 43:12the steps? And then they can add to that graphic.
- 43:14They could add in, OK, this is what I'm going
- 43:16to do at this step in the process, which takes
- 43:19us into supporting executive function and self
- 43:21-regulation and all these great skills. At another
- 43:24level, we could be setting up a school AI sidekick
- 43:28space that gives students an opportunity to ask
- 43:31the chat bot questions. And that's a really safe
- 43:36place for many kids. I know the word safe can
- 43:38be weird, but the reality is it's low risk, maybe
- 43:44is a better term. It's low risk to ask a chat
- 43:47bot questions about how to complete an assignment.
- 43:50Way lower risk than asking a teacher and for
- 43:53sure lower risk than asking a classmate and feeling
- 43:56like you're stupid. which is a whole other set
- 43:59of SEL we could talk about. So giving kids access
- 44:02to those tools as supports that they get to then
- 44:06choose when they access and when they use them
- 44:09based on their own needs is incredibly exciting.
- 44:12And it gives them that AI literacy of making
- 44:15a decision of when to use AI appropriately. So
- 44:18we're doing way more. than just scaffolding an
- 44:21assignment now, we're also empowering them to
- 44:23say, when does AI limit my learning? And when
- 44:25does it amplify my thinking? And that's, I think,
- 44:30where things get really exciting. And could we
- 44:33dive into that a little bit too? There's a ton
- 44:37of stuff with regard to students where a lot
- 44:41of conversation's happening around there are
- 44:44massive benefits to it. There's also some concerns
- 44:48about the cognitive offloading side where we're
- 44:54having the machines basically perform the task.
- 44:59And then there's conversations around what are
- 45:01the best ways to go about using these. And I
- 45:03always say, too, in our trainings, it's not just
- 45:06students. It's us as professionals, too. There's
- 45:08that danger of offloading your own thinking into
- 45:12these so that you're not really going through
- 45:15the creative process or the struggle and things
- 45:17like that. Have you had conversations around
- 45:22that piece of cognitive offloading that is a
- 45:26danger? And then also with that, do you have
- 45:31any concerns about the social aspect too? A lot
- 45:36of articles lately have been saying things about
- 45:38students having relationships with chat bots
- 45:41and you know the fear of that replacing the social
- 45:46aspect or like emotional growth. So basically
- 45:49the kind of the two things I'm wondering your
- 45:51thoughts on are the dangers of the intelligence
- 45:55piece, the cognitive offloading, and then the
- 45:57emotional piece of, I guess you could say social
- 46:00offloading, like you're not engaging with real
- 46:04humans, you know, face to face, or even like
- 46:08in this version where we're talking virtually,
- 46:12obviously. Do you have any concerns there with
- 46:13those two things? A hundred percent. Any, and
- 46:18I have lots of TikTok walks where I talk about
- 46:21outsourcing. and outsourcing humanity and outsourcing
- 46:25our creativity, like those are things that are
- 46:28kind of scary. Now, I will say the MIT study
- 46:31with chat GPT, I feel like that was headlined
- 46:34to be clickbait. It was a really unfortunate
- 46:39framing of that entire study, in my opinion,
- 46:42because they didn't actually, they didn't study
- 46:47people who used AI. appropriately. Like the study
- 46:52shows yeah kids generated essays with AI and
- 46:57then copied and pasted it into a document and
- 47:00hit submit and didn't know what they said. Like
- 47:02oh yeah okay like who is funding these studies?
- 47:05Sometimes I want to be like I would like a study
- 47:07with a no -brainer ending. So yeah obviously
- 47:12that didn't require any cognitive lift. That's
- 47:16not how we should be using AI. So It's really
- 47:20interesting as we think about my central question,
- 47:24does this use of AI limit my learning? That I
- 47:27think does help students evaluate, what am I
- 47:30supposed to actually be doing here? And this
- 47:33gets us into like a whole trajectory rabbit trail
- 47:36conversation on what are we actually assessing?
- 47:40Because I'll give you a really simplistic example.
- 47:44But when my son was a first grader, He came home
- 47:47and told me about a timed math assessment where
- 47:50he had to write numbers into a grid, write like
- 47:54a hundred grid. And then there was a backside
- 47:57and he was telling me how he didn't get to finish
- 47:59because the timer went off. But he told me, he's
- 48:03like, but if I would have finished, I would have
- 48:05gotten to 245. And I was like, well, how do you
- 48:08know that you didn't finish? And he goes, oh,
- 48:10because The timer went off, we had to put our
- 48:12pencils down, but then I just counted the rows
- 48:14remaining, and I know I would have gotten to
- 48:15245. So as a first grader, he knew the math.
- 48:21And he goes, but I didn't get to show that I
- 48:23knew that because the time ran out. And I said,
- 48:25oh. So it wasn't assessing your ability to count.
- 48:29It was assessing your ability to write quickly.
- 48:34That's what she's scoring. She's scoring your
- 48:36ability to write quickly. And not to be critical
- 48:40of his first grade teacher who was wonderful,
- 48:42but I think a lot of times in education we forget
- 48:44what we're actually assessing. And sometimes
- 48:47we give these assignments and we want to assess.
- 48:50Like the student writing and we forget that we're
- 48:54also assessing. along the way, a lot of other
- 48:59things that we maybe don't need to be. Like maybe
- 49:01we're assessing student creativity or we're assessing
- 49:04their spelling or we're assessing their ability
- 49:06to stay on task for significant periods of time,
- 49:09or we're assessing their ability to take a very
- 49:11large project and break it into chunks and then
- 49:13self -manage all the normal steps that go into
- 49:15a project. Like that's what we're actually assessing.
- 49:18And so removing some of those barriers for kids.
- 49:22does allow us to get to what we actually want
- 49:25to assess, which is can they produce high quality
- 49:30work in many cases? And if we don't want them
- 49:34to have any supports to produce high quality
- 49:36work, then that is now part of the assessment,
- 49:39is can you do this 100 % independently? Or did
- 49:41you get help? And who did you get help from?
- 49:44Or what did you get help from? Because we've
- 49:47had these support mechanisms in place for a long
- 49:49time. The difference with generative AI that
- 49:52I hear about a lot is that it over helps, right?
- 49:56It goes beyond the line that a human would or
- 49:59a tutor or someone at home or some other tools
- 50:03that we've had in the past. And so I believe
- 50:06we have to allow students to help to understand
- 50:09that line because I don't think we're helping
- 50:12them for their future. If all we do is tell them
- 50:15like, no, you're not allowed to use AI on this.
- 50:17or yes you are, and we don't help them see why.
- 50:22Like, what am I assessing and what am I trying
- 50:23to do? Does that make sense? Yeah, absolutely.
- 50:27It's just like a much bigger conversation than
- 50:29like... Yeah, and I'm sure all the listeners
- 50:31aren't going to see me keep nodding my head on
- 50:34the screen, but like through all of that part
- 50:37that you just talked about, that's exactly what
- 50:39I was doing. I know we're coming up here to the
- 50:42end of our time that we have for today, but so
- 50:45can we just kind of go back to, you know, obviously
- 50:48I know that the title didn't come till the end,
- 50:51but still, you know, why should educators be
- 50:55AI optimistic? Because most educators, I'm just
- 51:01going to be a blanket statement here. Most educators
- 51:04went into education because they believe they
- 51:06could make a difference. And they love that feeling
- 51:09of like imparting knowledge or helping others,
- 51:12helping students. If we look at personality assessments
- 51:15of educators, because we're big on that as a
- 51:18company at I2E. And anyway, they're like hands
- 51:22down relators. They're connectors. They're people
- 51:26who want to invest in other humans. And so I
- 51:30think our only option is to be excited about
- 51:35the possibilities of where we can let this go
- 51:38to support our learners. And ultimately AI Optimism
- 51:43is about acknowledging the challenges, but focusing
- 51:48on the potential. And that's an important caveat
- 51:52where we're gonna nod our heads like you were.
- 51:56Like yes, there are some issues here, but I'm
- 51:58gonna choose to look at what's possible for the
- 52:02betterment of my students. and what this means
- 52:05to support them in a world that will inevitably
- 52:07require them to use AI. And that's what AI Optimism
- 52:11is all about. Great. Well, we really appreciate
- 52:15you joining us for the podcast. And we would
- 52:20really encourage everybody to remain AI optimistic.
- 52:24Please go get the book from Amazon or wherever
- 52:28you can find the book. It's a good read. You'll
- 52:32learn a lot of great things from it. And check
- 52:34our show notes. We'll have links to the book
- 52:37in the show notes. Becky, how can people get
- 52:41a hold of you if they'd like to? The socials,
- 52:44are there any upcoming events and things that
- 52:46you want to plug to let folks know? Well, a great
- 52:50way to get in touch with me is my website, beckykean
- 52:53.com. I'm the only Becky Keene, like not in the
- 52:57world, but if you just like Google Becky Keene,
- 53:00I will come up or ask Chad GPT, how do I get
- 53:03in touch with Becky? I will come up. So that's
- 53:06easy. And that links to all my social sites.
- 53:09People can follow me there. It's also links to
- 53:11both books. And there's book resources as well,
- 53:14like a book study guide for people who are looking
- 53:16to do it as a book club group. I do virtual author
- 53:19talks for those groups. So lots of resources
- 53:21there. Becky Keene dot com. Um, and then coming
- 53:25up, let's see. I mean, I don't know when this
- 53:27is going out, but I, I, there's a couple of,
- 53:32uh, conferences that I'll be at as featured speaker
- 53:35in the coming school year. And I'm not able to
- 53:39announce that quite yet. Um, because I need to
- 53:43wait for them to announce me first. Um, but I'll
- 53:46be around, uh, yeah, especially during conference
- 53:51season. So that'll be, that's exciting. Okay,
- 53:54well, if you can, we'll for sure. Keep an eye
- 53:55on my socials. Yeah, check the socials. We'll,
- 53:58for sure, if we can, we'll put them in the show
- 54:00notes if we, if we're allowed to, but. Perfect,
- 54:03yeah. If not, after the fact. Once it's out,
- 54:05I can share more, yeah, but it'll be fun to see
- 54:08educators around the U .S. in the coming year.
- 54:11Good. Well, thanks again. We appreciate it. Thanks
- 54:14so much for having me. Yeah, I appreciate it.
- 54:17All right, bye guys. Have a great day. Thanks
- 54:20for listening to Integrate This. Want to take
- 54:23your learning further? Head over to our YouTube
- 54:25channel for quick tutorials, deep dives, and
- 54:29more ed tech tips you can use right away. Don't
- 54:32forget to subscribe and catch new podcast episodes
- 54:35every Thursday on Apple, Spotify, or wherever
- 54:39you get your podcasts. Until next time, keep
- 54:42exploring, keep innovating, and keep integrating
- 54:45this.