Latest / Brains & Bots / AI Tutors and Student Privacy: Who Protects Our Kids?
Transcript
- speaker-0: you you speaker-1: Good morning and welcome to Brains and Vots, a podcast brought to you by the National Association of Scholars. I'm Lila Nora Kish, and I'm your host today. We are going to talk about privacy, child protection, and artificial intelligence. And our guest today is Mr. Paul Wayne from Jersey, the country, not from New Jersey, the state. And Paul is Jersey's information commissioner. He joined the office as Deputy Commissioner in two thousand and four and became Commissioner in twenty twenty one July. His earlier career involved policing and financial regulation. He also serves on the Global Privacy Assembly Executive Committee. Paul, welcome to the show. speaker-0: Excellent, very good to see you after all these years. speaker-1: Great. So I have some questions for you. You are Jersey's the country's information commissioner. And I would like to ask you what do you do? since our topic for today is AI privacy and chat protection, which set to be the priorities for your office for this couple of years term. I would like to ask you what do you do? speaker-0: So I guess my role is to lead the Jersey Office of the Information Commissioner and we regulate data protection law here in Jersey as well as Freedom of Information law as well. So we've got two hats if you like. But I guess in practice what that means is that we handle complaints from Islanders, we receive breach reports from local businesses, we investigate complaints and wrongdoings under data protection law. We've got a very broad outreach section as well. we advise organisations on how to comply with the law. And then of course we take enforcement action where we need to, where it's necessary and proportionate. we have a like the UK, we have fining powers and our sanctions run from reprimands and binding orders that we can issue right through to public statements. And for private sector bodies, we can issue administrative fines up to £10 million. So that's in a nutshell what we do. You mentioned there in terms of AI. Yep, that's one of our three strategic priorities that we have for the next three years because we believe that's where most of the risk facing Islanders sits at the moment, particularly AI. we're, you that's a massive subject in its own right. So we have to focus down into one element. So we've decided to look at AI in terms of human resources and automated decision making. Again, not dissimilar from what the UK Information Commission are doing now. Then we also have cyber security, but again, a big subject. So we focus that down to breach reporting and the quality of those breach reports and trying to of improve that area. And then we have children's privacy, which will kick off probably early part of next year. And that's in terms of the use of education technologies, edtech and how schools are using those and delivering on them. And then we have a sort of with that, a new approach as well. we essentially advise, then we assess, then we act. So advising is about sort of setting those expectations of what we want to see from, from organizations and with them again, and give guidance and direction as to how to achieve that. Then the assessment part is where we go out and run sweeps and surveys and audits to help build our evidence base about what's going on. And then act is where we enforce. So that's like it range from words and advice and from helping organizations on their compliance journey to get to where they need to be, particularly if they're new and they've not crossed our paths before. but also to the more serious enforcement action against organisations who are demonstrating repeated behaviours or deliberate neglect of the law. So that's kind of where we are. We launched that strategy earlier this year and we know that as a small authority we can't do that on our own. So we have to rely on other stakeholders. So we work with the NSPCC Jersey branch in terms of children's data and the Children's Commissioner as well. And we also work with our Jersey Cyber Security Center and Digital Jersey, who sort of promotes entrepreneurs and new digital businesses to come to Jersey. So we don't work alone. We need all those collaborators so that we can achieve our strategic outcomes. speaker-1: That's great to know. And thank you for the you know, overview of what you do and what kind of priorities and cases you deal with. I think it's as you mentioned, you it's pretty broad. but I would like to focus on that three areas you have as a strategic priorities and you know why is it important to talk about Jersey and just to give a little bit of context to our audience. the tax justice network ranks Jersey as the number eight in the the December twenty twenty five corporate tax haven index. I know that you guys like the Financi International Financial Center expression more than the tax haven one and and I understand it. you also warn about you know economic inter importance of retaining European data adequacy and complying with with broader European rules on on data protection and so. So I these are of course different issues, but together they they bring the public trust when it comes to financial issues and data protection. and I would like to ask you how do you distinguish different legitimate privacy c concerns from financial secrecy? And can you you know explain for example what an international financial centre can regulate in both relate related to government and private businesses. speaker-0: Okay. Well, I'm not a tax regulator, so I can't really comment on the tax justice networks ranking of us. But that distinction you're drawing is correct and the law drafts it quite clearly. We don't like being referred to as a tax haven because in our eyes we're not. The taxes here are low, but that's deliberate. And yeah, like I said, I'm not a tax lawyer or regulator or expert. I won't comment any further on that before I stray out of my territory. But I think in terms of the data protection, that's a right held by individuals. A company themselves doesn't have any data protection rights and a trust structure as it's established in trust law here has no rights to privacy under the data protection law that I regulate. But I think also the law also contains express exemptions for crime prevention, for tax collection and regulatory functions, those kinds of things. It doesn't stop lawful disclosure to tax authorities and law enforcement of financial services supervisors where transparency and privacy genuinely meet. In my view, as with things like beneficial ownership registers and that kind of thing. It's on the question of proportionality, who needs access to what, for what purpose, what are the safeguards that are there. And that's a balancing exercise. It's not a veto of those services, which I think is quite an important distinction to point out. On the adequacy part, our adequacy status, which was first granted in 2008, I seem to recall. and reconfirmed only in 2024 by the European Commission. That matters economically to us precisely because it rests on that independent supervision. I think if my office were seen to go easy on the finance industry or on government for that matter, then that would put that status at risk because our independence would be questioned and that's not something that we want to ever... and experience. independence is not a courtesy to me, it's a critical component of the island's economic infrastructure and without it then that could cause problems for us. speaker-1: Thank you for explaining these relationships. I would like to navigate us to AI issues and the office's priorities related to that. in February your office joined an international statement supported by sixty-one privacy authorities and it asks for safeguards, transparency, and effective ways to remove harmful imaginary imagery. I want you understand what happens after this statement when someone is targeted, for example, by AI generated images, what can your office do? speaker-0: That's a great question and I'm a huge advocate for what happens next after these kind of statements are made. for those in the audience that don't know, in February, that statement signed in February was signed by 61 authorities throughout the global privacy assemblies, international enforcement working group. And that kind of sets out what we expect AI image providers. The safeguards against things like non-consensual intimate imagery, real transparency about what the tools can do, fast, effective removal routes that they need to have, extra protection for children as well. And that statement I would say is a marker, it's a bit of a line in the sand. Enforcement is what comes next and it comes jointly because a bit like I said with our strategic priorities, know, no single small regulator can really take on a global platform on their own. That would just be asking for trouble in my eyes. So I think what we can do is where an image of an identifiable person is personal data and it's captured by the law, where an organization creates or hosts or processes it, a victim can ask for erasure and complain and that's a legitimate right they have under the law and that's so they can complain to us if that right is refused to them. And then we can investigate and if necessary we can order that processing to stop, we can issue a reprimand and maybe fine as well if it's necessary and it's proportionate. And our law also reaches overseas organisations because we have extra territorial effects. So if an organisation is is deliberately targeting Jersey citizens or offering services to those citizens, then we can take action there as well. But we would most likely work in coordination with our fellow regulators or cross-border cases anyway. And then in terms of the powers to stop, I think an individual acting in a purely personal capacity, that falls largely outside protection law. It's really only organizations who are data controllers as defined under the law. creating or sharing indecent or abusive images, that's largely a crime. So our states of Jersey police would look after those. And we're not a content regulator either. That is something that we're working on. Jersey is currently consulting on an online safety provision, which will be an amendment to the data protection law as and it happens. And that's because a local scrutiny panel hearing in the state's assembly found that local safeguards here were weaker than in the UK and Europe. So we really welcome that gap being filled sometime quite soon. speaker-1: Wow. can we see a hypothetical case where, for example, a pupil's image is taken and manipulated and posted online? who would or who should the the family contact first? what can your office do? you know, you mentioned some of this in the previous answer. And can you give a couple of examples? for example if if there's anything, of course, without mentioning any names. speaker-0: Yeah, and that's difficult in an island the size of Jersey. I think the order matters. So if the image is sexual or indecent, then go to the police first. That will be your first border call. Don't forward it, don't download it. Even as evidence, you've got to know where it is and where you saw it and then go in and report it. Secondly, I'd say tell the school's designated safeguarding lead so they can put some measures in place to protect. the child are managing what's happening among pupils. And then third, report it to the platform using its own tools. So if it's meta, use the meta tools that are available there. And then I think, you know, then it's about support really and making sure that those children are safeguarded so young people can get support from things like Childline here in the UK, which is also available here in Jersey. through the NSPCC that I mentioned before. So there's lots of support mechanisms that you can follow. Then where my office gets involved, if a platform refuses an erasure request, or if a school mishandles the child's information when they're responding, the family can then complain to us. And that's what we would certainly advise. And we can also look at whether an organization's systems made abuse easier, for example, and that might, something like a school app that exposed people's photos, you know, we can have a look at those kinds of systems. On examples, yeah, I can't really discuss individual complaints because like I said, Jersey is very small and it's very easy to identify and unpick those kinds of examples. I mean, there's plenty that people can Google if they really want to, but I think the public example this year that I can talk about was a police investigation into an account which posted deep fake content targeting school staff, so teachers. And that was in part what prompted this Crown Dependency Advisory with Guernsey and the Island Man. That was really one of the things that gave rise to that piece of work. speaker-1: That's very interesting. Thank you so much for sharing it. I would like to bring in an American comparison, the Take It Down Act that was supported by nothing no one else than Melania Trump. and this act forces covert platforms to remove some image images within forty eight hours after a valid request for the removal. that is a very targeted rule. should AI related privacy enforcement focus on the harmful conduct or you know, the technology or something else? What how do you see this? speaker-0: I think the answer to question is both, without any doubt, but using different tools. That's the key. think conduct rules like the US Take It Down Act, they're valuable because they give victims a clear and fast remedy. 48 hours there is a meaningful standard, but conduct rules tend to act after the harm has happened. Data protection on the other hand, it adds that upstream duty. Firstly, you've got to build a tool that can produce, or if you build a tool that can produce intimate images of real people, you must assess that risk before launching that tool and making sure all the safeguards are by design built into that platform. So that's data protection by design in default really. And that's about regulating design choices, not banning the technology because the technology is not going away. have to use it, we have to adopt it and adapt to it. But that means having to think a bit more proactively about the outcome. Obviously, I can't comment on US politics because I know very little about it because I don't live there. But the standard I'd apply to any public figure is quite simple. Don't use synthetic media to put words or actions into a real person's mouth without their consent. Simple as that. And if you are going to do that, then label anything that's been materially altered. And you start to see now content on social media with a little sort banner in the corner saying created by AI. That's good. At least people can see what's true and what's not. know, used to be a saying back in the old days, Certainly when I was growing up, the camera never lies. Well, it does now. So you can't necessarily rely on what you're seeing on the screen. Your viewers might look at us and think, they're deep fakes or not. How can we be sure? So you have to be cognizant of what your audience is saying. And I think leaders really set the norms for everyone else to follow in that respect. speaker-1: Yeah, that's a great point. And the c related to the camera doesn't lie. And now we see all those AI generated content and some of them are pretty obvious, but sometimes it's just impossible to tell that it was manipulated by AI. And and and you know, we we are going to have elections in the US soon, I mean midterms, and I think it's particularly important from a voter safety perspective. there are a bunch of political ads and so and I know that it's not really a privacy issue but but still it's related to AI generated and manipulated content. So when it comes to political advertisement, how do you see, you know, the AI disclaimer notices like w what should what should authorities request from content creators? speaker-0: That's tricky. Which figure does that 69 % refer to? I've not seen a report and I'm not an electoral regulator, so I can only really speak on the privacy side. But I think in any case, a vows should be told quite plainly when an advert depicts a real person, a real place, a real event or... that has been generated or materially altered by AI. I don't think necessarily that small icon in the corner is enough. Voters also need to know why they are seeing the ad. They also need to know why, if their personal data was used to target them, then they're entitled to know about that and entitled to object to it as well. The principle is the one we apply everywhere. People can't exercise good judgement or their rights over processing if they don't know that it's happening in the first place. So think that's really important. speaker-1: Yeah, yeah, that's a great point. I would like to navigate us to cybersecurity and international cooperation related to cybersecurity. the ITB reported that the beacon incident affected a jersey charities. for small organizations the the service provider may be overseas, even though the people whose data involved live in Jersey. when a breach begins outside Jersey. What responsibility stays with local organisations and what affects and what affected people can expect from from regulators? speaker-0: Yeah, good question. I think the beacon example is a good illustration. There's a company which is a UK provider of donor management software, which is used by thousands of charities across the UK. this came out to August, I think it was, it disclosed that a compromised access key had been used to copy customer database backups, which were likely downloaded further down the line. We established that eight Jersey charities had reported to our office, but because Beacon falls itself under the banner of the UK Information Commission, we have to work with the ICO and our Guernsey colleagues as well to decide what happens next. But the responsibility that stays in Jersey is an important one. The local charity is the data controller. Outsourcing the system to a UK provider doesn't outsource that accountability that they're required to comply with. So they've got to assess the risk to their supporters. They've got to report it to us within 72 hours where required. They've got to tell the people affected when the risk to them is high and review whether it shows it's contracted. supply properly. Affected people can expect us to check that their local organization has met those duties and to share intelligence with the regulator that holds the supplier and to take a complaint if they're not satisfied with how they were treated in the first place. speaker-1: Thank you. You also responded my to my next question. So that would have been what what can charities and schools and organizations expect and should ask before they trust the provider. my my next question is related to your maturity survey, that says that sixty seven percent of respondents had never completed a data protection impact assessment or similar assessment. I think in the US it would be even less. because people just don't know much about their own privacy protection network and safeguards. what does that tell you about the gap between knowing the rules and applying them? speaker-0: Interesting. mean, I was quite staggered when I saw the results of that. have to be honest because that's quite a big number under the European model and given how long GDPR has been in play and our laws are broadly equivalent to GDPR. I was quite surprised how many said they'd never done one. And I do recognize that. I think that there are an extraordinary number of small and micro businesses in Jersey and doing a DPIA is not always proportionate for the type of services that those organizations are providing. So I wouldn't necessarily expect them. But given who responded, I was surprised and I think Like I said, maturity is closely tied to size and that tells me that most organizations know the principles, but it's that risk assessment bit that's missing. That's the step that turns principles into decisions. It's seen as paperwork for bigger organizations a lot of the time. But that exactly matters most for the priorities that I I mentioned earlier on, a school that's adopting an AI tutor, for example, or a charity moving debt donors onto a cloud platform is a textbook case for doing a DPIA. So I'd say our response is twofold. We'll make it easier through things like short checklists, templates that we can help with, which are aimed at small organizations. And we've got, we're just revising our toolkit at the moment. for small businesses, which will help them with that. Then through our sweeps and our audits, we'll check whether those assessments are actually being done. And then if they're not, then let's understand why they're not being done. Is it because they don't reach the required threshold or is it because they've completely forgotten or they just haven't bothered because they don't think it's important? You know, there's some big distinctions there, so you've got to look at it. of case by case, but you know, learning and knowing the rules was the first eight years of data protection law in Jersey since 2018. Applying them and then properly and consistently should be the focus for the next eight years, I would say, because we've done the hard work. Now it should becoming common practice and should be part of your everyday regime when you're thinking about, we've got a new product, a new service here, and it's going to involve personal data. How do we protect that? And that should be the question right at the very start, not waiting till the 11th hour and all of the regulator comes knocking and saying, you haven't done your DPIA, how have you managed the risk? And they're like, well, what risk? Come on guys, you know, it's not rocket science. speaker-1: Yeah, it's not. But I remember I was back in Hungary in the t twenty fifteen, twenty eighteen time frame where the GDPR had to be you know, complied with. And I remember that right before the deadline, about two months before, companies started to come to us. I I worked for a law firm and they were like, Could you draft a data protection rule for us. I don't care what's in it, just, make it make the GDPR thread go away. I don't want to g get a fine from the local data protection authority that I'm not compliant. It's like, okay, could you tell us what kind of data you are handling? And they were like, I don't know. it was so weird that they just wanted to have a piece of paper, put it on the website and let the problem go away. And that really shows this n h huge knowledge gap when it comes to data issues. speaker-0: Definitely is. But like I said earlier on, I'm also very understanding that small businesses and micro businesses find it really difficult. Actually, it's the opposite. Because you're small and because you're micro, it's actually really simple because the safeguards you need are going to be minimal because you're not a multinational corporation. Depending on the type of data that you're holding, you don't need gold-plated data protection compliance regimes. You just need the basics. And that could be a folder of policies and procedures which are thinner than the last report I wrote. So it doesn't need to be this huge thing. It can be something much more proportionate and pragmatic and practical. That's the key. practical for those organisations that are only say one or two people. speaker-1: Yeah. But as you mentioned, depending on the type of data they are managing, the these rules can be very different, what is applicable to them. Like a dentistry is g going to treat sensitive health data while a bakery is not. So speaker-0: Exactly. So you've got to look at the context. Context is key, always. speaker-1: Yeah. I would like to ask you also about children and data protection. there's a difficult issue and I know that we can't go to too much detail, but I know that your crown dependency advisory addresses AI generated indecent images involving children. how should the law address AI generated child sexual abuse material, including the material for which no child was Found. speaker-0: I think the law should treat it exactly as what it is. It's child sexual abuse material. The test should be what the image depicts, not how it was made necessarily. So our advisory, I think it states that quite plainly really, that creating or sharing indecent images of a child, that's a criminal offence, whether or not AI is used. That's kind of secondary actually. The criminal law. is there and covers exactly that. And that's for the courts and for the police, not for us as data protection regulators. And like I said earlier, that's the route that an individual should take if we're talking about that kind of imagery. I think where data protection perhaps adds something again is upstream. Companies that build image generators should be able to show that they've assessed and blocked this kind of misuse and that their training data that they train their systems on doesn't contain abuse material or children's images that have been scraped without some kind of lawful basis underpinning it. And I think that's really important that they act on those reports. It's a design and accountability question more than anything and that sits squarely within the remit of of us as privacy regulators acting together to achieve that. speaker-1: Yeah, and I I usually see these quite mind blowing arguments in the US that, well, there were no children involved, so what's the problem? And the problem is the it it doesn't matter. The problem is the conduct that is, you know, producing these materials and using these, consuming these materials. And actually it's really crazy that some people try to use privacy as a defense in this in this case. Right? That's it's my privacy, you shouldn't be aware of that. And there's a lot of debate around this in the in the United States that you know, how should regulators answer when there's no harm, because there's no children involved. There's still harm. And this is what is kind of intangible. speaker-0: Yeah, I know, I'd 100 % agree with that. I don't know, I'm an ex-policeman. So, you know, came a long, long time ago. But it never ceases to amaze me this, why are we even having the conversation? It's either illegal or it's not. The manner in which it's created is immaterial, really. And it's interesting because... You hear all the hype about AI and other new emerging technologies. For me, just another form of processing. It's another medium for processing personal data. How do we apply the principles of data protection to that? Making sure that the response is fair and reasonable and proportionate and the usual test that we would apply to any other type of processing. So that I think sits very nicely with that argument that, you know, it doesn't matter how the image was created. If it's indecent or abusive, that's illegal. Simple. speaker-1: Yeah. And the other thing is just labeling child sexual abuse material as child pornography, implying that it is just one form of adult entertainment, just it happens to be with children, is is very problematic because language do matter or does matter. And and in this case we we cannot forget or you know, ignore that this is a crime. So i yeah. speaker-0: Amazing speaker-1: i and it it doesn't make any difference that it was generated by AI and not recorded any other ways. thank you for making the distinguishing distinction here. I think it's it's very important. in the international privacy sweep, participants could bypass each assurance on you know, seventy two percent of the services reviewed. Often where users simply declare their own age, how can we check age to use different platforms and protect children without collecting even more sensitive data or government issued IDs by private platforms? So how could you do the age verification problem? speaker-0: yeah, really difficult, I would say. The answer isn't really ID for everything. It's about proportion. I think a low risk service should simply be designed to be safe for a child by default. As risk rises, then stronger assurances is justified. There are privacy protecting ways you can do that. Age estimation is done on a device, for example, and then discarded. or you might have a situation where a trusted provider that confirms over 18 to the website without passing on who you are. So I think the rules are set pretty much as the data protection law dictates, know, collect the minimum, you know, take account of the data minimization rules, keep nothing... for longer than it's needed. So if a check only takes 30 seconds, then it's held for 30 seconds and then it's gone. Never reuse that data for anything else, because that's again a purpose limitation. And audit your providers. What are the mechanisms that they're using? Are they storing that imagery after all that verification or whatever it might be? I think they use check. that creates a new database of children's faces or passport photos has really just only swapped one risk for another. The sweep finding itself is stark. Those checks are far more common, but there's still quite a lot of work to do in that field, I'll show you. speaker-1: Yeah, and in the US the child protection online is quite a bipartisan issue, which is really refreshing, especially in a very partisan political environment. But I see that the solutions are very different because one party wants to collect more data and do, you know, government issued IDs, verifications online. The other part doesn't want to hear about providing more power to private platforms to act as quasi state actors and check IDs and so and store more data. So it's it's a really great point what you mentioned about, you know, just changing the problem, exchanging the problem to another one. speaker-0: Yeah, that's it. I think that's where, know, it's much the same in Europe and the discussions I have even just here in Jersey when we're talking about age verification and we think, well, you don't you want, need to understand the platforms that are out there already and that some aren't really resolving the problem. They're just creating a new one. So, which you then got to manage. And then all of a sudden you've got two things you've got to manage rather than one and you're actually making life more complicated and more difficult for yourself. So it pays to do that groundwork and understand exactly what you're dealing with with the platforms that you're looking at. But even before that is to identify the need. What are you trying to achieve here? And how can that age verification be done in a way which is the least privacy impact to an individual. Whether that's children or adults, doesn't really matter. The same principles apply. speaker-1: Yeah, indeed. let us bring this to a classroom and a kitchen table. You mentioned the ad educational technology and AI tutors before, and just from a personal standpoint, I I think that it has a great potential to bring in AI tutors and provide ideally personalized education to children, especially if the parents are not affluent and they can't afford having personal tutors from every subject the child might need it. However, that also mean a lot of privacy concerns and data protection you know, potential threats. So the practical question what the parents need to know before their child start using an AI tutor. What should you know they check before choosing the tutor and the services or a classroom app. And especially when they are teaching at home. speaker-0: Yeah, great question. I totally agree by the way, think, you know, personalised education is, you know, to have that availability now and, you know, relatively cheaply as well, I think is a huge advantage. But like everything, it's got to be used properly. So advice to parents, I would say, perhaps think about spending 10 minutes before your child spends 10 hours. First, I would ask about the company. Where is it based? Can you contact a real person? Or is it hidden away in the back somewhere and they're avoiding having direct contact? That always flags a big warning sign for me. If I'm with any product or service, if I can't contact a real person, then I'm not going to touch it. Second, read the privacy notice. Privacy notices are a thing on their own really because we've all seen them and we've all had the experience of reams and reams of paragraphs of illegal jargon and gobbledygook that nobody understands. If it isn't written so a parent can understand it, then that in itself is an answer. Don't touch it because it's trying to bamboozle you with legal speak that actually means nothing. And then third, think, check whether the product itself was designed with children in mind with age-appropriate defaults or is it just an adult product with an age box bolted on. So I think that can lead to problems in itself because you're not going to have that privacy by design built into it in the same way if it's just an adult. And then I think fourthly, try it yourself first before you try it out on your child. Have a look at it yourself. Do your research. Look at all the things I've just pointed out. But there's no better way of gauging whether something's good or bad unless you've tried it yourself. It's a bit like introducing a child to food. And they say they don't like it when they've not even tried it. I've had the same with my son years ago. Try it. it might be perfect for you. If it's not or you don't like the taste of it, then move on to something else. I think for home educators, the stakes are the same as they are within the school. as you rightly say, if there's no procurement team or no data protection officers putting those guardrails in place. So that's something we're looking at actually with our EdTech guidance and our checklist that we're going to be doing next year. under that children's privacy priority because we want to make sure it's written in a way that a parent at a kitchen table can use them, not just a school. speaker-1: Yeah, it's very important to talk about these green and red flags when it comes to educational technology. And you mentioned excuse me. speaker-0: Avoid the beige flags. speaker-1: Yeah. what are the beige flags? speaker-0: They're the ones in the middle where you're not sure. speaker-1: Okay, so the gray zone. Yeah, that that makes sense. And and you mentioned a couple of things, like if you cannot find you know, a person behind the technology that's obviously a red flag. and of course by the time you try it you will notice if if it's something you could use, then you obviously can share it with your child. But if you feel it's not good, then probably you are not. But are there any other like very dark red flags that parents and educators should be aware of. speaker-0: I think you can look at some of the questions around general basic data protection questions. What kind of information are they collecting? Who can see it? How long has it kept? All of those kind of questions really. I've always seen it more about data protection is about empowering people to ask the right questions. I think that's really where we sit in this is So that parents and home educators can know what questions to ask. They can highlight and identify what those dark red flags are. It's just ask the right questions. And if you're not sure what questions to ask, then phone us and we'll tell you. speaker-1: And does your advice change for an AI tutor that can take actions across other applications rather than just, you know, simply answer a question? And what control should remain with the parents, the students and the teachers? speaker-0: Yeah, I think the short answer is yes. think any tool which answers questions is a risk to the data that you give it. So a tool that acts in a way such as opening email or submitting work or browsing on a child's behalf. It's agentic AI, isn't it? So it's a risk to everything it can reach. The more you give it, the more risk than there is. So I think the principle becomes least privileged, give it access only for what the task needs and only for as long as the task needs, no more than that. I think in terms of control, that should stay with the individual, with the person approving anything that sends, buys or shares or Deleting a clear log of what the tool did is really useful. Switching off its access in one step, not having to go through multiple layers to say no. I think it should be the adult that sets the boundaries ultimately and the student that does the learning. Neither should hand their judgment to the agent. It's for the parent and the student to decide. speaker-1: Would there be different privacy concerns for home schoolers? speaker-0: honestly, I don't know, certainly not yet in Jersey. so I'd rather say that than try and guess, you know, there's certainly no evidence of different privacy needs between home educators and school communities that I've seen here. That's why our plan starts by mapping out what ed tech use is there across schools at the moment and and that's a year one baseline we're trying to get. That will give us the evidence to know for schools and then we can look at the home education gap after that. And certainly I'd welcome working alongside any home educating families who've got first hand experience of that alongside Jersey's education department because I think that's a great opportunity for collaboration but you'll see the full picture more holistically I think. in that sense. So I don't know. I would say that the home educators, need to be empowered and armed with the right information so that they're, like I said before, they're hitting the same standard that a school would. But of course, they don't have the luxury of data protection offices. that's why my office is there to help. speaker-1: Yeah, that's a great point. And I know that homeschoolers and the number of homeschoolers are growing on both in Jersey and here in the United States. So it's particularly important to be aware of these potential concerns. speaker-0: Definitely. There is a structural difference, I guess, which is more clear. School is an organisation that's got legal duties that we can regulate directly. A parent choosing that for their child is generally acting in a personal capacity. So for home learning, that burden falls more heavily on the provider to be safe by design because there's no institution there in the middle to check it. So I think it's really important to separate the two and for a home educator or parent, get that advice and help that you need to make that right informed decision. Don't just try and think you've got to navigate this on your own when actually you don't. Even ask the school what they do and get their advice and input because they're the ones that have got the knowledge and experience of working with these platforms and they'll be able to be in best position to advise you. speaker-1: That's awesome. thank you for this clarification as well. couple of more questions on your institution and the institutional independence. How do you protect the regulators' own information and independence in this AI, cyber security threats combined age we live in? And are there experiences you can discuss publicly? speaker-0: How do we do it? Well, think independence is built into our statute and it's a distinct requirement of the EU adequacy assessment that we can take advantage of. Our own authority was set up by its own law in 2018. It's a body that's set up separately from government. And we have a board that oversees the work that I do. And so I'm not a standalone final arbiter. I am in terms of FOI, but in terms of data protection, no, just serve as the chief executive of the authority. And my board have some of the major decisions in terms of finding powers and issuing public statements, neither of which I can do autonomously. I have to defer to them for that. So I think that's, we manage that independence quite well. as an organisation and we're fiercely fighting for our independence at every level because it's so important to demonstrate us as being independent from government, being independent from business, not influenced adversely in our decision making. That's really important. We decide what cases we're going to investigate, what sanctions to impose. ministers can't direct us on individual matters. Our decisions are ultimately appealable to the courts, not to politicians. So that's quite important. I think the public evidence of that independence is our record. As mentioned earlier, we have issued formal findings against organizations, which includes government departments and against organizations like our financial services regulator. I think an office that only reprimands small private firms would not be independent in any meaningful sense if we just did that. our range of powers applies to everybody and we have to treat everybody fairly and equally as the law requires. speaker-1: Great. could you share how do you hold your own office like internally accountable and to the same standards you expect from anyone else? speaker-0: That's a fair question. Yeah, well, we're a controller, we're a data controller ourselves and we hold some of the most sensitive information in the island. We know we've got complaints about people and about organisations, we've got breach reports, we hold lots of evidence. So of course, we've got to apply our own law and the standards that we preach on everybody else, directly to ourselves, we have to be whiter than white. So we do the same as everybody else, we log our own incidents as and when they occur. Thankfully, there's not many. We assess risks before we adopt any kind of new systems. We train our staff extensively. They all hold data protection qualifications. I think that's important for our reputation and for making people understand that we are a reputable organization with trained staff. So you've got to have that. We have our financial accounts and our performance scrutinized externally. So we have an independent provider come in and an auditor every year and they go through everything with a fine tooth comb. And then we have to prepare an annual report which is laid before our state's assembly every year. So our 2025 one is up on our website for those that are interested and want to have a look at what we do. What protects our ability to challenge? think there's three things, statutory independence, a stable funding model that doesn't depend on keeping any one organization happy, and transparency, publishing out our reasons, our decisions, our decisions stand and fall on the evidence, on who the organization is. speaker-1: Yeah, that's that's very important. And I appreciate that you find important find it important to train your staff. And it's very reassuring to know that the people who are deciding they they really have the expertise and the most updated information on all these data protection and privacy issues. And sometimes I wish that the average people like just basic citizens would have more knowledge about these issues. we could avoid a lot of the problems. speaker-0: Well, I agree to an extent. That's why data protection authorities exist and data protection consultancy services as well. They are seen as the experts in the field. As a citizen outside of this field, I couldn't begin to understand the nuances and the practical application of the law in a day-to-day sense. And no training would ever give me that necessarily. That application of the law comes with experience and applying the law case after case after case, year after year. understanding, like I said, context is key. Understanding the context of what you're dealing with because every single case You might have five subject access request cases sit on my desk and every single one of them will be different and will require a slightly different approach. You can't just apply the same rule to every single one of them because you've got to understand the context and the nuances of what's sit beneath it. And that's where the challenge is and that's where the training and the experience come in. speaker-1: Yes. I I think it just generally it's important to raise awareness in the for the public when it comes to privacy issues. And I I also know that you are doing a lot to achieve that. There's always you know, opportunities to do more. And this reminds me to a case a couple of years ago. It was in back in my hometown and one of the hospitals were downsizing their outdated technological tools, you know, computers from the nineties. But before they did that, they forget to erase the content. They thought nobody's going to use that. So they just gave away the computers and you know, tech guys voted because they can use certain compartments and parts of those those computers. And it turned out that they stored a bunch of data with you know like healthcare data. and there was a massive data breach around that. And that was the moment when it turned out that yeah, these are data protection concerns and there were couple of safeguards they should have been, you know, managing to do before they they sell those computers. speaker-0: Yeah, because they think of it in hardware terms, in what the hardware or the software that sits within it actually holds because it's invisible. And I say, really important. speaker-1: No, they didn't even consider that there are any data protection issues there. And that is this knowledge gap that is very concerning in my view. what is one practical change a po parent or educator should make tomorrow? And what should we look for when assessing the progress, your institutional progress by twenty twenty eight? speaker-0: All right. Okay. So a couple of different questions there. So one practical change for tomorrow. Okay. It's probably goes back to what we're talking about with parents and home educating, but pick one app or AI tool that your child uses, open its settings and turn off anything that shares data or uses data to train AI models. turn it off. Then have a 10 minute conversation with your child about what they share and with whom. Because I think that conversation is worth anything. It's worth way more than just changing a setting. So yeah, look at what app, go into the settings, turn everything that shares information off, then have a chat with your child. That would be the one thing I would do. And then the question about 2028. I think hold us to the measures that we published and what we said we'd do in our strategic report. You know, have schools and their data protection officers been reached? Has EdTech been mapped? Are there fewer breaches being reported or caused by unauthorised disclosure and access? And when we take enforcement action, does the compliance stick? Or is it just treated as a one-off thing, we pay the fine, we'll do what we have to do and then we forget about it. No, we want it to stick. We want people to learn from their behavior and their mistakes and change that behavior so that they've got the right measures in place to stop it from happening again. That's what I mean by compliance sticking. I think, yeah, hold us to the measures that we said that we're going to... check things against and challenge us if we haven't done it. speaker-1: Are there any misunderstandings about privacy that you would wish to correct? speaker-0: think there are thousands of misunderstandings. One myth I would absolutely crush is when I hear, and I've heard this more times than I've had hot dinners over the 22 years of my career, that privacy means secrecy or only people with something to hide need privacy. Privacy is... is about control and fairness. It's nothing more than that. It's about knowing what is collected about you, who sees it, why it's collected, and having a say in how that information is used. A child who is safer online or a donor whose details aren't stolen or an employee who isn't judged by some kind of opaque algorithm, those are privacy outcomes and none of them involve hiding anything. So I've always... The hairs on the back of my neck go up when I hear somebody say, well, I've got nothing to hide. So, so it doesn't matter. Well, I'm sure the minute something does happen to them, they will realize that actually we've all got something to hide because it's private and we have a right to privacy. It's a fundamental human right at the end of the day. And it's there for a reason. It's, it's, it's not just one of these things that sticks on the side of the desk. It's a, it's. Its roots are well and truly documented in, if you go back to the atrocities of the second world war and far before that, that is where it comes from. That is why we need a right to privacy and that's why data protection or access. speaker-1: Yeah. And I think for these exant reasons, Europeans are a little bit more concerned about these privacy and data protection issues than on the this side of the Atlantic. But here I so sorry? speaker-0: Yeah, maybe you could be right. I would have not thought of it like that. speaker-1: But I was thinking that here other types of privacy concerns could appear. For example, scams, when it comes to, using your private information that's just out there to to steal from you. And consumer fraud and you know, scams are really big issues that need to be targeted in the long run. speaker-0: Definitely. speaker-1: I started this conversation by asking you what do you do? now I would like to ask you what is ahead of you for the next couple of years. What are your priorities and goals? speaker-0: So I think, like I said, our strategic plan is out there now. Those are the things that we're going to be focusing on. AI and human resources, which we're looking at at the moment. The next one, which is we're now in October. So in Jersey, it's Cybersecurity Awareness Month in October. So during this month, we'll be launching the second part of our strategic priorities, which is all about breach reporting and how you report those breaches and trying to reduce the number of instances of unlawful or unauthorized access to information. And then the third part which will kick off in the new year is about children's privacy and specifically education technologies. And this is really not about educating young people on how to use the tools, it's more the other way around, making sure that the teachers have done and the schools have done their due diligence on the providers that they're using and understanding the answers to those questions that we've asked throughout this session. So that's kind of our direction travel for the next three years. And then, yeah, and we've got... measurements that sit within that that we'll be trying to improve upon. So stakeholder engagement, the percentage of engagement sessions resulting in improved participant understanding is generally what we will look at there. An increase in guidance uptake, know, who's looking at that guidance and can we measure that? We'll be doing audits and looking at monitoring activities and improving AI and HR governance controls, that kind of thing. And the effectiveness of enforcement, as I mentioned earlier, what's the percentage of enforcement actions that actually result in sustained compliance, not just a short term win? So I think success for us... by the time we reach 2028 isn't we've had a bigger number of fines. It's that fewer people are harmed. There's faster and more honest breach reporting. our organizations are doing those risk assessments before the incident, not after it. speaker-1: I would like to have you back as a guest in in the time when you have the lessons and takeaways for educators on eth tech and privacy issues if you are open to that. speaker-0: We'd be delighted to talk to you again, Lilo. It's always a pleasure. speaker-1: Thank you so much for your time and your insights. it was a pleasure to host you on Brains and Bots. speaker-0: No, thank you very much. No, I really enjoyed it. Thank you. speaker-1: Thank you so much again for joining us today. We love to have you. If you enjoy the content, please consider subscribing to our YouTube channel, Apple Podcast Channel, or Spotify. Any SSM membership organization, we would love to have you join us and share your views. Come on the podcast as a guest or write for our platforms like Minding the Campus. So see you soon.