Latest / Dataministeriet - A data protection and privacy podcast / 83. Vulnerability, sensitive data, and ADM. II Guest: Gianclaudio Malgieri (ENG)

Transcript

83. Vulnerability, sensitive data, and ADM. II Guest: Gianclaudio Malgieri (ENG) · Dataministeriet - A data protection and privacy podcast

• 0:00
  Hello and welcome to another episode of Dota Ministeriat with
• 0:05
  me, your host Philip Jonson. And today I'm really happy to
• 0:10
  have another guest, Jean Claudio Maglieri.
• 0:15
  Hello and welcome. Hi Philip, Thank you for having
• 0:18
  me. And you are associate professor
• 0:20
  of Low and technology at Lighting University.
• 0:24
  You're also managing editor of Computer Low and Security Review
• 0:28
  and you coordinate VULNERA, the International Observatory of
• 0:33
  Vulnerability in data protection.
• 0:35
  You published huge number of articles.
• 0:39
  It's more than 60 now and not the least monograph
• 0:43
  vulnerability and data protection law.
• 0:46
  Yes, correct. So it's not an overstatement to
• 0:51
  say you have very deep knowledge in some of the most important
• 0:56
  topics in data protection, if you ask me.
• 1:00
  Hopefully so. Normally I will always start
• 1:03
  with. One question is in how did you
• 1:05
  even get into this area of Lua? That's, that's not a simple
• 1:10
  question actually, because I've, I've been wondering myself and I
• 1:14
  think there are two answers. One is the more scientific
• 1:18
  answer and one is the more personal answer.
• 1:21
  We start with the personal answer, which is also a way for
• 1:24
  me to open more my, my personal aspects.
• 1:27
  That is always a good way to start.
• 1:29
  So vulnerability and data protection are topics very dear
• 1:34
  to me because I've I'm usually an outlier.
• 1:37
  I'm usually a minority in several contexts where I've
• 1:41
  been, so I am an expert. I work as a Mediterranean
• 1:47
  person, I work in the Netherlands, and as a queer
• 1:52
  person. I was a minority in my village
• 1:55
  when I was born, and then also in general in Italy.
• 1:59
  So I would say that being a group, being part of groups at
• 2:04
  higher risk of marginalization was always interesting for me to
• 2:07
  explore what vulnerability of marginalization means.
• 2:10
  But talking about the more scientific reason why data
• 2:13
  protection and why vulnerability, I think the
• 2:16
  reason is I was following more private law and consumer
• 2:21
  protection, but in years where the world's big data or, you
• 2:27
  know, the data protection directive were really taking
• 2:30
  flight. And so it was really a great
• 2:33
  momentum for me to look at this. And I think all these aspects
• 2:40
  pushed me more and more towards the data economy, the data low.
• 2:45
  And then the GDPR came and it was a big tornado for research.
• 2:50
  And once you're in, you're in. When did you decide, instead of
• 2:54
  like being a practitioner, to be an academic?
• 2:58
  Oh yeah, that's also a good point.
• 3:00
  I think I've always felt in me the need for being an academic.
• 3:04
  So for me the most beautiful thing is to think and to write
• 3:09
  and to read. So when I pose and I, when I,
• 3:13
  when I cannot sleep or when I walk, ideas come to my mind and
• 3:18
  I think I, I, I feel the, the, the urge to write or to engage
• 3:22
  with literature and to read. So, and also because academia
• 3:26
  gives the possibility to work on, on your own intellectual
• 3:30
  profile and not just to work for someone else.
• 3:34
  Then of course there are pros and cons, right?
• 3:36
  If you work for a company, then when you are on holiday, you're
• 3:40
  on holiday. When you work for yourself and
• 3:42
  for your intellectual profile, you're always thinking, always
• 3:46
  mumbling. But I think that's part of the
• 3:49
  trade off and I accepted it. Yeah, and I'm a little bit like
• 3:53
  you because I can't stop my brain working even though I work
• 3:56
  in a private corporation. My brain is working during
• 3:59
  vacations and and nights and evenings.
• 4:02
  But I try not to bore my family too much.
• 4:05
  Sure, same. But getting into this
• 4:07
  vulnerability concept, because you've wrote, as I said, you've
• 4:11
  written a book and you've written an extensive number of
• 4:14
  papers on the on the subjects. And when I started reading what
• 4:20
  you have written, maybe I was a little bit nerved like it's,
• 4:23
  it's really so complicated. And then I eventually ended up,
• 4:27
  yeah, it is kind of complicated. So what what were you were like
• 4:31
  you you mentioned how you why you got into this area, specific
• 4:35
  area, but what did you find? Yeah, well, I keep finding
• 4:40
  stuff. And when I published my book, I
• 4:42
  think the best ideas came after. It's always like that, right?
• 4:48
  You write something and then more ideas come.
• 4:50
  Also because when you present a book, you have the opportunity
• 4:54
  to watch your work from a more distant position, from a more
• 4:57
  external position. So in a nutshell, vulnerability
• 5:02
  in data protection law is a topic that is extremely fruitful
• 5:07
  also because it's ambiguous. We use in the law the concept of
• 5:11
  vulnerability. Usually we have been using
• 5:14
  vulnerability under two different perspectives.
• 5:18
  The first was consumer protection law.
• 5:20
  So we have had the concept of vulnerable consumers or
• 5:24
  vulnerability in consumer protection law for many years.
• 5:29
  The I'm for Commercial Practices Directive of 2005, but also the
• 5:32
  Court of Justice had been already defining vulnerability
• 5:35
  before. Usually in that case,
• 5:38
  vulnerability really refers to mental capacity, mental
• 5:40
  competence, age, mental disability, credulity.
• 5:47
  And it's a very situated aspect now liberal approach to
• 5:51
  vulnerability, vulnerability as skills, lack of skills.
• 5:55
  Then you have the other great thread on vulnerability, which
• 6:00
  is vulnerability to marginalization, vulnerability,
• 6:04
  non discrimination law, so historical vulnerability.
• 6:08
  Then this two thread have been quite separated during the last
• 6:13
  decades. But then the technological
• 6:16
  revolution, the data economy came and they blurred the
• 6:20
  borderline between these two. Why?
• 6:23
  Because online we are all customers, consumers, so with
• 6:27
  vulnerability to our mental skills to give consent and so
• 6:30
  on. But also we are extremely
• 6:33
  exposed as citizens and extremely exposed because
• 6:38
  sometimes minorities, marginal oppressed groups.
• 6:43
  Then combining these two things brought to the topic of
• 6:48
  vulnerability and data protection law.
• 6:49
  If you ask around who is a vulnerable data subject, usually
• 6:53
  some people will tell you children or the elderly or
• 6:58
  people with lack of capacity to get consent.
• 7:01
  But other people will tell you, well, vulnerable data subjects
• 7:05
  are racial minorities because there are at high risk
• 7:08
  discrimination. Then there there should have
• 7:10
  been something to connect these two things, a more general
• 7:14
  abstract theory. And that was my interest to find
• 7:17
  for the bigger picture on why the mentally incapable and the
• 7:22
  racialized or marginalized group, what they had in common
• 7:27
  and what they had in common, in a nutshell, was two things,
• 7:32
  dependence and different levels of resilience.
• 7:36
  So dependence and resilience are really the two elements of
• 7:39
  vulnerability. Dependence is is a human thing.
• 7:42
  When we were born, we were dependent on our guardians.
• 7:48
  We were dependent on the hospital we were dependent on.
• 7:51
  We cannot, we are not made to live in a cave.
• 7:53
  Some of us are, but in general human beings are dependent on
• 7:57
  others, and this dependency can reach levels of great asymmetry
• 8:02
  sometimes when when for the exercise of our fundamental
• 8:06
  rights and so for the satisfaction of our fundamental
• 8:10
  needs, we rely on one or two or very few entities.
• 8:14
  In that case it's a very asymmetrical unilateral
• 8:18
  dependency. I depend on, I don't know, the
• 8:21
  hospital for my health. I depend on the national
• 8:24
  healthcare Service because I don't have any insurance.
• 8:26
  I depend on my employer here in a reality where there is very
• 8:30
  little very closed job market. So this dependency is extremely
• 8:36
  high in the data economy because we have today, let's say the the
• 8:41
  five biggest economy and the five biggest companies in the
• 8:44
  world, four of them have to do with data and have to do with
• 8:48
  digital platforms and the 50s media, so also producing chips
• 8:52
  for the AI. So in a, in a way, we, there are
• 8:55
  this great monopolies of data and that's how fundamental rise
• 8:59
  today. If I want to be in touch with my
• 9:00
  family, I use apps of Meta. If I want to be, if I want to
• 9:05
  express my ideas, if I want to professionally express my idea,
• 9:08
  I use an app by Microsoft, which is LinkedIn and so on.
• 9:12
  So this great dependency is 1 great element.
• 9:15
  The second element is resilience.
• 9:17
  Resilience is our capacity to react after harm to our
• 9:21
  fundamental right happens. And resilience is also very
• 9:26
  personal and it changes. And it's not just a personal
• 9:28
  element, it's also structural. If the society is ready or is
• 9:34
  well equipped to protect after a harm occurs or to make systems
• 9:39
  accountable after problems happen, then the system makes me
• 9:43
  more resilient and not just my personal resilience is at is at
• 9:47
  issue. So in a nutshell, I would say we
• 9:51
  generally consider vulnerability as something belonging to
• 9:54
  personal characteristics. So for example, I'm vulnerable
• 9:58
  because of my age, I'm vulnerable because of my sexual
• 10:00
  orientation. And this is the traditional
• 10:02
  approach that labels people. But if we broaden the picture,
• 10:08
  in addition to that, in addition to this element of personal
• 10:11
  characteristics, dependence and vulnerability are made of
• 10:15
  another aspect, which is the relationality.
• 10:17
  So I'm not vulnerable to I'm not vulnerable in general, but I'm
• 10:21
  vulnerable to someone, I'm vulnerable to my employer, to
• 10:25
  the state, to the border officer and so and so on.
• 10:28
  And 3rd, an important aspect that lawyers should never
• 10:32
  forget, we are embedded in a system socio economic,
• 10:36
  political, historical system. So the level of oppression in
• 10:39
  society, the level of marginalization, the openness of
• 10:42
  the market, the level of competition or monopolies, the
• 10:45
  level of technological advancement, the level of civil
• 10:48
  rights movements, and so on define the encounter between my
• 10:52
  personal characteristics and the power of the counterparts.
• 10:56
  And in data protection, this is this is very true.
• 10:59
  I will just conclude with two representations in data
• 11:02
  protection. One is vulnerability during data
• 11:06
  processing. So to whatever refers to data
• 11:09
  processing, for example, I'm vulnerable to consent or to
• 11:12
  understand the policies if I am in a certain situation of
• 11:16
  dependency. For example, can I really say no
• 11:20
  to cookie policies in an airport of a foreign land where I'm just
• 11:23
  landed, that I have no network and I want to communicate with
• 11:26
  my family. So in that case, I will give a
• 11:29
  very fake consent and that's a form of dependence and
• 11:32
  vulnerability during the data processing.
• 11:35
  And there are other examples like reading long privacy notice
• 11:38
  or not understanding how to exercise my rights and so on.
• 11:43
  The second data processing vulnerability is vulnerability
• 11:46
  to the effects of data processing.
• 11:48
  So for for example, I'm vulnerable because to to to
• 11:51
  discrimination after they process data bounce my sexual
• 11:54
  orientation by certain data controller in a certain
• 11:57
  political economic context. So in that case it's
• 12:01
  vulnerability as a consequence of data processing and not just
• 12:04
  during the data processing. So how would you if you were a
• 12:08
  controller like about? Do you have to identify the
• 12:12
  specific individuals or groups that you are processing?
• 12:17
  That's a very good question, whether we should really or not
• 12:22
  profile people more as vulnerable.
• 12:26
  It's a, it's a big of a, it's a bit of a paradox of this, of
• 12:29
  this the field because we would say, OK, I want to protect you,
• 12:32
  but for privacy, data protection, I should not
• 12:35
  understand if you're so vulnerable.
• 12:38
  Well, I would say that in general, there's a layer where
• 12:42
  everybody is vulnerable with when we, when we deal with
• 12:46
  certain aspects of technologies and certain aspects of society.
• 12:50
  So we can simply, we can clearly say that when we deal with big
• 12:55
  social networks and social media, there's a level of
• 12:58
  general vulnerability that should impose restrictions of
• 13:02
  certain practices. For example, the practices that
• 13:06
  the European Commission just found as problematic or
• 13:10
  addictive driven, like addictive for example, infamous scrolling
• 13:14
  or autoplay push notification, etcetera.
• 13:18
  That's just one aspect, but your question was more if we should
• 13:21
  identify specific groups. Then of course, it's not easy
• 13:26
  because it would require a higher processing of a sensitive
• 13:30
  data, special categories of data to identify why and whether a
• 13:33
  certain person is in a situation of higher dependence and
• 13:36
  vulnerability. I would say that there are
• 13:39
  several options, several possibilities to do that.
• 13:42
  First of all, impact assessments should be made participatory.
• 13:46
  So when we do a data protection impact assessment, we should
• 13:49
  understand who is at higher risk, who is more vulnerable.
• 13:53
  And we don't need to identify single individuals, but we can
• 13:55
  identify clusters based on participation of a
• 13:58
  representative of different groups, representative of
• 14:01
  children who are using my services, representatives of
• 14:05
  people with disability. And then we have in mind that we
• 14:08
  should always include more people at the table at the next
• 14:11
  round. So the question, like the
• 14:13
  question on the wall should be always who is missing at the
• 14:16
  table. And then we keep adding people.
• 14:18
  This is 1 approach to make sure that different forms of
• 14:21
  vulnerability can be protected, but to avoid that every data
• 14:26
  controller has this big passport too for analysing
• 14:31
  vulnerabilities that might also be exploited.
• 14:35
  I think there are also other ways, for example we could have
• 14:37
  third parties that can help to identify clusters of dependency.
• 14:41
  We have already third parties for social media, for example,
• 14:44
  the trusted flaggers for misinformation and
• 14:47
  disinformation and so on. Why couldn't we have some
• 14:50
  external auditors that might help to analyse who are people
• 14:55
  in higher risks of vulnerability in certain services and then
• 14:58
  propose a certain solutions for these people?
• 15:01
  For example, you don't ask for consent to these people because
• 15:04
  their consent will not be free. And so you have for example to
• 15:08
  rely on legitimate interest, which means to have higher
• 15:10
  safeguards in the balancing. Or you make some rights
• 15:14
  extremely more accessible, like right to be forgotten.
• 15:17
  You keep asking these people, you wanted your data are
• 15:19
  deleted, you wanted the data are deleted.
• 15:21
  So to make things easier or last but not, we could focus also on
• 15:26
  rebalancing the power of people. For example, with data
• 15:30
  representatives like data unions, we have something
• 15:34
  already in the law, but it's not really well connected.
• 15:37
  So we have the figure of data representatives in the GDPR.
• 15:41
  We have the figure of data cooperatives in the Data
• 15:44
  Governance Act. And for me it's extremely
• 15:48
  interesting because it's like labour unions for data subjects.
• 15:52
  So to to have also a bottom up protection and a rebalancing of
• 15:57
  power imbalance from let's say union perspective.
• 16:00
  There's not much at the moment. You have just some NGOs or, you
• 16:03
  know, data protection advocates, but I'm sure we can work on
• 16:08
  that. So I'm thinking about that with
• 16:10
  the dependencies is 1 aspect of being vulnerable.
• 16:15
  Then you're obviously, as you said, you're, you're dependent
• 16:17
  on the government on the on the big side, but you could narrow
• 16:21
  it down. You, you're also dependent on
• 16:23
  basically all services. If, if you don't live in a city,
• 16:27
  you're dependent on the grocery store because there's only one
• 16:30
  and maybe they have video surveillance.
• 16:32
  But are you vulnerable in all of the situations even you don't
• 16:36
  have this threats, young, old and so on?
• 16:40
  Yeah, dependency is really contextual.
• 16:44
  So the examples that you made are very clear examples of how
• 16:48
  contextual things are. A person with limited
• 16:51
  infrastructure, even though it's not a minority group, can be
• 16:56
  extremely dependent, but perhaps might be more resilient or
• 17:00
  perhaps there might be other mechanisms that help that person
• 17:03
  to cope with their, with her or their needs anyway.
• 17:08
  But what is interesting is that the personal characteristics
• 17:11
  matter anyway. So for example, if you are abled
• 17:15
  privileged person, then even in a situation of high dependency,
• 17:20
  perhaps there's lower risk of discrimination or lower risk of
• 17:24
  manipulation. It's really, it's really
• 17:26
  extremely contextual. The, the first thing would be
• 17:29
  for me to look at the bigger companies that have in
• 17:33
  unanimously an extreme driven business model.
• 17:39
  And then of course we could look at the small groceries etcetera.
• 17:42
  But in that case it's also the the, the trade off between
• 17:46
  benefits and so the cost, the cost benefit analysis would also
• 17:51
  bring to the continuation of companies as they are.
• 17:55
  We need the grocery to do CCT, camera, whatever because they
• 18:00
  might need to protect their property and so on.
• 18:04
  So, yeah, I think, I think it's it has a lot to do also with the
• 18:07
  economic model that we live in. And how about like services that
• 18:12
  you don't actually know? You mentioned LinkedIn for
• 18:14
  example. I mean, you don't have to use
• 18:16
  LinkedIn. Would that play into being
• 18:19
  vulnerable, that you show something?
• 18:22
  Yeah, that brings to the, that brings to a very important
• 18:25
  research we are conducting now within a project that I lead.
• 18:29
  The project is called Resocial and it's a project funded by the
• 18:32
  Dutch Research Agenda. And in this project we are
• 18:36
  analyzing vulnerability on social media.
• 18:39
  And so you mentioned LinkedIn, which is also social media, and
• 18:43
  we try to understand how people perceive their dependency on
• 18:49
  social media. There are two forms of
• 18:50
  dependency and I think it's important to look at them.
• 18:54
  One is functional or structural dependency.
• 18:57
  So I need something and I, for example, I, as you said, I need
• 19:02
  the grocery, I need my mother when I'm a kid, I need the
• 19:07
  hospital, I need my doctor. I need LinkedIn because it's the
• 19:11
  most and perhaps unique way to be known by the academic
• 19:16
  community. Before we had Twitter and now
• 19:18
  Twitter has been a bit abandoned by the scholars, for example, or
• 19:22
  the predictioners of law, ETC. Then this is functional
• 19:26
  dependency. The other dependency is
• 19:29
  addiction. So it's psychological
• 19:31
  dependency, problematic use, compulsive use.
• 19:34
  And we noticed in our study. So we did this big survey on a
• 19:37
  950 people and we try to understand how people perceive
• 19:41
  their dependency and their vulnerability.
• 19:44
  We noticed that people in higher risk of functional dependency
• 19:48
  are also people in higher risk of psychological dependency.
• 19:53
  So all people that gave us good motivations for using social
• 19:56
  media, being in touch with my family, growing with my business
• 20:00
  or art, producing ideas, expressing my views,
• 20:04
  entertainment, are also people that throw other questions, show
• 20:09
  the high level of psychological dependence on the social media.
• 20:12
  For example, people that say yeah, but I open Instagram.
• 20:15
  I don't know why. I just it's very automatic to
• 20:17
  open it. So this is, this is very
• 20:20
  interesting and makes us understand that this design, the
• 20:23
  systems are designed in a way that's really trap our attention
• 20:27
  even when we need them for a genuine use.
• 20:31
  But to go back to your question, you said perhaps I'm not obliged
• 20:34
  to go on LinkedIn or whatever. And I, I think this is an
• 20:39
  important question. So the, the personal resilience
• 20:42
  to say no, but all these systems are based on network effect.
• 20:46
  So we are the more we are, the more people want to be and the
• 20:50
  more people are online and the more we feel that we need to be
• 20:53
  there. And when I ask students, why
• 20:56
  don't you delete Instagram? Why don't you delete Tiktok,
• 20:58
  even if you know that it's extremely harmful.
• 21:00
  I teach privacy, need protection, but I teach also
• 21:03
  fundamental rights in digital technologies.
• 21:05
  So these questions come in both courses.
• 21:07
  And I, I ask people, how many of you are not on Instagram?
• 21:10
  And this year I had for the first time one student that is
• 21:13
  on zero social media. Maybe there's also generational
• 21:16
  change for Gen. Z, but most of them respond to
• 21:19
  one simple with one simple word, which is formal fear of missing
• 21:24
  out. So I would say it's not true
• 21:26
  that I can live without LinkedIn.
• 21:28
  Yes, I can, but I'm also embedded in a society made of
• 21:31
  networks, made of links. And so my psychological and
• 21:35
  functional urge to be connected to central social media and to
• 21:39
  be online, I wouldn't consider it as just a simple vicious
• 21:45
  thing. It's being part of a society
• 21:48
  that is built in a certain way. And so in a way, I would say,
• 21:51
  yes, you can live without certain social media, but at the
• 21:55
  same time you cannot live without it, which is a nice
• 21:58
  paradox. So I say it's becoming a little
• 22:00
  bit like the grocery store. You need to to have a grocery
• 22:03
  store or something to, to buy you groceries.
• 22:06
  And what you're saying is, I think that for example, if you
• 22:11
  are operating a hospital, all the people with coming in as
• 22:16
  patients, not the doctors and so on, should probably be treated
• 22:20
  as vulnerable because at least if you're up for a surgery,
• 22:25
  because you will be left out for the doctors.
• 22:29
  Yeah, yeah, totally. And also the GDPR refers to
• 22:33
  examples of relationship between doctors and patients and
• 22:39
  relationship between employer and employees, and relationship
• 22:41
  between public administration and citizens when referring to
• 22:47
  problems, for example, of consent, freedom of consent or
• 22:50
  autonomy. Some of these examples are
• 22:52
  mostly elaborations of the European Data Protection Board.
• 22:56
  But yes, I think most of there are very sensitive areas where
• 23:00
  we are in a situation of higher risk that should be considered
• 23:04
  always vulnerability, yes. And then there are other
• 23:08
  controllers that may be more needs to kind of look at the
• 23:13
  groups of people they have as customers or whatever and maybe
• 23:16
  say like, yes, we actually do have children, They should be
• 23:20
  treated as vulnerable. Yes, we actually do have people
• 23:23
  with less mental capacity. They should be treated as
• 23:26
  vulnerability. Then we have a general person
• 23:30
  with middle-aged white men. They in most cases should not be
• 23:35
  treated as vulnerable, but there will be exceptions.
• 23:38
  Yeah. And of course we need to
• 23:40
  understand that all these exercises are approximations and
• 23:45
  it's better to have a safer Internet for everybody than a
• 23:49
  safer Internet just for somebody.
• 23:51
  And this brings also to the point of banning social media
• 23:54
  for our children. We want a safer Internet for
• 23:57
  everybody, including children. We don't want that we banning
• 24:01
  children. We create an environment which
• 24:03
  is less safe for the adults in a way.
• 24:07
  So yes, I think and the the very important point is the
• 24:10
  circularity of this process. We should wonder every year and
• 24:15
  data protection impact assessments are made for that on
• 24:17
  a regular basis. We wonder what's changing are
• 24:20
  the new risks are the new groups.
• 24:23
  And the reference to affected groups is also in the
• 24:25
  description of impact assessments in other laws like
• 24:29
  the AI Act or the 3F and the mental right impact assessment
• 24:32
  as a reference to the affected groups.
• 24:34
  And that the effects on this group should the analysis during
• 24:37
  the free and also the DSA risk assessment.
• 24:41
  So in a way, it's already there, but of course, it's just trying.
• 24:46
  It's not the perfect solution. So when I read some of your
• 24:49
  places, that's a very good point of view because I think that
• 24:53
  some privacy lawyers or professionals in general like
• 24:56
  focus very much on GDPR and not that you have a lot of tools.
• 25:00
  You mentioned the Digital Services Act or the AI Act GDPR
• 25:05
  is previously directive and they're like a whole package of
• 25:10
  different digital acts. This do you find that some
• 25:15
  privacy professional maybe are too narrowed focused and and not
• 25:19
  using all the tools in the box? Yeah, it's the case, but it's
• 25:24
  also true that it's difficult just to to kind of defend some
• 25:27
  of them. It's very difficult now to
• 25:29
  navigate in such a big ocean of different laws where the
• 25:34
  overlaps and interplays are also a bit problematic.
• 25:37
  But yes, I think it's, it would be very important that every
• 25:41
  practitioners, lawyers or callers has in mind that there
• 25:45
  are so many connections between the GDPR to TSA to DMA, the GDPR
• 25:50
  to sorry, the AI Act, the Data Governance Act, the Data Act,
• 25:54
  etcetera. And why I say that and because
• 25:58
  the to make fluid of some of the elements of the GDPR, sometimes
• 26:02
  we need other laws. I made the example before of how
• 26:05
  data representatives might be connected to data Co operatives
• 26:08
  in the DGA in the Data Governance Act, but there might
• 26:11
  be other examples. You cannot really understand
• 26:13
  consent under the GDPR if you don't understand the limits for
• 26:17
  consent for gatekeepers in the DMA in the Digital Markets Act
• 26:22
  and if you don't understand the privacy directive with all the
• 26:26
  relevance to consent to cookies and implicit consent and if you
• 26:30
  don't connected with data altruism in the Data Governance
• 26:34
  Act. So there are so many links that
• 26:36
  should be investigated. And another example, very, very
• 26:41
  good example is sensitive data. So special categories of data,
• 26:45
  we think the special categories of data are something that is
• 26:49
  very well situated in the GDPR Article 9.
• 26:52
  And then you have other references for the safeguards
• 26:55
  and duties, but it's totally incomplete now because the
• 27:00
  biggest implication of sensitive of data in the digital market in
• 27:03
  the digital economy today come from other laws.
• 27:06
  For example, the Digital Services Act forbids the
• 27:10
  profiling based on forbids digital advertisement based on
• 27:16
  profiling based on sensitive data.
• 27:18
  And this is a big implication. It means I I should never
• 27:22
  receive an advertisement on meta on instrument because I'm gay or
• 27:27
  because I am of my nationality. But this is not really the case,
• 27:31
  right? We we sometimes perceive that
• 27:33
  our sensitive data are there. So this is 1 big implication.
• 27:36
  Another link for social sorry for sensitive data is the AI
• 27:40
  act. All AI providers need to put in
• 27:45
  place systems to debias algorithms, perhaps using more
• 27:48
  sensitive data. So these are all things that
• 27:51
  make us understand that the GDPR is kind of a skeleton on which
• 27:56
  many other things are added. Some years ago with the quote or
• 28:01
  we said that these were silent reforms of the GDPR.
• 28:04
  So even before the digital omnibus, actually all these
• 28:08
  small changes, deviations, parentheses and appendixes and
• 28:13
  spin offs were really important elements that were kind
• 28:18
  reforming the GDPR already. Yeah, and I think we we
• 28:21
  definitely need you researchers to to help us practitioners in,
• 28:26
  in this aspect. But I'm glad that you brought in
• 28:29
  the sensitive personal data because that was a topic I
• 28:32
  wanted to touch upon with you because obviously something you
• 28:36
  also done some extensive work on.
• 28:39
  So I mean, as you said, it's clearly defined in, in, in
• 28:43
  Article 9 like health data, sexual orientation and so on.
• 28:48
  On the other hand, it's we say Kells law that kind of it's also
• 28:52
  if you only infer the sensitivity that should be
• 28:55
  included with like with names. So what's your opinion right
• 29:00
  now? In a nutshell, my opinion is
• 29:03
  that sensitive data are a spectrum.
• 29:06
  Even though the law needs a yes or no and the law needs black
• 29:09
  and white because we need to understand whether something is
• 29:12
  sensitive data or is not. Actually sensitivity is a
• 29:15
  spectrum. We have some data that many
• 29:18
  years ago with the code to write defined quasi sensitive data.
• 29:21
  And we have other data that a few years ago in a blog post I
• 29:25
  defined super sensitive data and I will make some examples.
• 29:29
  My name is a quasi sensitive data because refers that I have
• 29:34
  Italian origins and national nationality and origins is
• 29:39
  sensitive data. It's a special category of data.
• 29:41
  But you know, we could also need to use my name for many purposes
• 29:46
  and would be very difficult to say that just using my name is
• 29:50
  sensitive data. But if we really look at the
• 29:53
  inference power, it's clear that from just a couple of data,
• 29:58
  yeah, I can apparently non sensitive data, I can infer so
• 30:02
  many sensitive characteristics. And so from an from a contextual
• 30:06
  point of view, the more data we have, the more generative power
• 30:11
  we have, the more computational power we have around.
• 30:15
  And in particular, if a company is particularly strong in
• 30:18
  inferences because it has huge amount of data, all quasi
• 30:23
  sensitive data might become sensitive data.
• 30:26
  There are very very practical examples that were used in the
• 30:30
  case law. For example, just naming if
• 30:35
  someone is married or not. It might also reveal a sexual
• 30:38
  life and sexual orientation if we understand the name or the
• 30:41
  gender of the partner. And something interesting is
• 30:45
  that the definition of sensitive data is not just for minority
• 30:49
  groups, it's neutral. So even being heterosexually
• 30:53
  sensitive data, even being healthy is sensitive data,
• 30:57
  pregnant or non pregnant. All all things that refer to
• 31:00
  these aspects are sensitive data.
• 31:03
  So I would say it's important to remember the the characteristics
• 31:09
  of the spectrum also because it's a cultural.
• 31:11
  So the sensitive data are really cultural.
• 31:13
  So the, the categories, there are categories that are based on
• 31:17
  what we think are taboos in our society.
• 31:21
  And I always do this exercise with students.
• 31:24
  I always ask them, what do you think is missing in the list of
• 31:27
  sensitive data? It's, it's a very funny exercise
• 31:30
  because they always come with great ideas.
• 31:32
  For example, they say the income, why don't we?
• 31:36
  Why? It's not a sensitive, it's a
• 31:37
  taboo. How much I have in my bank
• 31:39
  account. I never, I never disclose it,
• 31:41
  not even with my family members. Why should I disclose it?
• 31:44
  Why is not sensitive data? And the answer is cultural
• 31:47
  because some of the Nordic countries disclose many of this
• 31:51
  data for tax reasons on publicly available web pages.
• 31:55
  And so there's a different culture of showing things and
• 31:58
  also belongs to different religious and cultural reasons
• 32:02
  in Europe, like Calvinism versus Catholicism versus other the
• 32:07
  kind of cultures. The same goes with sexual life.
• 32:10
  Some cultures are extremely protective of anything that
• 32:14
  might be sexual life data and others are much more relaxed.
• 32:18
  But if we just keep these categories and we'll look at the
• 32:21
  spectrum, the other I mentioned quasi sensitive data, but on the
• 32:25
  other parts of the spectrum there are for example, the Super
• 32:27
  sensitive data. I will make an example, a
• 32:30
  diagnosis of a sexual transmissible disease.
• 32:33
  It's per SE contains at least three or two or three categories
• 32:37
  of sensitive data all together. It's health, it's sexual life
• 32:41
  and sometimes sexual orientation.
• 32:43
  So just to show, and it's extremely a source of stigma.
• 32:48
  But the same goes with the psychiatric diagnosis.
• 32:52
  There's a big stigma because we see in society that if someone
• 32:56
  has psychiatric diagnosis, there's a well, lower level of
• 33:01
  autonomy. And so, you know, there are,
• 33:02
  there are so many reasons. So in a nutshell, this was kind
• 33:06
  of theoretical, but in a nutshell I would say that the
• 33:09
  definition of sensitive data is expanding.
• 33:13
  The digital omnibus on the GDPR tried in a in a leaked version
• 33:19
  before the official one, they wanted to reduce it to known
• 33:23
  inferences, so just to directly sensitive data.
• 33:27
  But this would have contradicted the course of justice which
• 33:31
  three years ago mentioned that also inferences should be
• 33:34
  considered sensitive data. So in a nutshell, I would say
• 33:38
  that we need the definition to be brought, but it's also
• 33:42
  important that not every data is sensitive.
• 33:45
  Why? Because otherwise we would lose
• 33:47
  the protection, special protection of sensitive data in
• 33:50
  a way. So there will be the so-called
• 33:52
  data inflation. So yeah, the let's say it's
• 33:55
  difficult to say yes or no to the question of sensitive data,
• 33:59
  but we should have a very contextual approach too.
• 34:03
  I know that in in at least one article on the last point, like
• 34:07
  the devaluation of sensitive data, are we approaching that
• 34:13
  limit now that the concept is expanding, expanding further and
• 34:19
  further out? I I don't think we are at the
• 34:23
  level of total inflation of sensitive data or the evaluation
• 34:27
  of sensitive data. It's true that this data and
• 34:32
  characteristics are much more than we think.
• 34:35
  But if we look at the specific protections they have, of
• 34:38
  course, of course, there are some aspects in which the risk
• 34:42
  of not taking them seriously is very concrete.
• 34:45
  For example, generative AI training data for general
• 34:48
  purpose AI and generative AI when there's big data scraping
• 34:53
  from online, social media, etcetera, to train AI models
• 34:57
  for, for example, large language models, it's full of sensitive
• 35:02
  data. Come on.
• 35:02
  When we share data on Facebook or Instagram, it's a lot about
• 35:06
  our political beliefs, religious beliefs, philosophical beliefs,
• 35:09
  health conditions, sometimes even having some eyeglasses as
• 35:13
  health condition, right? Or not having them in another
• 35:16
  way. So and then there should be a
• 35:18
  specific or lawful basis for that.
• 35:21
  But and and so specific consent. We don't give specific consent
• 35:25
  for data scraping. It's impossible, right?
• 35:28
  You should ask consent to 1,000,000 person for just one
• 35:31
  data set or even more than you ask.
• 35:35
  So what other bases could be there?
• 35:37
  Substantial public interest. Is there a substantial public
• 35:39
  interest in training generative AI?
• 35:42
  Yeah, Well, maybe only if general TVI is for, I don't
• 35:45
  know, curing cancer and it's a health, National Health program,
• 35:50
  whatever. Then that's a clear example
• 35:53
  where exactly there's a risk of devaluation or inflation of the
• 35:57
  concept. Because since sensitive are so
• 36:01
  everywhere and since training data sets is kind of based
• 36:06
  really on also indifferentiate collection of data also, for
• 36:12
  example, sensitive data, then of course the protections of
• 36:16
  Article 9, paragraph 2. So the specific cases are not
• 36:20
  there. But it's also true that two
• 36:24
  things. First, there are some
• 36:26
  technological tools to reduce the amount of sensitive data in
• 36:31
  a data set, even though it's open to debate.
• 36:36
  So there are papers in particular ACM fact papers or
• 36:40
  other papers in computer science journals that underline how
• 36:46
  desensitize, if we can use that word, desensitize personal data
• 36:50
  is extremely limited, a limited exercise.
• 36:54
  And on the other hand, the legislator is aware of this.
• 36:58
  And so in the digital ominibus on AI, so the reform of the AI
• 37:01
  that has been approved just few days ago, the one of the
• 37:05
  compromised, then we will see how the final text will be.
• 37:08
  We are still waiting for the final text, but the there is
• 37:13
  this acknowledgement of how sensitive data are impossible to
• 37:17
  be deselected from training data sets for AI systems.
• 37:21
  And so there is a specific fuel that they want to add, which
• 37:26
  will will make sensitive data, will make always lawful to use
• 37:32
  sensitive data as long as they are not removable from the data
• 37:37
  set to train AI. And in a way this is a way to
• 37:41
  avoid the hypocrisy that OK, it's sensitive data, but what
• 37:46
  can I do? I will just process it.
• 37:47
  I don't care. So in a way, and I'm not saying
• 37:51
  it's good to what the digital omnibus on AI is doing, I'm just
• 37:53
  saying that taking a position was anyhow expected.
• 37:56
  So. In one of your pieces on
• 37:59
  sensitive data, you were talking about the purpose of the
• 38:02
  controller, why you're processing the data and whether
• 38:05
  or not that should be accounted for.
• 38:08
  And I think that's really interesting as well.
• 38:11
  And partly what you mentioned about AI like your train, but
• 38:15
  you're not actually interested in that there is sensitive data.
• 38:19
  And then if you go look at the biometric part of it, it's
• 38:23
  actually a photo. It's not biometric, even though
• 38:26
  you might have some personal trait that you're actually that
• 38:31
  you personally could do biometrics with your brain.
• 38:35
  But being a bank also we we have a lot of transactions and from
• 38:39
  the transactions, you might actually say what you paid.
• 38:43
  Maybe you paid a hospital psychiatrist and then you will
• 38:47
  know, but we're not interested in that.
• 38:49
  Should it matter? Yeah, this is the there are
• 38:52
  these two parallel approaches to consider sensitive data, either
• 38:55
  contextual or purpose based. The contextual is, yeah, from
• 38:59
  whatever I can infer something. So it's sensitive data.
• 39:02
  So the fact that the bank uses my photographs with my
• 39:05
  eyeglasses means that contextually the bank could
• 39:09
  process health data of me about my eye functions.
• 39:13
  And then that's the contextual approach.
• 39:15
  But the purpose approach is more to look at really what is the
• 39:19
  purpose of the data processing. And we might tend to think that
• 39:24
  the second approach is less protective, but there are cases
• 39:26
  in which the purpose can define more.
• 39:29
  For example, I'm using some data that apparently are not
• 39:31
  sensitive at all, but the purpose is to infer your sexual
• 39:36
  orientation. So for example, I want to know
• 39:38
  your music or the the the songs that you like or things like
• 39:41
  that, but they have. But the purpose is not to just
• 39:44
  understand your music preferences, but to infer
• 39:46
  something about your sexual orientation or sexual life.
• 39:49
  So in a way, I would say that also purpose matters.
• 39:52
  In the GDPR, the only explicitly purpose based category of
• 39:57
  sensitive data is biometric, because it's biometric for the
• 40:00
  purpose of uniquely identifying individuals.
• 40:02
  But for example, the AI Act has kind of redefined biometric data
• 40:06
  in a broader way because we have a broader definition of
• 40:09
  biometrics and also broader limits of biometric.
• 40:13
  For example, there's a prohibition in the AI Act to do
• 40:16
  biometric inferences. Biometric categorisation on the
• 40:20
  basis of sensitive data, for example, or or protective
• 40:23
  categories. So yeah, in a nutshell, I would
• 40:26
  say it's it's it's, it's always good to look at the context and
• 40:31
  the purpose. And I would not assume that any
• 40:36
  possible data can refer to any possible sensitivity sensitive
• 40:40
  aspects depending on the data controller.
• 40:43
  It's then there a challenge to prove that they didn't process
• 40:48
  in a sensitive way also. OK.
• 40:49
  Thank you. And I wanted to lastly discuss
• 40:54
  automatic decision making because I know that's an
• 40:57
  interest of you. And actually it's one of a
• 41:01
  favorite topic right now for me as well.
• 41:04
  And not the least that with AI algorithm.
• 41:07
  More and more I think actual automatic decisions are made in
• 41:12
  light of like the show for case and so on.
• 41:14
  Maybe there are more automatic decisions making going on than
• 41:18
  we are aware of. That is not like decisions as we
• 41:22
  think of them normally, but actually some some part of a
• 41:27
  process like the show for case is automatic decisions.
• 41:31
  Yeah. So I mean, you've done some
• 41:33
  research on this as well. And today sometimes you talk
• 41:37
  about not you, but in general, like AI being a black box and
• 41:41
  you build to build to be transparent as a controller,
• 41:45
  obviously. And how, how?
• 41:47
  How should you? And is it possible to like
• 41:50
  fulfil all the requirements? Yeah, yeah, that's, that's never
• 41:57
  ending topic, of course, because we, because there was this Court
• 42:01
  of Justice case law recently, not just SHUFA, but we also, so
• 42:06
  we had also done in Bradstreet about the right to explanation
• 42:10
  and so on. And also the digital omnibus
• 42:14
  wants to change Article 22 of the GDPR, extending it not just
• 42:20
  to when it's necessary for a contract and for consent or
• 42:23
  member state law, but also for any time there's a contract.
• 42:27
  So this will extend the possibility ability to do
• 42:30
  automated decisions also to cases that are do doable by
• 42:34
  humans, for example, counter moderation.
• 42:37
  If there's a terms and terms and condition, I could say yes for
• 42:40
  that. For the purpose of this
• 42:41
  contract, then I will use automated decision making.
• 42:45
  But also a contract is the contract between students and
• 42:49
  universities. So in a way that's also a way to
• 42:52
  open the door to more AI on students or on employees in a
• 42:57
  way. Of course, there are some
• 42:58
  restrictions in the yeah. Now your question was whether
• 43:01
  it's. So whether also fragments of
• 43:04
  automated decisions can matter. So for example, in the SHUFA
• 43:07
  case was taking the advice of an automated system about the
• 43:11
  credit liability or credit worthiness of something and of
• 43:15
  someone and then taking the decisions on that.
• 43:17
  And this was, this was a very important case because when the
• 43:20
  GDPR was approved and then published, we were all uncertain
• 43:25
  about how to interpret that solely automated basis.
• 43:28
  So it's yeah, automated decision making is decision based solely
• 43:34
  on automated basis. And the the point is what solely
• 43:39
  automated means at the beginning, the work papers
• 43:43
  saying also important scholars saying the solely automated
• 43:46
  means that any human intervention should be excluded
• 43:49
  to be in the scope of the article. 22 and I was one with
• 43:54
  some colleagues and co-authors to say no.
• 43:57
  I mean, it would be so easy to circumvent the system if, for
• 44:01
  example, you put a useless human in the in the in the system.
• 44:06
  And we always made the example, which is not banks, but very
• 44:09
  similar insurances. So for example, you have an
• 44:13
  algorithm saying that Philip is at risk 8 out of 10, and then
• 44:18
  there's a powerless insurance employee who says, OK, risk 8
• 44:25
  means that this person will pay more than others.
• 44:29
  It seems to a human decision because there's a human at the
• 44:32
  end of the loop, but that human had no decision making power and
• 44:36
  no epistemic power, we could say.
• 44:38
  So in a nutshell, I'm very happy that the court in SHUFA said
• 44:42
  made this analysis saying, look, 80% of the decisions of the
• 44:47
  Shufa data set and of the automated systems in Germany is
• 44:53
  taken into serious consideration by banks.
• 44:55
  So for the 80% of cases, decisions are based on what are
• 44:59
  confirming what Shufa says for whether to, to, to give or not
• 45:04
  to give a mortgage to someone alone or whatever.
• 45:07
  Then the court set this 80% standard, which I, I'm not sure
• 45:12
  it's a standard that can be applied forever and in the
• 45:15
  future or if, even if it's a binding standard, but it gives
• 45:21
  us an idea. It's more, it means it's more
• 45:25
  than not. So it's more convincing than
• 45:28
  not. It's more so it's the majority
• 45:31
  of cases are relying on that automated decision.
• 45:35
  So in that case, I would say that it's we should really look
• 45:39
  at the substance and also at the possibility of humans to take
• 45:45
  other decisions. There was some very beautiful
• 45:48
  human computer interaction studies conducted some years ago
• 45:53
  about quasi automated decisions and semi automated decisions,
• 45:57
  for example, for judges. And what they realized was that
• 46:01
  usually there's this confirmation bias.
• 46:04
  So we want to confirm what the machine says, right?
• 46:07
  And so even though there's not a specific duty to conform with
• 46:12
  automated decision, it's it's very often the case that we
• 46:16
  conform with it. And a way to avoid this
• 46:20
  automatism might be to say, OK, I give a very detailed human
• 46:25
  based explanation on why I agree with the machine.
• 46:27
  The problem is that today you can write this explanation with
• 46:30
  generative AI. So today you can say, please
• 46:34
  tell me what to do. And then you say, OK, please
• 46:36
  explain to me why in very specific ways this might be a
• 46:40
  good way, a good position. And that's why we don't need
• 46:43
  just explanation, we need justification.
• 46:47
  So we need to show how certain decision is reasonable, but it's
• 46:51
  also rooted on lawfulness, fairness and purpose, limitation
• 46:56
  and other principles. So I was thinking about this
• 46:59
  human in the lope. If you have a few more minutes,
• 47:03
  as you said, it needs to be a person that actually
• 47:06
  knowledgeable in the subject obviously might have the right
• 47:09
  to kind of take a new decision. But if you are an, it's really
• 47:15
  hard to actually understand how algorithm work and explain it
• 47:22
  and challenge the outcome, especially if you have used the
• 47:27
  same algorithm like for a year and you had the human envelope,
• 47:32
  it never made an error. So that's one of the risks with
• 47:35
  a lie that you're kind of getting very comfortable with a
• 47:39
  lie to do the right thing and you kind of lower your awareness
• 47:44
  of that. It might make a mistake one day.
• 47:47
  That's a very, very true point. And this brings also the passion
• 47:52
  of liability, whether following or not following the advice of
• 47:56
  AI might be a source of liability.
• 47:58
  For example, you committed a mistake following it, so it was
• 48:01
  your fault that you didn't do a good human intervention exercise
• 48:05
  or you didn't follow it, so it was your fault because it was
• 48:08
  already there, it was so clear and you didn't follow the
• 48:10
  advice. So of course this is a, this is
• 48:13
  a big question. The answer in one word might be
• 48:16
  literacy. So what the Artificial
• 48:19
  Intelligence Act already suggests that we should have AI
• 48:23
  literacy. So we should all companies that
• 48:26
  use AI in any way should have programs for employees, for
• 48:31
  workers and also contractors to learn about the functioning but
• 48:37
  also the risks. So also legal literacy and I
• 48:41
  would add something also emotional literacy.
• 48:44
  Having to deal with the AI is really also in particular
• 48:48
  generative AI and chat bots and so on, brings really the problem
• 48:52
  of how we emotionally are intelligent and capable, capable
• 48:55
  to deal with something that is extremely new and is non human
• 48:58
  but resembles humanity a lot by design.
• 49:01
  But now going back to your point, the AI act requires that
• 49:04
  there is human intervention for a high risk system.
• 49:07
  So in a way, it would be always the case that some humans are
• 49:11
  able to intervene in the system and also interpretability
• 49:15
  Article 13. So of the so we should be able
• 49:18
  to always interpret the IA system.
• 49:20
  And that's also something that would always create a tension.
• 49:25
  I mean, in an ideal word, in an utopistic word, there wouldn't
• 49:28
  be this passive reliance on the algorithm that has always been
• 49:32
  used in that way. But there should be a waste of
• 49:34
  questioning. Is it correct?
• 49:36
  Should we continue? Should we deal with that?
• 49:39
  So this kind of approach, so this is one thing and in general
• 49:43
  literacy is very important. In this very specific case the
• 49:47
  the European institutions want to downgrade literacy from a
• 49:51
  duty to a recommendation and even the duty had no sanctions
• 49:55
  in case was violated. So if you don't do literacy
• 50:00
  under the current AI Act, you don't have any sanction.
• 50:02
  But if you have a duty to do literacy and then you are not
• 50:06
  able to do right explanation, human intervention, human
• 50:10
  interpretability, then the fact that you had no literacy
• 50:13
  programming a tissue would amplify your liability, I would
• 50:17
  say, and your negligence. So in, in tort law, I think the
• 50:21
  idea of literacy could be used, but now they want to downgrade
• 50:24
  it to a mere recommendation, which is, I don't know why we
• 50:27
  should have a low with some just recommendations.
• 50:30
  I mean, other systems do that. Other countries do that.
• 50:33
  I don't think it's the best way to proceed.
• 50:35
  But by the way, I think that would be a very important
• 50:39
  aspects to, to, to, to focus on and to never give algorithms for
• 50:43
  granted just just for the sake of questioning, right?
• 50:47
  These are not perfect systems. They're full of structural
• 50:49
  biases. They're full of structural
• 50:51
  mistakes and errors. So just relying on it is not the
• 50:54
  case. And I also tell this to my
• 50:56
  student, they use a lot of generative AI, as you can
• 50:59
  imagine. And sometimes they they are not
• 51:01
  even able to critically analyse what's written there.
• 51:05
  So the point would be at least be a very nasty reviewer of what
• 51:10
  AI is doing. It's the basis, right?
• 51:12
  Yeah, I completely agree that I, I am a little bit worried about
• 51:16
  the future because if you as as you said, if you are brought up
• 51:22
  with AI, are you losing critical thinking?
• 51:24
  Are you losing some of the like basic threads to actually
• 51:28
  questioning the output? That's something for the future.
• 51:32
  I have two kids myself and I let them play around with it under
• 51:38
  my supervision and tell them like it's not a person, it's a
• 51:42
  machine and it's guessing what the next word is all the time.
• 51:46
  Exactly, exactly. It's a probabilistic tool.
• 51:49
  Mostly yes. So I am so grateful for you
• 51:53
  taking the time today and I encourage all the listeners to
• 51:58
  go to your web page or Google your name and find the most
• 52:02
  interesting papers, articles, and the book.
• 52:06
  Thank you. Because I've written so much, I
• 52:08
  think it's here to like first narrow down the scope.
• 52:11
  Do you want to read what Jean Claudia has written about LDM or
• 52:17
  is it about vulnerable vulnerability?
• 52:19
  So you focus on where in the area?
• 52:21
  Otherwise we will be overwhelmed.
• 52:23
  And I want to send out the big thank for all you researchers or
• 52:28
  academia for, for bringing up so much interesting topics and
• 52:31
  actually taking the time to dig deep dive, jump down the rabbit
• 52:36
  hole. Sometimes you, it seems like
• 52:39
  you're never coming out on the other end, but it's so
• 52:43
  interesting. And I think it's more of
• 52:45
  practitioners should read not only what you have written, but
• 52:49
  your colleagues as well, because it it's it creates a deep for
• 52:54
  understanding of the protection of JD PR and the interplay with
• 52:57
  other digital legislations. So big thank you for
• 53:02
  participating today and all the hard work.
• 53:04
  Thank you so much. Thank you for reading, for
• 53:06
  taking the time to read. It's not something that you give
• 53:09
  for granted. And I also would say that also
• 53:12
  scholars should sometimes talk more to practitioners to get
• 53:17
  from the clouds to reality. So thank you so much.
• 53:20
  And I also, I also thank you for this podcast, which is an
• 53:24
  important source of connection between practitioners and
• 53:27
  scholars. So thank you so much.
• 53:29
  And thank you all to the listeners.