Latest / Paths of Curiosity in an AI World / Q3 - AI in Data, Behavior, and Human Systems with Jathan Sadowski, PhD

Transcript

Q3 - AI in Data, Behavior, and Human Systems with Jathan Sadowski, PhD · Paths of Curiosity in an AI World

• 0:04
  SPEAKER_00: Welcome to Pass of Curiosity in an AI world where we explore Tamara's careers through the lens of AI.
• 0:12
  SPEAKER_00: You've been sorted, scored, and steered by systems you never agreed to.
• 0:18
  SPEAKER_00: Someone built those systems, and someone profits from them.
• 0:22
  SPEAKER_00: Jason Sadowski is here to tell you exactly who.
• 0:26
  SPEAKER_00: Let's get curious.
• 0:34
  SPEAKER_00: Jason Sadowski is an associate professor and ARC future fellow in the Emerging Technologies Research Lab at Monashu University in Melbourne, Australia, and an associate investigator with the ARC Centre of Excellence for Automated Decision Making and Society.
• 0:54
  SPEAKER_00: He is the author of Too Smart, How Digital Capitalism is Extracting Data, Controlling Our Lives, and Taking Over the World.
• 1:01
  SPEAKER_00: As well as The Mechanic and the Luddite, a Rufused Criticism of Technology and Capitalism.
• 1:08
  SPEAKER_00: He hosts the podcast This Machine Kills and his research on the political economy of technology has been published in Science, Technology, and Human Values and Social Studies of Science, among others.
• 1:22
  SPEAKER_00: Dr.
• 1:23
  SPEAKER_00: Sidowski, thank you so much for joining us.
• 1:26
  SPEAKER_01: Thanks for having me.
• 1:28
  SPEAKER_00: Your new book is called The Mechanic and the Luddite.
• 1:31
  SPEAKER_00: Most of our listeners have heard Luddite used as an insult.
• 1:35
  SPEAKER_00: Someone who's bad with technology.
• 1:37
  SPEAKER_00: What does it actually mean?
• 1:39
  SPEAKER_00: And why does it matter for understanding AI right now?
• 1:44
  SPEAKER_01: Yeah, it's a great question.
• 1:45
  SPEAKER_01: And you're right.
• 1:46
  SPEAKER_01: Luddite is very much used as this derogatory term.
• 1:49
  SPEAKER_01: It's how we've been taught to use it, right?
• 1:53
  SPEAKER_01: But my book really is trying to recover the original meaning of what the Luddites were, who the Luddites were, right?
• 2:01
  SPEAKER_01: So this is 200 years ago at the beginning of the Industrial Revolution, you have the original Luddites, the Luddite rebellion.
• 2:10
  SPEAKER_01: And really, ultimately, what it was was not this movement based on fear of technology, right?
• 2:17
  SPEAKER_01: As if people were just kind of blindly smashing whatever machine they came across.
• 2:22
  SPEAKER_01: You know, this very kind of primitivist idea that we have with Luddite now, where it's either someone who's bad with technology or someone who is afraid of it, wants to get rid of it, right?
• 2:32
  SPEAKER_01: But the original Luddites were themselves artisans, they were craftspeople, they were mechanics, they were very skilled workers in the industrial age.
• 2:43
  SPEAKER_01: And they used technologies every day of their life.
• 2:47
  SPEAKER_01: They built many of these technologies that they then later smashed.
• 2:51
  SPEAKER_01: So it wasn't this fear of technology that motivated them.
• 2:54
  SPEAKER_01: What it was ultimately was a shift in the social relationships that govern how we use technology and for what end.
• 3:03
  SPEAKER_01: And so for the original Luddite, it was about the coming of the this new age of industrial capitalism, right?
• 3:10
  SPEAKER_01: So the kind of factory system was coming into being, you know, the uh small or capitalist bosses that had large amounts of money, right?
• 3:21
  SPEAKER_01: This kind of small elite group started taking over ownership of production and the machines, and were using these technologies, these industrial machines, to speed up the pace of work, to degrade the quality of work, right, to make work more dangerous and precarious.
• 3:39
  SPEAKER_01: And it was this that motivated the original Luddites to not initially push back and smash machines, right, but instead try to negotiate with the bosses, try to reach some kind of consensus if that didn't work, right, and it didn't.
• 3:54
  SPEAKER_01: Then they, you know, picketed, they petitioned, they protest, right?
• 3:59
  SPEAKER_01: And if that didn't work, and it didn't, right?
• 4:02
  SPEAKER_01: And then the last thing they tried was, well, we've tried everything we possibly can to try to make our lives better, to try to maintain some agency and control over the ways that we work and over the communities that depend on this work.
• 4:17
  SPEAKER_01: And when at every turn, when they were dismissed or ignored, then it was at that moment that they joined together under these kind of secret uh societies that were known as the Luddites and began smashing the machines, right?
• 4:32
  SPEAKER_01: Smashing machines, again, not because they were afraid of technology, but because they saw the technologies, the machines, as tools of larger political, economic, and social systems that were oppressing them, exploiting them, extracting their labor and value in various ways.
• 4:49
  SPEAKER_01: And so smashing the machines was a way to fight back against this new age of exploitative industrial capital.
• 4:59
  SPEAKER_01: You know, the Luddites, I always say it was a labor movement, right?
• 5:02
  SPEAKER_01: It was not an anti-technology movement, it was a pro-labor movement.
• 5:06
  SPEAKER_01: And so, in my work and the work of many other people who are now writing very favorably about the Luddites and trying to resuscitate this history and say what lessons can we learn for how the Luddites understood technology and responded to it, it's all about seeing this as a question of power, right?
• 5:26
  SPEAKER_01: Whose power is materialized in the technologies, whose power is uh exercise and increase or enhancing the technologies.
• 5:36
  SPEAKER_01: And on the other side of that, right, is who is disempowered in various ways through technologies?
• 5:43
  SPEAKER_00: That's a great question for our students to be answering.
• 5:47
  SPEAKER_00: Though, how would you describe what you do today to your student self?
• 5:51
  SPEAKER_01: That's a very good question.
• 5:54
  SPEAKER_01: Um I mean, I have a very I think all of us have this nonlinear pathway, right?
• 6:01
  SPEAKER_01: Uh it it's I think it's very rare to kind of set your mind when you're a student, when you're you know a young person, say, this is the thing I'm going to do in my life, and then you just go step by step in a linear way, and you achieve that thing.
• 6:16
  SPEAKER_01: You say, okay, I did exactly what I said I was gonna do when I was 15 years old or 18 or 25 or whatever it might be, right?
• 6:25
  SPEAKER_01: Um so for me, how what I would explain, what I do to my student self, is that I study technology, right?
• 6:35
  SPEAKER_01: And but importantly, what I do is I study technology as a way to study society, as a way to study power dynamics, as a way to study the ways in which technologies open up new possibilities, but also are used to foreclose other alternatives, right?
• 6:53
  SPEAKER_01: Some societies are brought into being, some futures are actualized because of certain technologies, but at the same time, other forms of society, other possible futures are closed off.
• 7:06
  SPEAKER_01: And so what I do is I study technology as a way to study what kind of futures we can and should be building, and and you know, who is benefited by that.
• 7:17
  SPEAKER_00: So can you walk us through your path?
• 7:19
  SPEAKER_00: How did you build a research career at the intersection of technology, capitalism, and power?
• 7:25
  SPEAKER_00: What were some of the pivots that mattered most?
• 7:29
  SPEAKER_01: Yeah, absolutely.
• 7:31
  SPEAKER_01: So I initially started my undergrad career as a polymer chemistry major, as a chemical engineering major.
• 7:40
  SPEAKER_01: I went to Rochester Institute of Technology, so a really good technology university, and I thought I was gonna do science, right?
• 7:49
  SPEAKER_01: Chemical engineering.
• 7:50
  SPEAKER_01: I was very as a high school student, I was very much into taking physics and to AP Chem, right?
• 7:58
  SPEAKER_01: I was taking AP English and all that stuff, but I really thought I was gonna be a scientist.
• 8:03
  SPEAKER_01: And when I was a kid, I would always say, when I grow up, I'm gonna be a scientist, right?
• 8:08
  SPEAKER_01: Um, everyone else was saying they want to be uh a football player or musician or an actor, and I always said I want to be a scientist.
• 8:17
  SPEAKER_01: And and it looked like maybe I was actually going to do that, right?
• 8:21
  SPEAKER_01: But once I started college and I was about halfway through a polymer chemistry degree, I was working in a lab as a lab assistant and all of this, I took an elective course because you know you have to take some electives, go outside of the science faculty, and and you know, learn some other stuff.
• 8:42
  SPEAKER_01: And I took an elective on critical thinking, which is basically just a philosophy 101 course.
• 8:48
  SPEAKER_01: And it was from there that something really sparked.
• 8:51
  SPEAKER_01: I was like, this is such an interesting way to think about the world, but also to think about science and technology and its relationship to science with or to society, was really thinking about science as a social construct, as a social institution, thinking about you know it the way that technology relates to society, asking these more philosophical questions about ethics and politics and and what these things mean and values, right?
• 9:21
  SPEAKER_01: It's something just really clicked with me.
• 9:23
  SPEAKER_01: So that one class really put me down this slippery slope to taking more and more philosophy courses, filling up my electors until eventually I was like, if I want to take more philosophy classes, I just have to change my major.
• 9:37
  SPEAKER_01: And so I did.
• 9:38
  SPEAKER_01: I changed my major from a uh polymer chemistry degree into a philosophy degree at a technology school, um, but I was really focusing then on like you know, ethics, politics, and philosophy of technology.
• 9:53
  SPEAKER_01: And so from there, I I wanted to, you know, I graduated, I always kind of knew I wanted to do more studying.
• 10:02
  SPEAKER_01: I I really enjoyed academia, I really enjoyed doing research and reading.
• 10:06
  SPEAKER_01: I was like, if I can make this a job, then that that sounds great.
• 10:10
  SPEAKER_01: Um, and so I ended up going into a master's program and then rolling that into a PhD program, really focusing more on bringing together my initial interest around science and technology with this new love and fascination I have around ethics and politics and social theory and philosophy.
• 10:31
  SPEAKER_01: Um, and ended up finding out about in grad school this field called science and technology studies, which is basically an interdisciplinary social science field that takes science and technology as objects of critical inquiry and study.
• 10:47
  SPEAKER_01: Um and so I ended up getting a PhD in that where I was able to bring these things together.
• 10:54
  SPEAKER_01: And over time my interest really shifted away from kind of science, you know, chemistry and things like that, and more towards questions around digital technologies, information technology.
• 11:05
  SPEAKER_01: So I you know did my PhD research in on smart cities, on smart urbanism, right?
• 11:12
  SPEAKER_01: And and my work is always kind of focused on that, which is how I ended up now being uh an associate professor in a faculty of information technology, where my students are computer science and software engineering students.
• 11:24
  SPEAKER_01: So it was this very uh circuitous path, starting with science, having, you know, getting sidetracked by philosophy, getting a PhD in social science, and now being a professor in an information technology uh school.
• 11:41
  SPEAKER_01: When something just clicks in your mind and you know you see the world in a new way, or rather, and more so I think is that it taps into interests that you have latently but aren't maybe focusing on or really activating in this way.
• 11:57
  SPEAKER_01: So it is really crazy to have these like sliding door moments, is what they're called, right?
• 12:04
  SPEAKER_01: Where you can kind of like see your life take a totally different pathway if that one event did not happen, right?
• 12:12
  SPEAKER_01: And so it's like if I didn't get this interest triggered by a philosophy course that then led me into social science and led me into where I am now, maybe I'd be working as a chemical engineer at, you know, at a somewhere and just kind of working in a lab every day and having a totally different life than I do now.
• 12:32
  SPEAKER_01: But um that one that one class really did, I think, put me on a different pathway.
• 12:38
  SPEAKER_00: Your first book, Too Smart, argues that smart technology from smart speakers to smart cities is driven by two imperatives: data and expanding control.
• 12:50
  SPEAKER_00: Can you give us one concrete example of how those two imperatives play out in something a student is using right now?
• 12:59
  SPEAKER_01: Yeah, for sure.
• 13:01
  SPEAKER_01: I mean, the that book is just full of examples of all these different smart systems, right?
• 13:06
  SPEAKER_01: So we think about smart technologies as ones that are data driven, networks together in some way, and automated, right?
• 13:15
  SPEAKER_01: And right now, you know, we we live in this kind of smart society where we're surrounded by things, whether it's our toasters, right, or smart toasters that are hooked up to the internet or hooked up to a smart home hub at our house that makes all of our appliances talk to each other and they're collecting data about what kind of bread you're toasting and sending that to manufacturers.
• 13:38
  SPEAKER_01: And maybe there's an automated feature where you can set the bread, you know, the toaster to uh turn on at a certain time of day, right?
• 13:46
  SPEAKER_01: Like we have everything from the toasters that we use to the coffee machines to the cars we drive every day are now um incredibly connected and full of sensors and automation or and are I mean, most people don't think about the car as just you know as containing thousands of sensors that are collecting data about when you drive, where you drive, how you drive, sending that all over the place, right?
• 14:14
  SPEAKER_01: There's all these examples everywhere.
• 14:16
  SPEAKER_01: I I think from a student point of view, though, when we think about these questions of extracting data and extending the control, so many of you know the ed tech, right, the education technology that is now part of teaching in school, whether it's in elementary school, high school, college, right?
• 14:37
  SPEAKER_01: At every level now, there's all kinds of platforms that are being used to manage uh you know your teaching, right?
• 14:45
  SPEAKER_01: Your teaching resources.
• 14:46
  SPEAKER_01: There's all kinds of software that's being used for remote proxying of exams or learning and things like that, right?
• 14:54
  SPEAKER_01: I mean all of these technologies that we use are ultimately premised on extracting data and expanding control, even if they're even if they have another purpose, right?
• 15:06
  SPEAKER_01: The technology is meant to make learning better or easier, more convenient, right?
• 15:11
  SPEAKER_01: Um to make communication between students and their teachers and parents a lot more a lot smoother and more frictionless, right?
• 15:20
  SPEAKER_01: Like we have these kinds of design decisions and priorities that are used to motivate the thing and to sell the thing to users, but at the end of the day, underneath that are also all kinds of priorities and goals around extracting enormous amounts of data.
• 15:38
  SPEAKER_01: So you know these platforms are collecting incredible amounts of data from students, from teachers, from the schools, and centralizing that in the companies and manufacturers who run the platforms, right?
• 15:54
  SPEAKER_01: This becomes extremely valuable data to be used for other purposes, right?
• 15:59
  SPEAKER_01: Maybe it's to create analytics around student learning, but maybe it's also to build other products for advertising or to sell those products to schools, right?
• 16:16
  SPEAKER_01: To get schools enrolled and captured into these bigger ecosystems that are controlled by a handful of technology companies, right?
• 16:24
  SPEAKER_01: You know, there's there's a huge amount of data extraction that happens, even when we consider that there's a lot higher privacy requirements when we're talking about students, right?
• 16:37
  SPEAKER_01: But the devil is in the details right there.
• 16:42
  SPEAKER_01: We have examples of these AI-first alpha schools that they're called, right?
• 16:47
  SPEAKER_01: Where you see this now being used in places around Texas, for example, or in California, right?
• 16:54
  SPEAKER_01: Where these there are these schools that are really about uh you know using AI as the teacher, you know, as the as the primary learning motivator and driver for students.
• 17:05
  SPEAKER_01: And you know, human teachers are just really there as more to help troubleshoot problems with the AI-driven platform and not really to teach the students, but to be like IT support for the students as they use these platforms.
• 17:21
  SPEAKER_01: And there's been reporting by Wired magazine showing as well that you know students are bringing you know their laptops home with them from these schools, and these laptops are you know uh installed with really uh extreme surveillance software because they want to be able to see every single thing the student is doing on the laptop, right?
• 17:44
  SPEAKER_01: Every keystroke, every activity, really log all of that, monitor that in incredible detail, which includes using the built-in microphone and camera on the laptop to record students, to observe students.
• 18:00
  SPEAKER_01: And there's been places where students and parents students you know go into the settings of the software and you know put and toggle some settings off around when or where the student can be recorded.
• 18:16
  SPEAKER_01: And you know, students were being recorded at home, right?
• 18:19
  SPEAKER_01: They bring their laptop and the laptop was recording them at home in their bedrooms.
• 18:24
  SPEAKER_01: Um, and so you have these real extreme cases of data extraction, and that data extraction is being used ultimately to empower forms of expansive control over students, right?
• 18:38
  SPEAKER_01: And so I think this is a primary motivation for collecting all this data about how students use platforms, how they access resources, how they spend their time.
• 18:50
  SPEAKER_01: It's ultimately to create these really strict forms of control over students.
• 18:56
  SPEAKER_01: And so, you know, it's it's long been said in sociology, you know, the famous sociologist and philosopher Michel Ficot, right, talked about the uh similarities between schools and prisons as these kinds of institutions that are designed to discipline and punish people in various ways, right?
• 19:18
  SPEAKER_01: To kind of drive people into being good citizens or good normal people in society through discipline.
• 19:25
  SPEAKER_01: And so it's interesting to see these kind of theoretical institutional analyses of the similarities between schools and prison.
• 19:34
  SPEAKER_01: Now the technologies are really making that explicit and very clear, right?
• 19:39
  SPEAKER_01: That like, you know, ultimately through ed tech, so much of it is around increasing the ability of schools to surveil discipline, control, punish, and you know, the students and ultimately teachers as well, right?
• 19:56
  SPEAKER_01: If we consider teachers as employees, then schools who themselves are subjected to all kinds of you know key performance indicators and monitoring requirements and forms of control and discipline as well.
• 20:11
  SPEAKER_01: So yeah, all that's to say is that my first book, Too Smart, is really really lays out how these kinds of systems exist in our everyday lives, in our homes, in our cities.
• 20:24
  SPEAKER_01: Um, but the schools are not exempt from this kind of creeping level of increased data extraction and technological control.
• 20:37
  SPEAKER_00: You've written about a concept called data as capital.
• 20:41
  SPEAKER_00: The idea that our personal data isn't just collected.
• 20:45
  SPEAKER_00: It's accumulated like a resource that compounds over time.
• 20:49
  SPEAKER_00: What does that mean for a student who's been generating data since they were a child?
• 20:55
  SPEAKER_01: Yeah.
• 20:57
  SPEAKER_01: And I think this understanding of data as capital is really key here because it shows that data is really subjected to these forms of economic imperatives and market pressures that other forms of capital like money or machinery or labor are also where there's this imperative to accumulate as much data as possible, oftentimes without having any real clear sense of what that what you're going to do with that data.
• 21:29
  SPEAKER_01: How is it valuable?
• 21:30
  SPEAKER_01: How is it going to be used?
• 21:31
  SPEAKER_01: The economic imperative, the market imperative is to just get as much of it as you can and then and then figure out how to create value from it later down the line.
• 21:43
  SPEAKER_01: And so what that means then is that a lot of our technology systems, a lot of our institutions are really built around this imperative of data accumulation, right?
• 21:56
  SPEAKER_01: It's not a surprise when we hear about the Enormous amounts of data that are being collected, right?
• 22:03
  SPEAKER_01: When we hear about the hundreds of data center developments that are now popping up all around the world, you know, and that is because the current kind of craze and bubble around artificial intelligence has made this imperative to accumulate data even more intensified, right?
• 22:27
  SPEAKER_01: That now AI gives this promise of being of giving something to do, a valuable, a way to derive value from all of this data that has been being collected and will be continue collected.
• 22:41
  SPEAKER_01: And so what does that mean for someone who, I mean for all of us, right, but especially for someone who their entire life has been betified, right, has been subjected to this form of data collection and accumulation.
• 22:53
  SPEAKER_01: I think what that what it ultimately means is it's more important than ever to understand that these systems exist, right?
• 23:01
  SPEAKER_01: I think there's only just now a growing public awareness of this data economy and how expensive and expansive it is, right?
• 23:12
  SPEAKER_01: I think with that growing awareness also needs to be a real critical appraisal of it.
• 23:20
  SPEAKER_01: It's no longer, and it never has been sufficient to say, well, I don't have anything to hide, so why should I care, right?
• 23:28
  SPEAKER_01: Or to say that, well, privacy is dead, the stuff is all just inevitable, there's nothing you can do about it, right?
• 23:34
  SPEAKER_01: These are the kinds of things that I think uh people in these companies have tried to say, this is what young people think, and therefore it doesn't matter what we do because young people don't care about privacy.
• 23:46
  SPEAKER_01: They don't care about sharing their whole lives on platforms and putting it into the hands of these companies and all of all of that.
• 23:54
  SPEAKER_01: I think all of that was an excuse to justify the kinds of data extraction that these companies wanted to do, and taking advantage of young people who didn't know that this stuff was going on and so weren't given the ability to have a considered opinion about it, let alone actually have a say in like, well, no, actually, I don't want that to happen, right?
• 24:19
  SPEAKER_01: And so to me, the the response here for people who have had their whole lives data-fine is to be aware of this data economy, that it that it exists, be aware of what it does, but also be aware that you have a choice, right?
• 24:32
  SPEAKER_01: That like you know that there is always a possibility to say no, to refuse.
• 24:38
  SPEAKER_01: And maybe that is hard, right?
• 24:40
  SPEAKER_01: And right now we live in a society where saying no is made harder and harder all the time.
• 24:46
  SPEAKER_01: But I also know I uh uh I I I've I've met a lot of great people in the Luddite clubs of New York City uh and in other places around the world where you see high school students and college students calling themselves Luddite, claiming this label as a way to do a form of very considered refusal of saying, I'm not going to engage with these platforms or smartphones even or these companies, not because you know I'm a primitivist and I just want to turn back the clock or something like that, but rather because I think it's a reaction to growing up with this sense that no is not an option, right?
• 25:33
  SPEAKER_01: That you don't have the ability to refuse, that all you can do is acquiesce, is accept it, right?
• 25:40
  SPEAKER_01: And I think you can only push people so far before they want to reclaim that agency, reclaim that control over their lives.
• 25:51
  SPEAKER_01: And part of that is reclaiming control over the data that's created about your life.
• 25:57
  SPEAKER_01: Um, and so I see this right now with this with the widespread um backlash against data center development, right?
• 26:04
  SPEAKER_01: The latest surveys show that over 70% of people are against data centered developments in their communities, right?
• 26:12
  SPEAKER_01: The majority of people do not think that AI is going to benefit them more than it's going to harm them, right?
• 26:19
  SPEAKER_01: And so I think this is good, actually, right?
• 26:22
  SPEAKER_01: I think this is good because the tech sector, as the you know, as one of the mass largest industries in the world, right, as some of the most wealthy and powerful companies in the world, the tech sector has really benefited from the benefit of the doubt, from this kind of quiet acquiescence to whatever they want to do, right?
• 26:42
  SPEAKER_01: From justifying their actions as good for us.
• 26:46
  SPEAKER_01: Um and the, and that there's nothing you can do about it because it's progress, right?
• 26:50
  SPEAKER_01: Um, and right now I think we're seeing a real widespread pushback against that.
• 26:55
  SPEAKER_01: And a lot of that is being driven by students, right?
• 26:58
  SPEAKER_01: Students who are booing commencement speakers at universities to say anything positive about AI, right?
• 27:05
  SPEAKER_01: Students who are refusing to use these technologies because they don't agree with the political, economic, ethical, or social impacts of them.
• 27:16
  SPEAKER_01: Um, and I think this is the kind of things that we should take our cues from.
• 27:20
  SPEAKER_01: That, you know, no is always an option and that we have to exercise our agency.
• 27:25
  SPEAKER_01: And if you've spent your whole life beta-fied up until this moment, that just means that there's even more uh of an responsibility for you to exercise that agency rather than just uh continue to let things happen to you.
• 27:42
  SPEAKER_00: You've written and just talked about how AI systems don't just reflect existing power structures.
• 27:48
  SPEAKER_00: They are actively reinforcing and extending them.
• 27:52
  SPEAKER_00: What does that mean in practice for a student who gets denied an opportunity by an algorithm?
• 28:01
  SPEAKER_01: Yeah, I mean I think this is the most I think this is one of the most important things to to learn about any technology, but you know, AI in particular here is that they are all the products of human decisions, human values, and human dis of human usage, right?
• 28:23
  SPEAKER_01: Like none of these technologies uh exist on their own.
• 28:26
  SPEAKER_01: They do not exist as like autonomous forces in the world, right?
• 28:31
  SPEAKER_01: And their outcomes are supported by, catalyzed by human choices.
• 28:38
  SPEAKER_01: And I think what we see right now is a lot of using technology as a scapegoat, as a way to deny that a human value or human choice was made, um, and that it was instead of the technology, right?
• 28:53
  SPEAKER_01: And so, you know, I didn't I didn't deny your job application, an algorithm did, right?
• 28:58
  SPEAKER_01: Uh, or I didn't deny your promotion, an algorithm did, or I didn't do this to you or that to you, an algorithm did, right?
• 29:07
  SPEAKER_01: But that's that's obstacle up, you know, that's deception, right?
• 29:11
  SPEAKER_01: That is that is uh lying that you know it's cynical because at the end of the day, the you know, an algorithm is never going to tap you on the shoulder and tell you to pack up.
• 29:22
  SPEAKER_01: This is my office now, right?
• 29:24
  SPEAKER_01: It's always a boss behind the algorithm or behind the robot who has made a decision to use that technology to downsize, to reduce head count, to uh lower wages, to degrade the quality of working conditions, right?
• 29:40
  SPEAKER_01: There's always a boss, uh a consultant, uh a company behind those decisions.
• 29:47
  SPEAKER_01: The technology just becomes used as a shield for it, right?
• 29:51
  SPEAKER_01: And I think as long as we continue to recognize that it's human agency and human choices and human values all the way down, then we recognize as well that it's not some like alien, inhuman, autonomous force outside of humanity or society that's doing these things to us.
• 30:10
  SPEAKER_01: It's just other people, right?
• 30:12
  SPEAKER_01: It's other people making decisions about our lives, it's other people um making decisions about what we can and can't do, and it's other people using technology as a way to hide or justify those decisions.
• 30:25
  SPEAKER_01: But the all of human civilization is uh is based on people trying to make decisions about others, and then those people saying, I don't agree with that decision, right?
• 30:39
  SPEAKER_01: That's the basis of politics, right?
• 30:41
  SPEAKER_01: Politics is uh is about conflict between people's values, conflict between people's visions about how the world should work, conflict about people's choices, about other people, right?
• 30:54
  SPEAKER_01: That's ultimately the the heart of politics with a small fee, right?
• 30:59
  SPEAKER_01: Not politics as we think of like the government, right?
• 31:02
  SPEAKER_01: But politics has this human conflict over values and choices and visions around the world.
• 31:09
  SPEAKER_01: And ultimately, I think of technology as uh produced by politics, right?
• 31:15
  SPEAKER_01: Produced by some people's values and priorities or you know, uh preferenced over other people's values.
• 31:24
  SPEAKER_01: Um, and politics is also about how those technologies are used, right?
• 31:29
  SPEAKER_01: Somebody has decided that it's more valuable to build algorithms that can automate the hiring and honoring process because it benefits them in some way, right?
• 31:41
  SPEAKER_01: But that just means that if it's always humans making these decisions, it means that there's always other decisions that other humans can make as long as they recognize that there is room for that kind of conflict and for those kinds of uh differences of value and opinion and goals.
• 32:02
  SPEAKER_00: The mechanic and the Luddite argues that we can't fix these systems by better technology.
• 32:08
  SPEAKER_00: We have to change the systems underneath.
• 32:11
  SPEAKER_00: For students who want to push back, what does that actually look like?
• 32:17
  SPEAKER_01: Yeah, absolutely.
• 32:18
  SPEAKER_01: Yeah, I think we can only get so far by trying to solve technological problems with technological solutions, right?
• 32:27
  SPEAKER_01: Just kind of tweaking around the parameters of the system.
• 32:31
  SPEAKER_01: Um ultimately, I think we have to take inspiration from the Luddite, right?
• 32:37
  SPEAKER_01: So in the book, right, I kind of pose these two complementary figures.
• 32:42
  SPEAKER_01: They're two sides of the same coin, the mechanic and the Luddite.
• 32:45
  SPEAKER_01: It's not an either or, you've got to be close.
• 32:47
  SPEAKER_01: The mechanic, to me, is this kind of role model for understanding how technologies work, what they do, right?
• 32:55
  SPEAKER_01: How you can fix them, repair them.
• 32:57
  SPEAKER_01: Um, but ultimately it's about really having a clear material sense of the operation of the technology, because so much of our understanding of technology is based on marketing, right?
• 33:08
  SPEAKER_01: It's it's marketing from technology companies telling you about how their system works, right?
• 33:14
  SPEAKER_01: But we all know you don't trust marketing, you don't trust advertising as being a hundred percent true, right?
• 33:21
  SPEAKER_01: There's there's always deception, there's always some lying, there's always bending the truth, or and there's always, especially when we're talking about technology, there's always hype claiming that the thing can do things that it cannot possibly do or is not able to do at this time, right?
• 33:38
  SPEAKER_01: So for me, the mechanic is really there's a responsibility on all of us to actually understand what these technologies do, how they work, and importantly what they cannot do, right?
• 33:51
  SPEAKER_01: And so, because that gives us power as well, right?
• 33:54
  SPEAKER_01: And I'm I'm not saying you have to like know how to build a machine learning model or know how to code a complex algorithm, but it's more that our understanding of these things should come from somewhere besides the companies selling the thing, right?
• 34:11
  SPEAKER_01: That we should find more trusted, independent, and critical sources of information about how the technologies work to then build our understanding of them.
• 34:22
  SPEAKER_01: And that has to be the basis for any kind of response to the technology, right?
• 34:26
  SPEAKER_01: And to me, this is where the the Luddite as this role model comes in.
• 34:30
  SPEAKER_01: You know, the original Luddites were mechanics, they were artisans, they understood how these technologies work, but then they took that understanding to a next level, right?
• 34:40
  SPEAKER_01: Because then they thought about who do the technologies work for?
• 34:44
  SPEAKER_01: What work do they do in society?
• 34:46
  SPEAKER_01: Are they working for me?
• 34:47
  SPEAKER_01: Are they benefiting me?
• 34:48
  SPEAKER_01: Do I have power and control over them?
• 34:51
  SPEAKER_01: Um, or is it someone else, right?
• 34:53
  SPEAKER_01: And am I just maybe I get maybe I get benefited, maybe I get harmed, but maybe I just don't really have a choice in the matter, right?
• 35:01
  SPEAKER_01: I just kind of have to live with someone else's decision.
• 35:03
  SPEAKER_01: The Luddite is really about saying, you know, who do the technologies work for, but then also ask the questions about should this technology exist?
• 35:13
  SPEAKER_01: Right?
• 35:14
  SPEAKER_01: Does this thing deserve to exist in society?
• 35:16
  SPEAKER_01: I think about this as the Bure Kondo principle, right?
• 35:20
  SPEAKER_01: We instead of you know the pre-condo, the you know, decluttering uh queen, right?
• 35:27
  SPEAKER_01: Who the principle was everything that you have, you should hold it up and say, does this thing spark joy?
• 35:32
  SPEAKER_01: Right?
• 35:33
  SPEAKER_01: And if the answer is no, then you should probably then maybe you should throw it away, right?
• 35:36
  SPEAKER_01: Get rid of it.
• 35:37
  SPEAKER_01: It's not it shouldn't be in your life.
• 35:40
  SPEAKER_01: To me, I think we should do the same thing with technologies, right?
• 35:42
  SPEAKER_01: We should hold up the technology and say, not does this thing spark joy, but does this thing make my life better?
• 35:50
  SPEAKER_01: Does it make you a better person?
• 35:52
  SPEAKER_01: Does it contribute to social welfare in some way, right?
• 35:56
  SPEAKER_01: Making a better society.
• 35:58
  SPEAKER_01: And once we hold up and compare technologies to that very simple criteria, I think you'd be surprised at how many of them fail to pass that simple test of does this make my life better, or does this make society better off?
• 36:14
  SPEAKER_01: Um and so from there, if the answer is no, then you know we have we have choices, right?
• 36:22
  SPEAKER_01: And I think most of us haven't even gotten to that point yet of holding up a technology and saying, does this thing make my life better, or does this make society better off?
• 36:31
  SPEAKER_01: I think a lot of people are starting to do it now with AI.
• 36:35
  SPEAKER_01: I love to see that.
• 36:36
  SPEAKER_01: But then we have to follow those conclusions to some endpoint, right?
• 36:41
  SPEAKER_01: If the answer is no, it doesn't make my life better, no, it doesn't deserve to exist, it doesn't make society better off, then what do we do about it, right?
• 36:49
  SPEAKER_01: Do we let it continue to exist, knowing that it's it's bad in some way, right?
• 36:55
  SPEAKER_01: That it doesn't deserve to exist, or do we do something about it, right?
• 36:59
  SPEAKER_01: Do we change it?
• 37:00
  SPEAKER_01: Do we transform it?
• 37:01
  SPEAKER_01: Do we destroy it?
• 37:02
  SPEAKER_01: Do we get rid of it?
• 37:04
  SPEAKER_01: And right now, I think those are questions that we have to ask ourselves individually, but more so socially, right?
• 37:13
  SPEAKER_01: I don't think it's enough to individually go through and do a digital detox, right?
• 37:19
  SPEAKER_01: Because you know it's great, right?
• 37:21
  SPEAKER_01: If you want to spend your weekends smartphone free, you know, turn off the internet, enjoy a kind of digitally detoxed life, and that makes you that makes you better as a person, makes you feel better, then do that by all means, right?
• 37:39
  SPEAKER_01: I'm not saying don't do that at all.
• 37:41
  SPEAKER_01: I'm very considered and intentional about the kinds of technologies I use in my life and how I use them.
• 37:47
  SPEAKER_01: Even though I work in the Faculty of Information Technology, right?
• 37:51
  SPEAKER_01: Um I it is not a uh just uh uh adopt everything enthusiastically, bring it all into my life.
• 37:58
  SPEAKER_01: It's like everything has to pass a test.
• 38:00
  SPEAKER_01: Does this, will this make my life better?
• 38:03
  SPEAKER_01: Um but I think what we also need is to take that and apply that principle as communities and as a society, right?
• 38:11
  SPEAKER_01: Because ultimately one person can only do so much.
• 38:15
  SPEAKER_01: And it's very limited what one person can do in the face of larger social structures and economic systems and technological industries and so on.
• 38:28
  SPEAKER_01: But what we can do as communities and as a society is a lot more, right?
• 38:33
  SPEAKER_01: If we can formalize these kinds of principles in policy or in our communities around what technologies we use, how we assess and evaluate the technologies, how we get rid of technologies that already exist but maybe should not exist, right?
• 38:52
  SPEAKER_01: Again, I think we see this happening in the debates around data centers and AI, and I would love to see a lot more of that and spread out a lot further as well.
• 39:03
  SPEAKER_01: AI and data centers are not unique in this way.
• 39:06
  SPEAKER_01: They are not the only technologies that have social impacts or that are political.
• 39:12
  SPEAKER_01: All technologies need to be treated as political, treated as the you know uh the materialization of human values and choices, uh which you know prioritize some people over other people, that includes some people and exclude other people, right?
• 39:33
  SPEAKER_01: That benefits some people but maybe harm even more people.
• 39:37
  SPEAKER_01: AI is not unique in this, data centers are not unique in this.
• 39:41
  SPEAKER_01: What I think we need to do starting from a young age to build the habit is really questioning every technology and holding it up to that simple test.
• 39:52
  SPEAKER_01: Does this make my life better?
• 39:54
  SPEAKER_01: Does this contribute to creating a kind of society that I want?
• 39:58
  SPEAKER_01: And if not, then we gotta do something about it.
• 40:02
  SPEAKER_00: You've studied what you call the landlord model of tech platforms, where companies own the infrastructure people depend on and extract rent from everyone who uses it.
• 40:14
  SPEAKER_00: How does AI deepen that model?
• 40:19
  SPEAKER_01: Yeah, I mean, uh most of the AI systems that we see now are based on this landlord model or rent year model, right?
• 40:28
  SPEAKER_01: And so this landlord model is away from you know, with this research, with this analysis that I have, specifically in my book, but also in a journal article I wrote called The Internet of Landlords, is really trying to understand how the business model of platforms and of a lot of technologies now is really built on uh this these forms of enclosure, right?
• 40:53
  SPEAKER_01: Kind of using software to enclose intellectual property, to enclose services that we need for our lives, for socialized, for our professionalized, whatever it might be, and kind of you know keeping them behind these gates, right?
• 41:11
  SPEAKER_01: Where you have to pay uh you have to pay a toll, you have to pay for access every time you want to use a platform, every time you want to use a large language model like ChatGPT or something like this, right?
• 41:24
  SPEAKER_01: Maybe that maybe you have to pay with money, right?
• 41:28
  SPEAKER_01: You have to buy tokens to use Chat GBT or to use quad.
• 41:33
  SPEAKER_01: Um, or maybe uh as is often the case, you have to also pay with data, right?
• 41:39
  SPEAKER_01: In order to use Instagram, you're not paying money to use Instagram, but you are paying in the form of giving Instagram an incredible amount of data, which is then used for targeted advertisement, sold on to other companies for other purposes, right?
• 41:58
  SPEAKER_01: I mentioned our cars earlier uh in the episode, right?
• 42:03
  SPEAKER_01: Our vehicles now are collecting incredible amounts of data about us.
• 42:08
  SPEAKER_01: I mean, just like it's mind-boggling the amount of data that our cars are collecting about us when we drive them.
• 42:15
  SPEAKER_01: And that data is a uh is a form of rent that we're paying to the manufacturers of our cars.
• 42:22
  SPEAKER_01: Long after you buy your car and own it outright, you're still paying rent to the manufacturer in the form of data because companies then take that data and they sell it on to insurance companies, to advertisers, to all manner of other industries, right?
• 42:40
  SPEAKER_01: And so what this means ultimately is that it's another way of taking control and agency away from users, away from the people who use these technologies, and centralizing it back into the manufacturers, into the companies that you know own the technology.
• 43:03
  SPEAKER_01: Because no user, none of us, any software that we use, we don't own it.
• 43:09
  SPEAKER_01: We all agree to software licensing terms when we use software.
• 43:15
  SPEAKER_01: What that means is we're licensing the software.
• 43:18
  SPEAKER_01: That is another way of saying you're renting it.
• 43:20
  SPEAKER_01: You're renting access to the software.
• 43:22
  SPEAKER_01: And what that means is that you don't have control over that software.
• 43:25
  SPEAKER_01: It can be turned off at any moment if you reach the terms of conditions, right?
• 43:30
  SPEAKER_01: We see this with, you know, there's been a lot of legal battles with farmers around what's called the right to repair, because a lot of uh like the John Deere tractors, for example, you know, all these vehicles are run by software.
• 43:43
  SPEAKER_01: They're all computers on wheels, right?
• 43:46
  SPEAKER_01: And if you're a farmer and you have a John Deere tractor or any other farm equipment, and it's absolutely crucial to your business and your livelihood, you can't do what you do without that equipment.
• 43:58
  SPEAKER_01: You might own the Vehicle, right?
• 44:01
  SPEAKER_01: The big hunk of metal and rubber.
• 44:03
  SPEAKER_01: But you're still licensing, you're renting the software.
• 44:07
  SPEAKER_01: And what that means is that there's a lot of terms that are put on you in terms of what you can and can't do with that software.
• 44:14
  SPEAKER_01: So for example, you cannot repair a John Deere tractor yourself.
• 44:19
  SPEAKER_01: You have to take it to an authorized repair shop that pays for authorization from John Deere, and then they have to run diagnostics on the machine, they have to do the repair, right?
• 44:34
  SPEAKER_01: And so you're having to pay for something that you might be able to do yourself, and the the repair shop is also having to pay John Deere for the right to be an authorized repair shop.
• 44:46
  SPEAKER_01: And so it creates this bracket, right, where you don't have control over your system.
• 44:51
  SPEAKER_01: And if you breach those terms of conditions, so you do some kind of um un you know uh some kind of modification or repair on your on your John Deere tractor that was unauthorized, well they'll turn the software off, right?
• 45:06
  SPEAKER_01: They'll break your tractor in the same way that they can break your phone, that Apple can break your phone, right?
• 45:11
  SPEAKER_01: Now you just have this unmovable object because the software license has been revoked.
• 45:17
  SPEAKER_01: That is the same exact kind of relationship that we have with anything we use in our lives that have software embedded into that.
• 45:26
  SPEAKER_01: Our toasters, our cars, our social media platforms, our education technologies, whatever it might be.
• 45:34
  SPEAKER_01: So I think it's really crucial to understand that what this means is that the modern technology industry has multiplied lots of little landlords that occupy parts of our lives, these little rentiers that have enclosed some software that we need to use in our life, and that we have to keep paying rent, monetary rents, data rents, whatever it might be, in order to remain in good standing with that software landlord.
• 46:04
  SPEAKER_01: And as and and but at any moment, our right to usage of that software can be revoked.
• 46:12
  SPEAKER_01: At any moment, right, we can the thing can just stop working, and it really reveals to us how little control and agency we actually have over the technologies that are integrated deeply into all of our lives.
• 46:28
  SPEAKER_00: What do you think is the single most important skill for a student to start building at the intersection of technology, capitalism, and power?
• 46:39
  SPEAKER_01: I think the single most important skill is critical reasoning, right?
• 46:46
  SPEAKER_01: Is to really be able to ask hard questions about what does this thing do?
• 46:53
  SPEAKER_01: How does it work?
• 46:54
  SPEAKER_01: Who does it work for, right?
• 46:56
  SPEAKER_01: Why does it exist?
• 46:58
  SPEAKER_01: Right?
• 46:58
  SPEAKER_01: What is it doing in my life as society?
• 47:01
  SPEAKER_01: Like, in other words, it's about not just accepting on the face that things exist because they have to exist, or because there's no other way for them to exist.
• 47:12
  SPEAKER_01: That the kind of social structures and power dynamics that govern our lives, the kinds of technological systems that really mediate everything that we do, um, the kind, you know, whatever it might be, right?
• 47:24
  SPEAKER_01: The kinds of economic imperatives, the you know, market pressures, like all of that should be open to questioning, right?
• 47:33
  SPEAKER_01: This should be open to critical inquiry rather than just accepted as the only thing, the only way things can be, and there's nothing I can do about it, right?
• 47:44
  SPEAKER_01: I think right now AI, if we want to go back to that example, is really premised around not getting people to engage critically with the technology, right?
• 47:56
  SPEAKER_01: You use Chat GPT, you use Clog, you use one of these large language models.
• 48:01
  SPEAKER_01: And the way they are designed to interact with users is to make you forget that you're interacting with a technology that's owned by a massive company, right?
• 48:14
  SPEAKER_01: Um to make you forget that the company has an imperative around engagement, right?
• 48:22
  SPEAKER_01: It wants you to keep using the technology, right?
• 48:25
  SPEAKER_01: Uh and these these things are put into our lives, we've integrate them into our lives, but they are designed to make us not think about them, to question them, right?
• 48:38
  SPEAKER_01: And right now, I think it's more important than ever to have that ability to really critically think about the thing in front of us, which is asking us to not think critically about it, right?
• 48:50
  SPEAKER_01: Whether that's a technology, whether that's a company, whether that's a policy, whatever it might be.
• 48:56
  SPEAKER_01: There's a lot of things in our lives that are designed to prevent people to think critically about why they exist, what they're doing, who they're working, who they're ultimately benefiting or working for.
• 49:08
  SPEAKER_01: And what we have to do is, oh, I'm not going to just kind of get sucked into the convenience, into the frictionless kind of interaction of not thinking critically about this thing that is taking over my life in various ways.
• 49:24
  SPEAKER_01: But instead saying everything needs to be open to that critical inquiry.
• 49:28
  SPEAKER_01: And you know, ultimately, I think that critical inquiry can lead us in a lot of great and interesting ways because to think to start thinking really critically and seriously about a technology like AI, for example, you need to know more about how it works, how it was built, why, right?
• 49:44
  SPEAKER_01: So now you have to go do a little bit of research, a little bit more thinking, right?
• 49:48
  SPEAKER_01: And that you know, that leads you down some other paths, that leads you down other paths.
• 49:52
  SPEAKER_01: So I think critical thinking, critical reasoning in this way also becomes a way to open up doors to just learn more about the world, to learn more about how things work in the world, about our place in the world.
• 50:05
  SPEAKER_01: And that then becomes a platform for thinking about how we can build a different world, a better society, to exercise our own agency in new and important ways.
• 50:16
  SPEAKER_01: So to me, the critical thinking is just like this slippery slope that leads us down a pathway that we should be running down.
• 50:27
  SPEAKER_00: And finally, are there any books, podcasts, or other media you'd recommend to a student who wants to go deeper?
• 50:36
  SPEAKER_01: Of course.
• 50:37
  SPEAKER_01: I mean, it's very self-indulgent to say, of course, I have a podcast and I have two books.
• 50:43
  SPEAKER_01: But I do.
• 50:44
  SPEAKER_01: And so um I do c I do host a podcast or co-host a podcast every week called This Machine Kills, which is really about the political economy of technology.
• 50:54
  SPEAKER_01: So we're really thinking about technology in this relationship between social power and financial value.
• 51:01
  SPEAKER_01: That's really what I mean by political economy, is thinking critically about those dynamics and thinking about how they relate to technology.
• 51:10
  SPEAKER_01: Um, so I do that podcast every week, just machine kills, and I have you know two books, Too Smart and The Mechanic and the Leadite, that I know are taught in high school classes and university classes all over the place.
• 51:24
  SPEAKER_01: And so I wrote them in a way to let people who are new to thinking about these topics really give them a way to get into it to understand it better.
• 51:35
  SPEAKER_01: So they're not like dense academic books, right?
• 51:39
  SPEAKER_01: Um and so you know, I it's a little self-indulgent to use this question as a way to plug my own work.
• 51:45
  SPEAKER_01: Um, but I do think that they are really great.
• 51:48
  SPEAKER_01: Those are really great starting points for anyone who wants to think more critically and seriously about technology and its place in our lives.
• 51:57
  SPEAKER_00: Well, Dr.
• 51:58
  SPEAKER_00: Sadowski, thank you so much for joining us today on Paths of Curiosity.
• 52:03
  SPEAKER_01: Thanks so much.
• 52:04
  SPEAKER_01: It was really fun.
• 52:10
  SPEAKER_00: This has been Paths of Curiosity in an AI world, exploring careers through the lens of AI.
• 52:17
  SPEAKER_00: Until then, stay curious.