Latest / Paths of Curiosity in an AI World / Q3 - AI in Data, Behavior, and Human Systems with Kentaro Toyama, AB, PhD

Transcript

Q3 - AI in Data, Behavior, and Human Systems with Kentaro Toyama, AB, PhD · Paths of Curiosity in an AI World

• 0:04
  SPEAKER_00: Welcome to passive curiosity in an AI world where we explore the hardest careers through the lens of AI.
• 0:13
  SPEAKER_00: Technology is never the final solution.
• 0:16
  SPEAKER_00: It makes already positive situations better while also making broken systems even worse.
• 0:24
  SPEAKER_00: Dr.
• 0:24
  SPEAKER_00: Kintaro Tayoma spent a decade at Microsoft learning that the hard way.
• 0:30
  SPEAKER_00: And he says AI is about to prove it all over again.
• 0:34
  SPEAKER_00: Let's get curious.
• 0:42
  SPEAKER_00: Kintaro Tayoma is a WKK Log professor of community information at the University of Michigan School of Information and a fellow of the Dalai Lama Center for Ethics and Transformative Views at MAT.
• 0:58
  SPEAKER_00: He is the author of Geek Heresy Rescuing Social Change from the Cult of Technology.
• 1:04
  SPEAKER_00: He co-founded Microsoft Research India, where he built and led the Technology for Emerging Markets Group connecting research across rural and urban communities in the developing world.
• 1:17
  SPEAKER_00: Before that, he researched computer vision and multimedia at Microsoft Research in Redmond and Cambridge.
• 1:26
  SPEAKER_00: He also taught mathematics at Assession University in Ghana, was inducted into the ACM SIG CHI Academy in 2025, and won an SIG CHI Societal Impact Award the same year.
• 1:42
  SPEAKER_00: He holds a Bachelor of Science in Physics from Harvard and a PhD in computer science from Yale.
• 1:49
  SPEAKER_00: Thank you so much for joining us today.
• 1:56
  SPEAKER_00: You spent years at Microsoft building technology meant to help some of the world's most underdeserved communities.
• 2:02
  SPEAKER_00: And you ended up writing a book called Geek Heresy.
• 2:06
  SPEAKER_00: What did you find that made you question everything you've been working towards?
• 2:12
  SPEAKER_01: Well, I'll tell you about one technology project that we tried.
• 2:16
  SPEAKER_01: We'd go to rural government schools in India and find that a few had computers, but they would only have a handful of them.
• 2:24
  SPEAKER_01: And so when a class of, say, 30 students visited a computer lab, anywhere from five to ten students would crowd around a single computer, and of course, not all of them would be able to interact.
• 2:34
  SPEAKER_01: In fact, usually one dominant child would take over and everybody else would just be looking on.
• 2:39
  SPEAKER_01: So we thought of a very simple solution to this, which was to plug in as many mice as there were children and then rewrite software so that there were multiple differently colored cursors on the screen, and you could design educational games for that.
• 2:52
  SPEAKER_01: And we found in tests that for some kinds of learning tests, you could have as many as five children at a single PC, and they could learn as much as if they had the whole computer to themselves.
• 3:02
  SPEAKER_01: So that was a great research project.
• 3:04
  SPEAKER_01: But then when uh as soon as we tried to take this idea to other schools, we kept running into a range of challenges that had nothing to do with that technology.
• 3:12
  SPEAKER_01: Some headmasters would keep their PCs locked up in a closet because they were afraid they would break.
• 3:17
  SPEAKER_01: Some teachers didn't want to teach anything with computers because they were themselves insecure in their own understanding.
• 3:23
  SPEAKER_01: And most schools didn't have anything like IT support.
• 3:26
  SPEAKER_01: It was unreliable as to whether uh the PCs were for children.
• 3:31
  SPEAKER_01: So I saw this over and over again, not just in that project, but in other contexts where research ideas with new technologies worked in the lab or in a controlled setting, but not when we took them to other places.
• 3:44
  SPEAKER_01: Um and so over time uh I basically came to the conclusion that technology by itself doesn't make things better.
• 3:56
  SPEAKER_00: How would you describe what you do today to your student self?
• 4:01
  SPEAKER_01: I mean, for as long as I've been conscious, I've thought that I wanted to be a research scientist, so I think I would tell my younger self that I have successfully been able to do that.
• 4:11
  SPEAKER_01: Yeah, these days I study digital technology's impact on society, especially on marginalized communities, and also how those communities use technology for their own purposes.
• 4:21
  SPEAKER_01: I think my younger self started off as a technologist.
• 4:24
  SPEAKER_01: I used to do research and developing artificial intelligence and other digital technologies.
• 4:30
  SPEAKER_01: Probably my younger self would find it interesting that I made a shift.
• 4:35
  SPEAKER_00: You started as a physicist at Harvard, got a PhD in computer science from Yale, went to Microsoft, and co-founded a research lab in India.
• 4:46
  SPEAKER_00: After decades building technology for humanitarian causes across that impressive trajectory, you concluded that no technology could cause a social change on its own.
• 4:58
  SPEAKER_00: Looking back at that path, was there a specific moment or project where you realized that?
• 5:04
  SPEAKER_01: Yeah, I wouldn't say there was a specific moment, but certainly there was a period of my life when I was at this research lab in India that I helped found for Microsoft, where I was engaged in over 50 different research projects, just like the Multiple Mice project I talked about earlier, but in different contexts for healthcare, for governance, for financial purposes, and so on.
• 5:29
  SPEAKER_01: And overall, over time, what I kept finding was that we could do research with new technologies that we designed for ourselves for the purpose that we thought it would be good for.
• 5:41
  SPEAKER_01: And again, they would work in a controlled setting, but when we took them to the real world, there were many, many other challenges that often didn't even let the technology be used.
• 5:51
  SPEAKER_01: And even if the technology actually got to the point of being used, there were other kinds of issues.
• 5:56
  SPEAKER_01: And after a few years of doing these kinds of projects, I looked back and the conclusion I came to was that in the end, what technology does primarily is not necessarily to improve things, but to amplify whatever underlying human forces there already are.
• 6:12
  SPEAKER_01: And so what that means is if the human forces are well-intentioned and capable, that's great.
• 6:17
  SPEAKER_01: Technology makes things better.
• 6:19
  SPEAKER_01: Uh but in so many of the challenges that we were trying to address, the underlying human forces were, you know, were indifferent or dysfunctional or sometimes outright corrupt.
• 6:29
  SPEAKER_01: And in those situations, even the best design technology doesn't uh fix the problem.
• 6:36
  SPEAKER_00: So you talk about the law of amplification, the idea that technology amplifies whatever human factors are already in place.
• 6:46
  SPEAKER_00: Could you give us one example where it played out in a way you didn't expect?
• 6:53
  SPEAKER_01: Yeah, so um, you know, again, I think the multiple mice example is one where we had a good technology, but that technology amplifies the underlying pedagogical and organizational capacity of a school or a teacher, right?
• 7:09
  SPEAKER_01: So if those capacities aren't in place, the technology doesn't do too much.
• 7:14
  SPEAKER_01: An example that I see every day in the way that people use, let's say, social media, is that on the one hand, it amplifies the positive things that people want to do, connecting with their families, trying to organize for a cause that they believe in and things like that.
• 7:29
  SPEAKER_01: But it also amplifies the negative side of people.
• 7:33
  SPEAKER_01: So you get online trolls who anonymously say terrible things to other people.
• 7:38
  SPEAKER_01: It spreads misinformation because we our collective ability to distinguish between good and bad information might not be uh up to par, and so on.
• 7:46
  SPEAKER_01: And so all of these are examples where the technology amplifies the underlying human element, whether the human element is positive or negative.
• 7:57
  SPEAKER_00: If the law of amplification is right, it cuts through a lot of the utopian promises we've been hearing about AI in education, healthcare, and hiring.
• 8:09
  SPEAKER_00: For our listeners who are actively being sorted by these systems right now, whether in college admissions, automated resume screening, or financial aid, what determines whether that amplification helps them or hurts them?
• 8:25
  SPEAKER_01: Right.
• 8:25
  SPEAKER_01: So uh what the law of amplification says is that you have to look at who has power over a technology to see what gets amplified.
• 8:34
  SPEAKER_01: And with AI, at least for now, it's very clear that it is uh a handful of people and companies that have the ability to create it, and then of course the uh the organizations that ultimately purchase the use of AI and how it will be used.
• 8:51
  SPEAKER_01: And unfortunately, those the people who are trying to build out AI right now don't seem to be leaving a great impression, at least not on me and a whole host of other people.
• 9:01
  SPEAKER_01: It's people like Sam Altman or Elon Musk and companies like Meta or Anthropic.
• 9:06
  SPEAKER_01: They all claim that their goal is to help AI benefit the world or be good for humanity.
• 9:11
  SPEAKER_01: Uh but meanwhile, they support legislation that would limit liability in case something goes wrong with AI.
• 9:17
  SPEAKER_01: They sell systems that will take away jobs and they build bunkers for themselves to survive a potential AI apocalypse.
• 9:24
  SPEAKER_01: So if AI is going to amplify all of that, it doesn't seem too great for the rest of us.
• 9:31
  SPEAKER_00: In your book, you point out a striking double standard.
• 9:35
  SPEAKER_00: Silicon Valley executives who spent their careers evangelizing technology for education often send their own children to electronic free schools.
• 9:45
  SPEAKER_00: What should students and parents understand about that gap?
• 9:49
  SPEAKER_01: I think we see that pattern over and over again.
• 9:52
  SPEAKER_01: Technologists say one thing about what their technology does, but in their private life they do something very different.
• 9:58
  SPEAKER_01: Um the that point that you made about technologists sending their kids to schools where there isn't a law of electronics, actually, there's a direct quote from Steve Jobs talking about what he would do with his children in schools.
• 10:11
  SPEAKER_01: But I don't think it's limited to him.
• 10:12
  SPEAKER_01: So many people who work in the technology industry, because of the wealth that they have, are able to send their kids to private schools.
• 10:20
  SPEAKER_01: And many of them, on the one hand, some of them want their children to learn about programming and so on, but many of those private schools have very strict policies about what kind of technology uh a student can bring to school or have out in a classroom.
• 10:33
  SPEAKER_01: Um these days, because technology is so cheap and ubiquitous, it's actually much harder to create an environment in which students don't have access to that technology and aren't using it in ways that make it hard to teach, right?
• 10:46
  SPEAKER_01: I'm at a university and many of my peers are now concerned that anything that they give as a take-home assignment is no longer done primarily by the student, but by the immense AI systems that they now have access to.
• 11:02
  SPEAKER_00: Our listeners are making decisions every day that AI systems are shaping.
• 11:07
  SPEAKER_00: What to apply for, what to study, who to follow.
• 11:10
  SPEAKER_00: What should the student understand about those systems?
• 11:13
  SPEAKER_00: They almost certainly don't right now.
• 11:17
  SPEAKER_01: Well, so there's a saying in Silicon Valley, which is that if the service is free, you are the product, right?
• 11:24
  SPEAKER_01: So most social media platforms sell your attention as a user to advertisers.
• 11:30
  SPEAKER_01: You know, we all think we're getting something out of it, and probably that's a little at least a little bit true.
• 11:35
  SPEAKER_01: But the people who are getting the most out of a social media platform are the advertisers who are willing to pay a large amount of money to get things in front of you.
• 11:44
  SPEAKER_01: They're the real paying customers.
• 11:46
  SPEAKER_01: Um but your attention is your time, and your time is your life.
• 11:53
  SPEAKER_01: Most of us want to live a long life, right?
• 11:55
  SPEAKER_01: Well, if you think about it, every moment that you spend is a bit of that life.
• 12:00
  SPEAKER_01: And so every time you watch a TikTok video or uh or read something on Facebook, you've basically donated a portion of your life so that a giant corporation can make more money.
• 12:11
  SPEAKER_01: And I think with AI, it will do all of that on steroids and possibly it will make us all dumber in the process.
• 12:18
  SPEAKER_01: So I actually think there's an update to that saying where if the AI guidance is free, your brain is the product.
• 12:26
  SPEAKER_01: And uh we're all being harvested in some sense.
• 12:31
  SPEAKER_00: You've built a research reputation that spans computer science, international development, education, and now AI ethics.
• 12:40
  SPEAKER_00: And your work has influenced everything from World Bank policy to Satea Nadella's thinking on education.
• 12:48
  SPEAKER_00: What does that career path that gets to work at that scale actually require of someone starting out?
• 12:56
  SPEAKER_01: Well, I certainly didn't set out to do those things at the beginning.
• 13:00
  SPEAKER_01: I did always want to be a research scientist, and so I went in a direction that I thought would uh allow me to do that.
• 13:07
  SPEAKER_01: And really, I think the I've been fortunate enough to be able to do jobs that I really uh love and want to do, and that's been true almost all my life.
• 13:17
  SPEAKER_01: And I would say the only other side of it is that if you take your work seriously and really put a lot into it, then it also often opens up opportunities for you to further be able to do things that you want.
• 13:30
  SPEAKER_01: So although my career has taken a number of twists and turns, at each point what I was doing was really passionately trying to do the thing that I love to do, and that often opened up doors that I couldn't have expected beforehand, but which have turned out to be really interesting and fun for me.
• 13:48
  SPEAKER_00: You wrote Geek Keresy well before the widespread use of generative AI as we know it today.
• 13:54
  SPEAKER_00: Does the law of amplification still apply to this new era of AI, or has the technology shifted your thinking?
• 14:03
  SPEAKER_01: Yeah, that's a great question.
• 14:05
  SPEAKER_01: I have kind of two answers to that.
• 14:07
  SPEAKER_01: The first is that I think for the most part, the law of amplification will still apply to AI.
• 14:14
  SPEAKER_01: And in the same way that I've been talking about how if you follow who has power over a technology, then it's those people's intentions and desires that often get enacted through the technology.
• 14:26
  SPEAKER_01: And I think a lot of that will still hold with AI.
• 14:29
  SPEAKER_01: Uh AI is an incredibly powerful technology, and therefore its amplification power is great, and it will amplify the desires and wants and needs of the people who ultimately have control over it.
• 14:42
  SPEAKER_01: So that will continue to happen.
• 14:44
  SPEAKER_01: On the other hand, I do think that AI is different from pretty much every previous technology in that we as human beings begin to treat it like other human beings.
• 14:58
  SPEAKER_01: And once that starts to happen, it means that AI can do things that previous technologies really couldn't, such as change our mind about things.
• 15:07
  SPEAKER_01: And increasingly there's research that shows that AI appears to be even more persuasive than regular people at changing our minds about some things.
• 15:15
  SPEAKER_01: And once a technology changes people's minds and what they want to do, I think that's no longer just amplification.
• 15:22
  SPEAKER_01: Something very, you know, uh at once intriguing but also scary is happening at the same time.
• 15:30
  SPEAKER_00: If we look closer at the mechanisms of implementation, historically social problems or packaged interventions fail because human bureaucracies or your ordinary American agencies execute them inconsistently.
• 15:46
  SPEAKER_00: If generative AI could automate that delivery with 100% consistency every single time, doesn't that bypass the human bottleneck and break the law of amplification?
• 15:58
  SPEAKER_01: Yeah, that's a great question, and it kind of follows on to the previous point.
• 16:02
  SPEAKER_01: I do think that there are ways in which AI could be used to uh improve the delivery of certain kinds of services, to enhance education, to make things better for people.
• 16:17
  SPEAKER_01: At the same time, I think, again, the question is who has power over these technologies and what are their interests?
• 16:24
  SPEAKER_01: One of the great challenges, I think, of education is that we've at this point seen decades of digital technology in educational systems.
• 16:33
  SPEAKER_01: And what's really frightening, actually, is that in over the last several years, possibly up to a decade or so, we're actually beginning to see as a society declines in educational achievement in our children.
• 16:45
  SPEAKER_01: And that, as far as the modern history goes, has never happened before.
• 16:49
  SPEAKER_01: It's hard, people can't necessarily um specify exactly why that's happening, but I actually think digital technology is a significant contributor to that phenomenon.
• 16:59
  SPEAKER_01: AI definitely can be used in a way to help people learn.
• 17:03
  SPEAKER_01: And one of the really interesting things that uh I'm finding in some of my research is that in some contexts, people are much happier to learn from AI than from human teachers.
• 17:13
  SPEAKER_01: We've seen situations where rural farmers and Ethiopia seem very happy to learn by using an AI system to do things that they could easily have asked a human expert about, but where they're either shy or embarrassed to ask and willing to ask an AI system.
• 17:30
  SPEAKER_01: Similarly, people seem to be willing to consult AI for healthcare information and knowledge that they might be too shy to ask a real person, right?
• 17:39
  SPEAKER_01: So a lot of these things are beginning to happen.
• 17:41
  SPEAKER_01: At the same time, if the AI isn't carefully monitored, if its use isn't carefully thought about in terms of pedagogy, then I think it's also very possible that it will do what our students are beginning to do right now, which is use AI to do their assignments, in which case they're not learning.
• 17:59
  SPEAKER_01: You know, assignments are just being submitted based on uh the computer doing most of the work.
• 18:07
  SPEAKER_00: What do you think?
• 18:08
  SPEAKER_00: Most students get wrong about working at the intersection of technology and social change.
• 18:14
  SPEAKER_01: I don't know about students, but I would say I think people in general have a general belief that better technology makes things better.
• 18:23
  SPEAKER_01: And a lot of us have seen that in our own lives, right?
• 18:26
  SPEAKER_01: There's a new, I don't know, dishwasher that ends up on the market and suddenly your dishes are washed better or faster or whatever it is.
• 18:34
  SPEAKER_01: And similarly with a lot of digital technologies, most of us would not be willing to give up our smartphones or social media, even if we're concerned about some of its negative impacts, because we think the positive outweighs the negative.
• 18:46
  SPEAKER_01: And that might be true in many contexts, but I actually think for the net impact, the jury is still out.
• 18:53
  SPEAKER_01: If it's really true that educational achievement is going down and is due to technology, is that really worth the trade-off we've been making?
• 19:00
  SPEAKER_01: If AI ends up making a small handful of people immensely rich and the rest of us are put out of a job, is that really worth whatever the benefit might be?
• 19:11
  SPEAKER_01: So I think these are questions that we have to continue to ask and we have to think beyond our personal experience with technology and what we want for ourselves and think about what its larger societal impact is.
• 19:24
  SPEAKER_00: For a student who wants to pursue this kind of work, whether in research, policy, or the technology sector itself, what's one concrete thing they can do in the next year to start?
• 19:36
  SPEAKER_01: First of all, I would say whatever it is that you find personally interesting, it's really important to keep following that and developing it.
• 19:45
  SPEAKER_01: The world rewards people who are experts at something, and you might as well be an expert at something that you find super interesting.
• 19:51
  SPEAKER_01: Otherwise, it will be a steep uphill educational process for you.
• 19:56
  SPEAKER_01: As with respect to kind of technology and policy and ethics, increasingly that is a major concern.
• 20:01
  SPEAKER_01: Uh, I'm at a department that's called a school of information, where these concerns are the main thing, or one of the main things that we teach our students, where our PhD students do research on.
• 20:12
  SPEAKER_01: There are other schools such as technology and policy or law and technology or centers where they study those kinds of phenomena.
• 20:22
  SPEAKER_01: And I think it's a reflection of where we are as a society that we're more and more concerned about what the technology is doing and wanting to regulate it so it minimizes the harm that is causing us.
• 20:35
  SPEAKER_00: What's one AI claim you keep hearing that you wish someone would pressure test publicly?
• 20:44
  SPEAKER_01: Yeah, so there are many claims made about AI in various directions.
• 20:51
  SPEAKER_01: So not so much pressure testing, but one of the challenges with AI is how easily we tend to think of it as something that's human-like.
• 21:03
  SPEAKER_01: But it's really not, at least not yet.
• 21:05
  SPEAKER_01: It is, in fact, very human-like, but it's not actually human.
• 21:09
  SPEAKER_01: And I mean that in a almost a technical way.
• 21:12
  SPEAKER_01: So, for example, most people have goals and motivations and desires, and most people have something like a reasonably coherent personality.
• 21:22
  SPEAKER_01: Uh and we're used to experiencing that from each other.
• 21:25
  SPEAKER_01: Um, AI seems like it has those things, and it responds in a way that sometimes suggests that it has those things, but in reality it's not, right?
• 21:35
  SPEAKER_01: You can have conversations with AI where over the course of the conversation, its personality and the way it responds to you fundamentally changes.
• 21:43
  SPEAKER_01: Right?
• 21:43
  SPEAKER_01: You can prompt AI to respond in a particular way or to have a particular attitude, which usually with people you really can't.
• 21:51
  SPEAKER_01: You can ask your friends to be more empathetic, but whether they do that or not is really up to them.
• 21:56
  SPEAKER_01: Meanwhile, AI systems, at least at this point, don't yet have their own clear motivations and goals.
• 22:02
  SPEAKER_01: Human beings do.
• 22:04
  SPEAKER_01: But it's because human beings tend to anthropomorphize so readily, we assume that an AI system has those things.
• 22:11
  SPEAKER_01: We assume that an AI chatbot has those things.
• 22:14
  SPEAKER_01: And I think that's super dangerous because our mental model for what the AI system is and what it does and what it can do is defined by our experience with human beings when actually it's a very different kind of intelligence.
• 22:30
  SPEAKER_00: What's the hardest thing to get technologists, people who build these systems for a living, to accept about what their products actually do?
• 22:41
  SPEAKER_01: Yeah, in the early part of the 20th century, there was a writer and journalist and kind of activist whose name was Upton Sinclair, and he had this very well-known saying that I think is particularly uh appropriate, and it goes something like, It's difficult to get a man to understand something when his livelihood depends on him not understanding it.
• 23:02
  SPEAKER_01: Uh and I think that's exactly the situation that many people who work for the technology industry are in, which is I do think many of them are increasingly concerned about the future of technology and AI and what it might do for human civilization.
• 23:16
  SPEAKER_01: But this is how they earn money.
• 23:19
  SPEAKER_01: And it's not so easy for them to quit their jobs, which I can perfectly understand.
• 23:23
  SPEAKER_01: I think it would be great if people who are in those positions amongst themselves and work a little bit harder to push back on what they see are the worst tendencies of their own companies and organizations.
• 23:37
  SPEAKER_00: Finally, are there any books or podcasts you'd recommend for a student who wants to go deeper?
• 23:43
  SPEAKER_01: That's a great question.
• 23:44
  SPEAKER_01: There are a series of books that I read, like by uh Evgeny Morozov, that go into you know why technology doesn't save the world or what expectations we had with technology and democracy and how those didn't pan out.
• 24:00
  SPEAKER_01: So increasingly, there are books about how technology ends up causing problems, which I think and all of those are generally worth reading.
• 24:11
  SPEAKER_01: I don't necessarily agree with every detail, but I think the more we understand about technology and how our individual experience of it is not necessarily the sum total of its impact, the better.
• 24:23
  SPEAKER_00: Well, thank you so much for joining us today.
• 24:26
  SPEAKER_01: Absolutely, my pleasure.
• 24:28
  SPEAKER_01: Thank you.
• 24:34
  SPEAKER_00: This has been Cause of Curiosity in an AI world.
• 24:38
  SPEAKER_00: Exploring careers through the lens of AI.
• 24:41
  SPEAKER_00: Until then, stay curious.