Latest / Autonomy Insiders / Designing Driverless Cars for Everyone (Not Just Early Adopters)
Transcript
- Daniel Abreu Marques: Today with Clare Mutzenich Welcome to Autonomy Insiders, the show where global industry leaders in autonomous driving unpack their real-world insights. welcome to Autonomy Insiders. I'm your host, Daniel, and my guest today studies the part of autonomous driving that almost nobody puts on a keynote slide. the sensors, not the safety case, the human sitting in the back of an empty robo taxi wondering who is in charge when something goes wrong. So Dr. Clare Mutzenich is professor of human and AI interaction at Loughborough University and founder of Anthrometric, ⁓ an human factors and behavioral science research consultancy, working where human factors, public policy, and AI meet. She has spent a decade on self-driving safety, remote operations, and how real people actually behave inside these vehicles. And we are recording this as the UK prepares for its first driverless taxi services, so the timing matters. And Clare I'm so happy to have you on the show today, and welcome to Autonomy Insiders. Clare: Thank you for having me, so delighted to be here. Daniel Abreu Marques: Perfect, let's start. maybe let's go wide and then go deeps. ⁓ I want to begin with the problem you think the industry is underpricing. So you argue that without a human driver, the system has to take a far more active role in supporting passengers. ⁓ what does the vehicle actually owe a passenger that companies are not building today? Clare: So I think there is a lot of work. building it, but perhaps the services aren't ready yet to provide a service that fits every passenger. They're working on elements of a service that the technology and says the technology is safe and the technology works ⁓ necessarily underpinning that with messaging about the service itself and how it will work for you as an individual user. ⁓ that covers a range of different approaches. We talk a lot about accessibility and inclusivity and I'm sure we're going to cover a lot of that today. But when we talk about an accessible service and supporting passengers, need it to feel like it's seamless. Every needs to feel that that service has been designed for them, not that they have to adapt in order to use the service or choose not to use it at all. Daniel Abreu Marques: And most coverage of this industry is about the driving itself and your work is basically about everything else inside a cabin. So ⁓ why is in cabin human experience the risk operators are least prepared for? Clare: So again, it's not necessarily not trying to prepare for it, but a lot of work is being done in this space. But human-centered design is key here, because a service can be well-designed in a technical way, but falls short for human experience. So by that we mean whether people feel safe. And even if the technology is safe, perception of that safety for each individual user is going to be key. Daniel Abreu Marques: And people generally like trust in A Vs as if it is maybe one thing, but maybe you could also break this ⁓ apart for us. So what are we really asking people to trust and at which exact moments does this trust also get tested? Clare: Yeah, it's a really good question because trust is going to be key. mean, trust has so many different layers. I mean, people trust brands, they trust the government, they trust that the regulatory process is going to be thorough and rigorous. They don't have to know what that entails. But in my research, a lot of people are telling us they already trust that the service or the technology will be safe. But what they need trust is going to be key. trust has so many different layers. people trust brands, they trust the government, they trust that the regulatory process is going to be thorough and rigorous. But in my research, a lot of people are telling us they already trust that the service or the technology will be safe. But what they need is if something goes wrong, they need to trust that there's going to be something that steps in, like a process or a protocol ⁓ going to support them. is if something goes wrong, they need to trust that there's going to be something that steps in, like a process or a protocol that is going to support them. But over trust is also another feature. not initially, but after services deployed, ⁓ people going to get very used to them quite quickly. And that's when over trust steps in and complacency. So people might start to build models where they think that they understand the AI, they understand what going to do in different situations but that's going to be based on own experience or what they've heard about the system rather than ⁓ know they won't be reading the safety case that an SD puts out so trust in other people as well is going to be a big factor because when you've taken out the driver there's going to be an element of interacting with passengers and other people that you're going to have to trust them as well. And that's really about traveling and sharing the vehicle with other people. ⁓ lots of layers to what we mean about trust in these types of new Daniel Abreu Marques: Also in general, the product driverless transportation evolves over time. the first ten thousand rides and the first ten million rides are basically different products to passenger. And what changes in the human experience as also the deployment scales. Clare: Yeah, I mean, we hear these numbers a lot, don't we? About 10,000 or 10 million or 10 billion. And to a person, anything above three doesn't really mean anything. It's like, we're not very good at calculating or holding really massive numbers in our heads. So you don't think I've done three million trips in my passenger car. I've driven, know, three million miles myself since I passed my test. Like nobody thinks in those kinds of values. we think about is ⁓ did the best feel like? The best experience I had in a self-driving car and what was the worst? ⁓ we don't even think just in terms of our own direct experience. ⁓ take on a lot of indirect experience as well. We listen to other people, our friends, our family. ⁓ listen to the media. So our own direct experience is shaping our ideas and our beliefs about self-driving cars, but it's also socially constructed by society and what they're saying about self-driving cars. cars. so when things go wrong, or a service surprises and delights us, ⁓ those go into our model of what we think of when we think about self driving cars. ⁓ And in order to actually, you know, ⁓ scale build a product to a passenger, you're to have to take the rough with the smooth. ⁓ And a lot of services that are kind of going first, and maybe putting their head above the parapet slightly, ⁓ they're to get a lot of criticism, and they're going to be the of poster child for self-driving cars and people will build their expectations and their judgments based on what they hear probably far faster and you before they have any experience of a self-driving car themselves and it's like ⁓ I this a sort of metaphor the other day, but it's like everyone has an opinion on horses. people love horses. Some people like I ride horses all the time, horses are my jam. ⁓ people like I would never get on a horse ever. And that's based on ⁓ any experience you had with a horse. It's just all shaped by your perception of what that means and what being a horse rider is or being ⁓ a person or whatever. you know, the point that we don't have to have experience of self-driving cars to pretty much lose people before they've even stepped foot inside one. Daniel Abreu Marques: Absolutely makes sense. And let's stay also a bit on the on the first years of deployment. and the first riders will also be more like novelty seekers, tech enthusiasts, because this is a new technology. And how does designing ⁓ for them ⁓ maybe mislead companies what the mainstream will actually need? Clare: Yeah, so I mean, most of the people who use the service first, like you say, they're really keen, they kind of want to be involved, and they're probably more forgiving of the system. They're maybe not picking it apart mentally because they want to believe in the technology. And that might give operators a false confidence that when they scale, when it becomes more widely sort of available, and also when people are paying for the service, that's when that intolerance to mistakes is going to come to the fore. So, you know, we know that human drivers make errors and yet will happily get in a taxi with a driver who's human and might make mistakes or drive in a particular way. They might also be extremely well trained and they may be a very good driver. Essentially, any kind of design that designs to a default user or these first types of users is actually going to mean that you're designing to an outlier. They don't really exist or they only exist at the edge. And then at the other extreme, at the other edge, might have users that also want defaults, but they are, they need more support. They need to be given a service that actually doesn't fit the mainstream so-called norm. And so we need to design across generations, across a multiple type of user. And we can't retrofit that in once the system goes live, it needs to be part of the design and it needs to be co-designed as well. So that means including those types of users from both sides of the outlier and also the default user so that you design a service that works for everyone. You know, I say quite often in my research, if it doesn't work for everyone, it doesn't work. Daniel Abreu Marques: and looking at the assumptions make what is in your the most common wrong assumption operators make when they think about how a first time rider behaves in an ⁓ empty robo taxi? Clare: I hear a lot is that people say that after the first novelty wears off, know, when they go in and then they kind of get quite excited, it's a self-driving taxi and then they kind of forget. And that's true, I've had that experience myself of being in different types of ⁓ vehicles. ⁓ do get to a point where you're having a conversation with someone and you can completely sort of stop noticing that it's being driven by a machine. But does happen then is suddenly there's like an uncanny valley. Do you know what I mean by that? Where it's something suddenly doesn't seem right and humans are very attuned to that. We're really instinctive creatures and that could be something like maybe, you know, the system suddenly slows down and feels jerky or it could be that it's actually too smooth. It doesn't feel quite right. Like you can sense that it's not being driven by a human because obviously we have, you know, different idiosyncrasies. we might even start to get to know idiosyncrasies of different providers. So, you know, maybe one type of brand drives in a certain way and another drives in another way. They might actually almost become ⁓ or characters of that service. ⁓ essentially something will break ⁓ moment where you don't really feel like you're in a self-driving taxi, there'll be suddenly something that you notice. ⁓ it could be the interaction with the voice assistant that maybe it feels fake or cultural maybe something just jars with a service that's coming from a different market ⁓ and quite work in your market. ⁓ So are all things in which... as humans were so subtly attuned to these cues what a robotaxi is going to be and whether it is a machine or a system ⁓ or it feels like something that's quite, you know, sort of natural and instinctive to use. And ⁓ at moment where something unexpected happens, doesn't even have to be like a real emergency, but definitely if there is an emergency, ⁓ that's the at which everyone's going to realise they're in a self-driving taxi and there's no driver and then what happens next. Daniel Abreu Marques: And as you may ⁓ mentioned also the driver did a lot of ⁓ other work, ⁓ maybe it emotional or social work, not just the steering of the car itself. So how much of the driver's job was that and and who picks it up now? Clare: Yeah, so I'm doing a project at the moment in Cambridge called the Connector, which is a self-driving bus service. So that's quite different from, you know, a taxi or self-driving taxi where it's private hire and this is a shared transport experience. And so we've interviewed many sort of users of the service and we've exposed them to a simulated service through VR and people speak very fondly of the driver in a whole host of different ways. roles, but rarely the driving. It's not really something else that they're doing. So that could be of conflict. It could be helping people onto the bus. It could be ⁓ information. For some people, it's actually the social interaction of greeting the driver when they get on the bus. So, you that social interaction from, know, loneliness is something which is becoming a real sort of social ⁓ issue. in our current world. And so for some people that might be the only human interaction they have in a day and that's what they feel the loss of, that role of that person, that human interaction. so they value the humor interaction and so by taking out the driver we need to ⁓ a way to kind of replace that with something that feels natural, feels ⁓ like gaining something, opportunity or some other way of a service ⁓ it's certainly not just one job or one role, it's Daniel Abreu Marques: And you also did the work. ⁓ you mentioned you can also ⁓ ⁓ give some more context you actually did there. But ⁓ one thing you looked at was also which non-driving activities are safe in a self-driving car. And ⁓ maybe tell us ⁓ what did you there and why does it matter also beyond safety for the product itself. Clare: Yeah, so this is another ⁓ separate study. So the Cambridge Connector is level four, it's a self-driving bus, whereas the work that I did for the UK government on the ⁓ role of the user in charge of a level three car. So this is a system which can do some of the driving for you some of the time, ⁓ but you as a in charge need to sit at the driving position ⁓ and you need to ready to take control if you're asked. So the important key detail is that you don't have to monitor the road while the car is in self-driving, but you must take control if you're asked. And so we were looking at what kinds of activities people might do during the periods that they're not. required to monitor the road. So it could be eating or drinking, could be doing sort of activities like a word search or reading, it could be watching movies, it could be playing on your phone. And we also looked at whether or not doing some of these activities in a tablet rather than using a handheld mobile phone, whether those had an impact on your ability to be able to take that control. Kind of like are some so immersive that breaking that activity and having to take control is too challenging. And we also wanted to understand how people did it, like how do they interact with the object? How do they, you know, do they put it away? Do they on watching the movie while they're driving? And I want to point out we did this in a driving simulator, so we did not put any of these people at risk and we didn't put them on the road. But did ask people to take back control and we them understand how to do that, we showed them a video, we sort of tried to train them a little bit and it was in two phases. In the first phase we didn't give them any information, we just said take control as soon as it's safe to do so and what people did was they took control immediately so they dropped the activity and kind take control and start driving ⁓ but they have any situational awareness and by this we mean did they really understand what was happening on the road? They've not been paying attention for like 10 minutes. have they looked in their mirrors? I I would think that was one of the primary things you should do before you take control of a car going 60 miles an hour on the motorway. that people were so focused on that kind of, ⁓ I must take control, I must assume this driving role, they didn't take any of those situational awareness checks in order to do so safely. So in the next phase, we focused on the mobile phone task, which we found had most impact in phase one. and they were either watching a film or they were doing admin activity like booking a holiday or buying house and we found if we showed them a video that kind of told them you know put it away carefully in your mirrors and then take control. This did have some impact. It helped people. ⁓ know, they all said that they were trying to follow the instructions. But you when we mobile phones, when were watching the movie, they weren't pausing movie. And so they were driving and they were still watching the film and sort of swerving all over the place. ⁓ tells us is this user in charge role when you have a level 3 aux vehicle and you're expected to take back control need to be assisted you need to be guided into what are the right things to do ⁓ and that a lot of the handover or takeover from these types of vehicles are going to need sort of support for the user in charge to help them understand what their responsibilities are and to do take control safely. Daniel Abreu Marques: Yeah. I think that's also ⁓ probably one of the reasons a lot of those level three are already out of service for many OEMs, for example, because ⁓ you this feature ⁓ with the promise that it's ⁓ more or less autonomous that you can relax and you can do something different, but you're still stressed in your head because you know what's happening when I need to take over the control. So Clare: you Yeah. Yeah, yeah. And people quite reasonably want to know, what can I do? Like, it's really difficult for regulators to arbitrate or mandate which of activity, you know, like if you're going to say, ⁓ you can watch a ⁓ movie on your phone, but you can't write an email, you know, who's going to police that? It's going to be really challenging. ⁓ you know, lawmakers and regulators, ⁓ need to have empirical evidence, you know, really robust evidence of what is the impact, how much longer does it take you to take control? How few glances in mirrors do you make with this task as opposed to it being in that task? And we've definitely found that the task was watching movie or doing admin in the tablet in the HMI, because that automatically cuts out ⁓ when get a takeover request, you have no option. You can't carry on with the activity. people taking back control. in a safer way, but it still had an impact because cognitively you might still be engaged in that task. It takes a little while for you to stop, you know, like mentally thinking about it and people were still like hitting the screen and there was swearing, like lots of people got really angry, because we designed a very poor website on purpose that required them to do a lot of ⁓ and scrolling back up and you know we did not optimise the experience. Daniel Abreu Marques: Mm. Clare: ⁓ design put it that way. ⁓ Daniel Abreu Marques: Makes sense. ⁓ also coming back to your bus use case. so shared ⁓ autonomous vehicles also ⁓ put stranger than a in a moving box with no authority figure, more or less. So the driver was also kind of a referee. So you have someone who is looking what you're doing. what happens to to the social norms that referee ⁓ disappears? Clare: We had some really interesting findings from a recent workshop that we did that people suggested of ripping up the script ⁓ what been already, you know, what a lot of people in the industry have been working along ⁓ this model for buses that ⁓ there's lot of rural... kind of routes that don't have a lot of buses on them or that buses don't run at certain times of the night, though people need to get home or it's dark, that potentially ⁓ might have issues let's say if you're a single woman waiting in a bus stop and ⁓ a bus pulls up and it's only got two men on it, ⁓ you a woman has to make that decision, do I climb on the bus with these two men that I don't know and no referee, as you say, no one to kind of be the authority figure and kind of the face of the ⁓ or I stay on a dark road at a bus stop that you know and I've missed my chance of getting home. These are conversations that come up a lot and you know operators and local authorities are really ⁓ you know engaged in this discussion but in our workshops people suggested that actually why are we putting driverless services on at that time, when instead we can run driverless services during the day, when there's lots of other people around to have that social contract, like you say, the social norms, and instead put the drivers on the times of day where people are really unpredictable and might behave badly ⁓ they're drunk or because it's dark and they think they can get away with it, etc. So we're this co-creation of thinking about what a driverless service looks like and working with passengers and ⁓ members of communities and the public in different social groups. We can actually start to design a service that's based on that co-creation and they're giving ideas that's not about the technology, it's about the service. Daniel Abreu Marques: And ⁓ maybe also this this vacuum of authority maybe is the most dangerous when an emergency happens. And ⁓ you also ⁓ looked at this kind of thing at your ⁓ study. You put people through that. ⁓ So ⁓ I think put ninety-one people into emergencies in a virtual reality setting, so like fire, flooding, collision. ⁓ So can you tell us bit more about that and also what surprised you most about how people actually behave? Clare: Yeah. Yeah. Yeah, so this is as far as I'm aware, the biggest VR study, study into this of driving technology. We not got 91 participants, but they were actually intersected by different protected characteristics. ⁓ So we people from all religions and ⁓ racial groups. different sexualities. We different age groups. had ⁓ pregnant people and transgender people and a variety of different types of protected characteristics all represented and intersected. obviously you're not just a male, you're a certain age, you're a certain ethnicity and so on. And so what this did is it allowed us to really explore whether any of these protected characteristics have an impact if there's an emergency like a fire or a flood or someone's trying to get into your taxi or taxi stops in the wrong place, know, all these different emergencies to really understand what people think the system needs to do for them and what can they do themselves, what are they capable of doing themselves and what really surprised us is that everybody expected a human to be in the loop. what really surprised is that everybody expected a human to be in the loop. everybody, they said, I want a button, I want to press a button, I want to speak to a human. everybody, they said, I want a button, I want to press a button, I want to speak to a human. ⁓ they called it a variety of things, a remote assistant or a conductor, or, ⁓ know, different kinds of ⁓ for what this person would do or what they were. ⁓ essentially, they're, they're just a human. And what's interesting is that a lot of media reports, particularly at the moment, have painted this remote assistance or the need for remote operation ⁓ as failure of the system, like the automation doesn't work, like it can't do it and so it needs to fall back on a human. But in actual fact people certainly shown from our research and simulation is pretty realistic, you know people even in the fire they were like ⁓ can we get out of this because I'm starting to feel really hot, even though it's just a simulated fire in a robot taxi. we really ⁓ was that People are unlikely to use the model, the AV service, they know there is a human in the loop. That is going to be the trust mode. So people will accept that it's being driven by a machine, ⁓ there are some parts to the system or some parts to the service where they just want to talk to another person. Daniel Abreu Marques: and you also deliberately included children, I think age eight to And what did these kids do that ⁓ adults did not do? And ⁓ also what does that force designers to change, maybe? Clare: Yeah. Yeah, well, mean, children aren't little adults. know, they are very distinct as a group. there's going to be a kind of a use case for driverless cars, driverless taxis, transporting kids to school or from practice, et cetera. ⁓ their size and the fact that they have undeveloped cognitive capabilities actually means that some features are really important to be considered ⁓ in of whether they're intuitive. So our youngest participant, who was eight, she talked a lot. She was very ⁓ eloquent and we all fell in love with her, she was such a sweetie. And she was saying, my daddy will tell me what to do if there was a fire. I would need someone to play that role, to be adult who's going to tell me what to do. they would normally look to that adult. And so in a driverless taxi, if there's any kind of emergency, they might not be able to read. They might not be able to see, you know, understand what things the cab that may be being instructions, they necessarily have a protocol ⁓ in minds yet, because their experience hasn't reached that point where they know what to do in these situations. And this for people that maybe have serious cognitive impairments. So they may be adults, but they have cognitive functioning difficulties. So we need to design a service that is intuitive and familiar that you almost don't need to think what to do if there's an emergency. so and regulators have that responsibility to consider these types of users as part of the default so that the service, those human design principles affected into every type of safety case. Daniel Abreu Marques: and in an emergency when is no driver, so who is actually in charge and how also does the vehicle have to establish the authority and fast enough to to be believed? Clare: in the VR we had some ⁓ rides and we had some shared rides and the shared rides, they be with people you knew or they could be with strangers. And we saw a lot of in the shared roles, people assumed very quickly. So in shared rides, know, often men assume some kind of authority role and ⁓ advise others on should be done. And, but quite often they're... experience particularly for white middle class older men was that it wasn't an emergency it was fine and that you know there was a sort of trust in the system and the technology. When it was a group of shared riders who perhaps were all female or were all homosexual everything was slightly different in terms of ⁓ where felt the risks were and that often contextual so it could be that you know depending where they were or what the situation was. So one of the scenarios was a pedestrian trying to get into your car and so in that respect they... wanted, know, unambiguous authority, they wanted the button again, ⁓ get in touch with someone who had some kind of ⁓ So that could be emergency services, it could be the police. So button is a design feature. ⁓ It to a human, but it connects to a multiple of roles of different humans, it's not going to be one person playing all of these roles. And so that needs to be a design feature, like what button for emergency, what button for information, which is the button for support or emotional But multimodal messaging was really important. So some people wanted to be able to get out, some people wanted to lock themselves in. was really different depending on the protected characteristics of the users. Daniel Abreu Marques: And from from that what we had now talked about, I feel like there is some kind of tension. So if you have too little intervention, the the people feel abandoned and ⁓ if you have too much, the car is like not driving itself as you said, or something like this. So ⁓ how you find that that perfect line? Clare: I mean, perfection is the goal perhaps, but yeah, I think the challenge is how do you, as you say, how do you move away from that tension? What we found was that people were really willing to share data about themselves if that meant that they could have that layer of personalized service. So they would be happy to talk about, you if they had neurodivergence, they wanted an app where they could include that kind of information. And on the basis that most of these services will be app based to kind of hail. wanted the in particularly in an emergency for the operator or handler to know some features about them. ⁓ if they were in a wheelchair. they wanted them to understand that without having to explain it all so that then the person who was actually kind of triaged to help them would be the right person so it would have that kind of seamless experience the person who comes to your help knows the help you need and that to many people would have been able to orchestrate that tension to it being a personalised service, but they were willing to share that information order to get it. Daniel Abreu Marques: And and as you mentioned, your study also went to recruit people that are sometimes maybe ⁓ forgotten ⁓ from the ⁓ you recruited disabled, no neurodivergent and also gender and faith diverse riders. ⁓ and for sure there is a moral ⁓ part of this, but can you also make the commercial case for why a robo taxi company should care on day one? Clare: I think easily, mean, it is the commercial case, isn't it? If you've got all of these different users and you haven't designed a service that they can use, then there is no business model. you're just doing a series of pilots and they're basically just demos. There is no commercial case for a widely used service. If everyone who's using it is technically literate, able-bodied, know, traveling alone in ideal conditions, know, none of that is the reality. Life is messy and edge cases are going to come up on a daily hourly basis. If you can't people where they're at, if you haven't included them in that design, then you think ⁓ you're taking a moral case when you kind of like go after them later. ⁓ ⁓ you just haven't designed a service that anybody's going to use. And ultimately when you try and scale, you're going to find that it doesn't matter how many millions of miles you've traveled safely, people won't use it because they'll just go, well, I can't because of X and you never asked. Daniel Abreu Marques: And ⁓ you also mentioned that accessibility ⁓ if your interface is an AI you need to talk to. So what does then ⁓ accessibility really mean in in this kind of era where the interface is an AI? Clare: So it's got to be multimodal. It's got to be an interface that is, as I say, intuitive. design is often really, really ⁓ ⁓ don't need to design everything from the ground up. ⁓ There have been recent cases where particular design of the vehicle in order to ⁓ open the it was the same kind of push button that you get on a train. And that's that of ⁓ familiar, ⁓ I how to use that. I don't even need to think about it. just, yeah, that's how I get in. That's how I get out. ⁓ It's built on model that already works. ⁓ It's got a of ⁓ tactile features that for different of users. So we should be actually leaning on what we've done well in the past rather than constantly trying to create something new just because it's a new technology. those are the kind of intuitive human design features that are to be more successful for different types of accessibility users. ⁓ But the same applies for what currently isn't working for some users in the transport system, ⁓ and let's design away from ⁓ So there's no point putting practice ⁓ features that we know don't work for some users and we should be definitely trying to focus on those ones to improve. Daniel Abreu Marques: and do you see maybe also a real risk that this inclusive A V design becomes more a compliance checkbox? or how do tell genuine practice from theater, so to speak? Clare: Yeah, think, I not believe anyone is doing a checkbox design. think ⁓ ⁓ the people that I speak to are doing this kind of service design, operators, local authorities, ⁓ know, the OEMs, they really do care and they want the service to work. are they including those seldom heard voices from the start? And that's where I think the disconnect is that they talk about a service that would work for everybody. But on the connector project, the real difference is that every single part of that ecosystem is involved in the human factors work. So the local authority, the operator, the OEM, the software provider. When we did sensory walk alongs with different types of users, people with mobility impairments, neurodivergent, pregnant women, who had complex cognitive needs and hearing impairments, they all sitting on the back of the bus, along with for the ride, ⁓ because they are ⁓ involved this element of how do we make that service work? What are those real factors for these real people? ⁓ And so how you make it not theatre. You actually get your hands ⁓ dirty, you involved, you listen, you are questions, not just have a consultancy deliver that information in a report that pretty much says the same thing that all the other reports say about, a big surprise the transport system doesn't work for disabled people you need a ramp. You know don't need a ramp, not everyone who's ⁓ impaired needs a ramp. And not everyone who's got a disability is in a wheelchair. And not every wheelchair user is the same as another wheelchair user. So instead, if we actually co-create, if we include these types of people in our service then at least get an understanding perhaps we can design for this user, but we can't design for this one. And that's important. might not be able to be a service that works for everybody. And that has to be something that's considered because then at least you can design the gap. You you recognize what self-driving vehicles can be and then what they can't be, you find another way to service that particular user group. Daniel Abreu Marques: And you think there is ⁓ specific group ⁓ of people ⁓ which the current industry is most blind to right now? Clare: Yeah, I think idea of QR codes, ⁓ a lot of people assume that everybody's digitally, you sort of included ⁓ and of with visual impairments can't ⁓ use QR because maybe they don't have smartphones and they, you know, they need screen readers. ⁓ So the of putting information on a QR code is actually a barrier for lots of users. And if it emergency information, on a barcode, like a QR code, then that's going to be even worse. Daniel Abreu Marques: And we already touched on a topic ⁓ which I want to ⁓ go a little bit deeper into. And this is the the part of remote operator. So before AVs talk to passengers well, there's still a human in the loop. And some people also want the human in the loop. ⁓ and I think the public barely understands what the the role of this human ⁓ operator is. and your earlier research was also on those kind of people. So Your earlier work was on the situation awareness of remote operators of AVs the also quietly leans on teleoperations. So you have this remote operators. So what does the public misunderstand about the human that is still in the loop right now? Clare: So I think I said before, assumption is that that signals ⁓ failure ⁓ the automation or that the AI doesn't work or that the human sort of had to step in and that's a failure. ⁓ should be sold as the safety case, that layer, that the technology can be 99.9999 % ⁓ there edge cases are still going to occur. And it's soft skills that a human can bring. the remote assistant is actually a trust mode. Daniel Abreu Marques: Yeah. And when you think about remote operator services, there's also oftentimes ⁓ that one supervisor has several vehicles at once. So you probably also heard, for example, ⁓ ratios from one to thirty or something like this. So what is the realistic limit before situation awareness breaks down and is the industry honest about that limit, in your opinion? Clare: ⁓ Yeah, I mean this is where we lean back on cognitive psychology and what we know about human mental capacity and cognitive load and like you say, situational awareness. by colleagues at Coventry University have empirically measured the ⁓ of like a operator having to monitor all these different kinds of vehicles. ⁓ And unsurprisingly, correlates with what we know about human attention, that we tend be able to ⁓ focus on about things, maybe, a few more, a few less. And this is what they found that, you know, it's about five. And once you've got five you're monitoring, any extras, you start to kind of deplete that attention ⁓ of capacity. ⁓ also leads to lots of things like you have what we call misplaced salience. So you might think you understand what the problem is, but actually you've missed something else ⁓ or ⁓ way which we ⁓ interface for remote operators. I've seen all manner of human factors, horrors in the interface design of remote operation stations, things the wrong color, multiple screens you have to search through to get the right information, ⁓ or helpfully all the information on one screen so you've got all these different little codes and all these different symbols and different colors. careful human factors research on the remote operator, remote assistant role, in order to really understand the different use cases, what kinds of interface we need, and for different awareness, common how we can ⁓ mitigate we can protect ⁓ and we can actually standardize ⁓ the in which remote assistants are receiving And the sooner we do that work, the more human factors work into the interfaces and shift work and workload and so on, the better. Daniel Abreu Marques: And ⁓ there's also oftentimes the the narrative that Europe is slower than the US and China on deployment. and do you this is purely a disadvantage or could this maybe also be a chance to get all the things we talked about, the human side right before ⁓ the scale locks the mistakes in? Clare: Yeah, I think it is a hotly argued debate and There's a lot of good points on both sides. Europe is being cautious and simulation perhaps can only take you so far. But the UK itself is a leader in AI and self-driving. And the UK Automated Vehicles Act 2024 is an outstanding piece of legislation for really understanding the legal as well as the engineering requirements for making sure that an AI or an AV driver is as safe ⁓ as careful competent human driver. And so this ⁓ really how we to the time to understand what that means and how we can satisfy a safety case ⁓ to a regulator that is a case as well. isn't just we've done it, we've got it right as people adapt, as people ⁓ get used to living with self-driving vehicles, that safety case may need to change. And so we have legislation in place now that can update and can continually assess safety case. ⁓ so being slower is only a disadvantage if we use the time badly. And if we rush, we might find that we have to do a lot of costly retrofitting afterwards. the... the kind of potential for reputational damage, like the loss of trust, if we go too fast and do it wrong, that's really what will put us back. And so in order to build a system that works and works for everybody, taking this slightly cautious approach, in my opinion, is the right one. Daniel Abreu Marques: And you build ⁓ ⁓ anthrometric exactly to advise across government, academia and industry. so where in the product cycle does human factors work usually ⁓ arrive too late in your opinion? And who is also maybe getting the timing right? Clare: Yeah, I think human factors fell out of trend slightly. I don't know at what point, but it's back and it's serious and it's got to be done at the start. It too often is arriving kind of after the vehicle design or the interface or the technology. know, lot of investors, you know, want to see the technology works. They want that R &D and then they start thinking about the service model. And that's the wrong way around. You know, as we said before, there is no commercial case if your users won't use it, if you've gone so far down the technology and kind of form factor approach then you've got a that nobody wants to use, that is a mistake. Human factors ⁓ is of the commercial glue that is going to ⁓ make business case and bring the users in. Daniel Abreu Marques: one last question to to round it up. So from everything you have described so who in this industry should be taking this most seriously and also maybe where would you want your next study to land? Clare: So I think. the service design and the passenger experience and safety is going to be key. think certainly something that has come across in sort of workshops that we've done recently and which I would definitely want to be the next study is the kind of role of women's safety. So I think there's a lot being said and I've repeated about the button and who the button gets in touch with. And I think this is a primary concern for lots of people within the ecosystem, but whether they're really hearing women what their concerns are ⁓ whether they're saying, we'll, there'll be a button and we'll monitor it. And women are saying to us, how are they monitoring it? What does look like? So for example, it just ⁓ a kind of camera? How do I know that's on? How do I know it's even working? A lot of the time in public services, you know, things are broken. do I know ⁓ the blind spots are? So ⁓ people may choose ⁓ vehicles because they can target women on their own. So how do women defend themselves? And yes, there may be some. safety element that you know you can call someone and they will come and assist you but you're still in that situation. You still have to manage and negotiate that horrible experience that's really ⁓ we want to start designing with safety in mind. Recruit women, make them part of that service design ⁓ safety so that they can actually give that feedback and feed that forward. one of the things that we were told in one of our recent workshop ⁓ that women wanted to have a camera that showed what being seen at the remote control centre. so that they could actually understand exactly what the view of the bus was. So they could make choices about where they sat to make sure that they were being seen and that that would be a deterrent. So it's that joined up piece of research that I'm most interested in starting to really listen to how we design a service that's driverless, maybe flipping the script as we said before about why make that a driverless nighttime service, why not that during the day but it doesn't mean that nothing's going to happen during the day. But really bring these types of users into the conversation and help them help us to shape that service design so that that perception of safety there not just whether people are safe they need to feel safe. Daniel Abreu Marques: That sounds super interesting, Clare. And ⁓ want to thank you for for taking the time today and shed some light on a topic that is not on on everyone's mind and when thinking about autonomous driving an autonomous vehicle. So it was super insightful a conversation. I think we covered a lot of ground today. happy to had you on today. And thank you so much for taking the time, Clare. Clare: ⁓ so much for having me on.