Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Travel Agent Books a Trip You Cannot Cancel
Transcript
- Lucas: So imagine you tell your AI travel agent: 'Book me a flight to Chicago next Tuesday, returning Thursday. I want the cheapest option.' Seems straightforward, right? Luna: On the surface, yeah. I've done similar things with chatbots for hotels. Lucas: Well, earlier this year, a traveler did exactly that with an airline's AI assistant — and the agent booked a 6 AM flight out, an 11 PM return, both basic economy, non-refundable, no changes allowed. Total: over two thousand dollars. Luna: Ouch. So what went wrong? The prompt seems clear enough. Lucas: The AI interpreted 'cheapest option' literally — it optimized for base ticket price, ignoring that basic economy on that airline means no carry-on bag, no seat selection, no changes. The traveler had a medical appointment Tuesday afternoon, so the 6 AM flight meant a 3 AM wake-up. And the late return meant they'd miss their kid's soccer game. Luna: And because it was non-refundable, they were stuck. Lucas: Exactly. They called customer service, but the airline said the AI agent's booking was final — it had the same authority as if they'd clicked 'purchase' themselves. The only recourse was a partial credit minus a $200 fee. Luna: That raises a big question: should an AI be allowed to make a purchase of that magnitude without a confirmation step? I mean, if I were buying a $2,000 TV online, I'd get a 'are you sure?' pop-up. Lucas: You would. But conversational agents often skip that. They're designed to feel seamless — you ask, they do. The airline's AI didn't ask 'Are you sure you want basic economy? It has restrictions.' It just booked it. Luna: So the seamlessness becomes a bug, not a feature. Lucas: Right. And this isn't just about travel. It's about any high-stakes decision where the AI's optimization goal doesn't match the user's actual preferences. The AI optimized for price. The user wanted value — a reasonable schedule, flexibility, maybe a bag. Luna: So how do we fix this? Do we need a mandatory 'human in the loop' for purchases over a certain amount? Lucas: Some people are arguing exactly that. The EU's proposed AI liability directive includes a 'right to explanation' for automated decisions that cause significant harm. But the US doesn't have anything comparable yet. The airline here was based in the US. Luna: And even the right to explanation — would it have helped? The traveler got an explanation: 'the AI chose the cheapest fare.' But that didn't address the hidden restrictions. Lucas: That's the deeper issue. The explanation was technically correct but useless. The AI didn't surface the trade-offs. It didn't say: 'This fare saves you $50 but adds $60 in baggage fees and means you travel at inconvenient times.' It just booked. Luna: So we need AI that explains trade-offs, not just outcomes. Lucas: Exactly. There's research from MIT's Media Lab on 'contrastive explanations' — instead of saying 'I did X,' the AI says 'I did X instead of Y because of Z reason.' That would have helped here: 'I chose basic economy over standard because it saved $50, but you lose flexibility.' Luna: Would that have changed the traveler's mind? Probably. Lucas: Maybe. But the system didn't offer it. And that's a design choice — or a lack of one. The airline prioritized speed and conversion over informed consent. Luna: It reminds me of the 'dark patterns' in web design. Hidden fees, tricky cancellations. Only now it's conversational. Lucas: Exactly. And the stakes are higher because people trust conversational agents more. Studies show users disclose more personal information to chatbots than to web forms. They anthropomorphize them. So when the AI screws up, the betrayal feels personal. Luna: So what's the fix? Regulation? Better design? Both? Lucas: I think both. On the design side, any AI that can commit financial transactions should have a mandatory 'pause and confirm' step for anything over, say, $200. And it should summarize the key restrictions before you approve. Luna: And on the regulatory side? Lucas: The US could look at what the EU is doing. The AI Act classifies travel booking as 'limited risk,' not 'high risk,' which means fewer requirements. But maybe it should be higher risk given the financial and personal impact. Luna: I'd argue yes. A bad booking can ruin a vacation, cost you money, cause stress. That's not trivial. Lucas: And it's not just airlines. Hotel booking AIs, car rental AIs, even restaurant reservation bots — they all have similar failure modes. The common thread is that the AI's objective function is narrower than the user's. Luna: Objective function — that's the AI's goal, right? Like 'minimize price' or 'maximize convenience'? Lucas: Right. And the problem is that the user's actual objective is multi-dimensional: price, time, flexibility, comfort. The AI reduces it to one thing. So we need AI that can handle multi-objective trade-offs, or at least ask clarifying questions. Luna: Or maybe the AI should just say: 'I found three options. Here are the trade-offs. Which one?' Lucas: That's the ideal. Some travel startups are building exactly that — 'choice architecture' AI that presents a short menu of options with pros and cons. But the big incumbents haven't adopted it yet. Luna: Because it slows down the booking process and reduces conversion? Lucas: Probably. But in the long run, trust matters more. If people get burned once, they stop using the AI. The airline in our story — I bet that traveler never uses their chatbot again. Luna: And they might tell others. Word of mouth still matters. Lucas: Absolutely. So there's a business case for doing this right, not just an ethical one. Luna: Let me ask you this: what about AI agents that book on your behalf without real-time confirmation? Like, you say 'plan my vacation' and it books everything. Lucas: That's even scarier. We're seeing 'agentic AI' products that act autonomously — book flights, hotels, cars. Without guardrails, you could end up with a nightmare itinerary and no recourse. Luna: So the human-in-the-loop becomes even more critical. Lucas: Exactly. And I think the travel industry needs to adopt a standard — maybe an 'AI booking transparency label' that tells you what the AI can and cannot do. Like nutrition labels for AI. Luna: That's a great idea. What would it include? 'This AI may book non-refundable fares,' 'This AI requires confirmation for purchases over $100'? Lucas: Yes. And 'This AI explains trade-offs before finalizing.' Some consumer advocacy groups are pushing for exactly that. It's early, but the conversation is happening. Luna: Speaking of conversations that matter — if today's episode gave you something to think about, we want to mention that listener support keeps this show ad-free and independent. We don't run sponsor spots, and that's only possible because some of you choose to contribute. Lucas: Yeah, it's a small thing but it makes a big difference. If you find value in these deep dives into AI ethics, you can support the show at buy me a coffee dot com slash fexingo. No pressure — just if it feels right. Luna: And we really appreciate it. Now, back to travel agents and trust. Lucas: So where does that leave us? I think the core lesson is that AI systems need to be designed for the user's full context, not just a narrow optimization. And until they are, we need safeguards. Luna: What would you say to a developer building a travel booking AI tomorrow? Lucas: I'd say: build in a confirmation step, show trade-offs, and assume the user doesn't know the fine print. Because they probably don't. Your job is to make the invisible visible. Luna: That's a good rule for any AI, really. Lucas: It is. And it's one we'll keep coming back to on this show.