Latest / Future of Work Tech with Fexingo: Remote Tools, AI Productivity, and Workplace Software / How AI Is Automating Your Internal Question Routing
Transcript
- Lucas: You know how when you have a question at work — maybe it's about expense policy, or how to reset a password — you send a Slack message to some random channel, or you email a general address, and then you just wait? Luna: Yeah, and sometimes you wait a day, sometimes you get an answer from the wrong person, sometimes you get three conflicting answers. Lucas: Right. So there's a whole category of software that tries to fix this — internal ticketing and question-routing systems. And what I'm seeing this year is that AI is starting to transform those systems in a really concrete way. Luna: Like, beyond just adding a chatbot that points to a FAQ? Lucas: Exactly. A chatbot is just the front door. The real shift is in how the system understands what you're asking and then decides who or what should handle it. I've been looking at this one large tech company — I won't name them, but they've got about 15,000 employees — and they rolled out an ai powered ticketing system about 18 months ago. Luna: What were they using before? Lucas: A traditional IT service management platform — basically a ticket queue where anything that came in was manually triaged by a small team. They had about 12 people doing triage full time. The average time from submission to first human touch was about four hours. And about 30% of tickets were just knowledge-base lookups — people asking questions that already had answers written down somewhere. Luna: So a huge waste of human attention. Lucas: Massive. So they put in an AI layer that does two things. First, intent classification — the system reads the natural language of the ticket and tags it with a category. Password reset, software request, policy question, whatever. Second, smart assignment — based on that intent, plus context like the employee's department and location, it routes the ticket to the right queue or person. Luna: And the chatbot handles the simple ones entirely? Lucas: That's the goal. For about 25% of tickets — the ones that are purely informational with a clear answer in the knowledge base — the AI just auto-responds with the relevant article and closes the ticket. No human involved at all. For the rest, it routes to the right person with the category already set, so the human doesn't have to read and tag it first. Luna: What happened to the average response time? Lucas: It dropped from four hours to about 45 minutes for tickets that need a human. And for the auto-responded ones, it's instant. The company estimates they saved about $2 million annually in triage labor costs, and they redeployed those 12 triage people into higher-value support roles — like handling complex technical escalations. Luna: So the AI isn't replacing people entirely, it's shifting them up the value chain. Lucas: That's the story they tell internally. But it's not all smooth sailing. The system has to be trained on a lot of historical tickets to get the intent classification right. And if your existing tickets are poorly written or mis-tagged, your training data is garbage. Luna: Right, garbage in, garbage out. How did they handle that? Lucas: They did a manual cleanup of about 10,000 tickets to create a high-quality training set. Then they used a pre-trained language model — think something like BERT or a similar transformer model — and fine-tuned it on that data. They also set up a feedback loop: whenever a human re-routes a ticket because the AI got the category wrong, that correction gets fed back into the model. Luna: So the model gets better over time. What about integrations with tools like Slack or Teams? Because most people don't want to open a separate ticketing portal. Lucas: Exactly. The system they built has a Slack bot front end. You type your question in natural language in a Slack channel, and the bot responds — either with the answer directly, or with a ticket number and an estimated time for a human response. The employee never has to leave Slack. Luna: That's huge for adoption. If you make people go to a new tool, they just won't use it. Lucas: The company saw adoption go from about 60% of employees using the old portal to over 90% using the Slack bot within three months. Because it's frictionless. You're already in Slack all day. Luna: What about the accuracy of the intent classification? Like, if I ask 'how do I get my laptop fixed?' versus 'what's the policy on remote work equipment?' — those are different intents but the phrasing might be similar. Lucas: Great question. In their initial rollout, the model had about 85% accuracy on intent classification. After six months of feedback and retraining, it got to 92%. The remaining 8% are edge cases — ambiguous language, or questions that span multiple categories. For those, the system falls back to a generic queue and a human sorts it out. Luna: So the human still handles the fuzzy stuff. That makes sense. But what about the knowledge base itself? If the AI is auto-responding with articles, the articles have to be good. Lucas: That's another piece. They actually used AI to audit their knowledge base — the system flagged articles that were outdated or had low readability scores. The company ended up rewriting about 40% of their top-100 most-viewed articles. That alone improved the auto-response success rate by about 15 percentage points. Luna: So the AI is improving the content as well as the routing. Lucas: It's a virtuous cycle. Now, this isn't just for IT. I'm seeing the same pattern in HR — people asking about benefits, payroll, time off. And in finance — expense report questions. The same technology applies. Luna: Are there any tools specifically for this that you're seeing gain traction? Lucas: There are a bunch. Some are built into larger platforms like ServiceNow or Jira Service Management — they've added AI capabilities. But there are also standalone tools like Alloy.ai or Espressive that focus purely on the AI triage layer. And of course, you can build your own using something like Rasa or a large language model API. Luna: What about privacy and security? If you're routing tickets about sensitive HR issues, you don't want the AI model learning from that data in a way that exposes someone. Lucas: That's a big consideration. The company I mentioned keeps all the data on-premise or in a private cloud. The model is trained only on de-identified ticket data. They also have a policy that any ticket flagged as 'sensitive' — like harassment complaints — is never used for training and is always routed to a human first, bypassing the AI entirely. Luna: So you need that human oversight built in from the start. Lucas: Absolutely. The technology is powerful, but it's a tool, not a replacement for judgment. Luna: And honestly, if today's conversation gave you something useful to think about — maybe you're considering this for your own team — we should mention that listener support is what keeps this show ad-free. If it was worth a coffee to you, that's the link: buy me a coffee dot com slash fexingo. Lucas: Yeah, and we mean that. A small gesture goes a long way to keep these deep dives coming. Now, back to the routing tech — one other thing I wanted to highlight is that this isn't just about tickets. Some companies are using similar AI to route questions in real time during live chat or even voice calls. Luna: So like a contact center AI that listens to the conversation and recommends an answer to the human agent? Lucas: Exactly. Or even fully automated voice responses for simple queries. But the core insight is the same: understand what the person needs, then get them to the right resource as fast as possible. Luna: I think the big takeaway is that the technology is mature enough now that it's not just a futuristic idea — it's saving real money and real time today. Lucas: Right. And the barrier to entry is dropping. You don't need a massive data science team. You can start with a simple chatbot and a set of FAQs, and then layer in smarter routing as you go. Luna: That's a good note to end on. Thanks, Lucas. Lucas: Thanks, Luna. Until next time.