Latest / Future of Work Tech with Fexingo: Remote Tools, AI Productivity, and Workplace Software / How AI Avatars Are Training on Your Meeting Recordings
Transcript
- Lucas: So there's a story out of a Fortune 500 company — I'm not naming them because the internal memo leaked and they're still sorting out the PR — where HR quietly started building AI avatars of their top sales performers. Trained entirely on months of recorded Zoom and Teams calls. Luna: Wait — like, they recorded the calls and then… what, made a digital clone that talks like them? Lucas: Exactly. The idea was to capture not just the words but the tone, the pacing, even the little verbal tics. So that when a junior rep is prepping for a pitch, they can practice against an avatar that sounds exactly like their best closer. The company billed it as a 'force multiplier for sales onboarding.' Luna: I mean, I can see the logic. But did they ask anyone's permission first? Lucas: That's the thing — they used a broad 'calls may be recorded for quality and training purposes' clause in the employment contract. But the employees whose data was used say they had no idea their specific voice and mannerisms were being turned into an interactive AI. Luna: Right, there's a difference between recording for compliance and building a digital twin of your actual performance. Lucas: And look, honestly, if today's conversation gave you something useful — maybe a new angle on how your own company handles recordings — that's exactly the kind of thing that keeps this show going. If it felt worth a few bucks to you, you know the link: buy me a coffee dot com slash fexingo. No pressure, no perks, just knowing the show matters. Luna: Yeah, it really does help. And it keeps us independent, no sponsors to answer to. Lucas: So back to the story. The company used a third-party platform — I'll call it AvatarCo — that specializes in 'persona replication.' They fed it about five hundred hours of recorded sales calls per person, along with transcripts and CRM data. The output was a voice model and a simple 2D avatar that moves its mouth and gestures based on the conversation. Luna: Five hundred hours per person? That's a lot of data. What about people who didn't want to participate? Lucas: The company claimed the clause covered it, but after a few employees complained to HR, they offered an opt-out. By then, though, the models were already trained. Opting out meant the avatar was deleted, but the underlying training data had already been used. Luna: So once the genie is out of the bottle, you can't really put it back. That's the core issue. Lucas: Exactly. And this isn't just one company. I've heard from sources at three different vendors that they're seeing a spike in interest from enterprise clients. Sales teams, customer support, even some executive coaching firms want to build AI replicas of their best communicators. Luna: What does the research say? Does practicing against an avatar actually improve performance? Lucas: The early data is mixed. One internal pilot showed a twelve percent improvement in deal conversion among reps who did at least three avatar practice sessions per week. But the control group — which used traditional role-play with a manager — improved by nine percent. So it's an edge, but not a revolution. Luna: Twelve versus nine. That's statistically significant, but not the kind of leap you'd risk a privacy backlash for. Lucas: Right. And there's a Hawthorne effect angle too — the reps who got picked for the avatar program felt flattered, so they might have just worked harder. The company didn't blind the study. Luna: So the novelty itself could be driving the results. That's not a durable advantage. Lucas: Exactly. And once the novelty wears off, you're left with the privacy hangover. The employees whose voices were cloned started comparing notes on Slack. They found out that some of the recordings used for training included off-topic conversations — personal calls, venting sessions, even a few moments where people forgot they were on mute. Luna: Yikes. That's the kind of thing that erodes trust fast. Lucas: It did. The company ended up pausing the program, hiring an outside ethics consultant, and promising a full review. But the avatars that were already built? They're still in a sandbox environment, waiting for a decision on whether to delete them or proceed with stricter consent. Luna: What would 'stricter consent' look like here? Because I feel like a general recording waiver isn't going to cut it anymore. Lucas: The consultant recommended a tiered model: one level for recording, a separate explicit consent for using the recording to create an avatar, and a third option to allow the avatar to be used by other employees. Plus the right to withdraw at any time and have the model destroyed. Luna: That's more like it. But it also means the training data becomes less rich if only a subset of people opt in. Lucas: That's the trade-off. The company had to decide: do you want perfect data and damaged trust, or good data and a workforce that feels respected? They chose the latter, at least publicly. Luna: Which is the right call long-term, but I wonder how many other companies are just going ahead without that level of transparency. Lucas: Based on what vendors are telling me, plenty. One startup I spoke with said they have a 'stealth onboarding' process where clients can upload recordings without ever telling employees. The avatar is built, deployed in a private sandbox, and if no one complains, it stays. Luna: That's essentially going dark. No oversight, no audit trail. Lucas: Right. And that's where the regulatory landscape matters. The E.U. is already looking at this under the A.I. Act — any system that creates a digital replica of a person's voice or likeness using biometric data is likely going to be classified as high-risk. That means conformity assessments, data protection impact assessments, the works. Luna: But in the U.S., it's much more fragmented. Some states have biometric privacy laws — Illinois, Texas, Washington — but most don't specifically cover AI avatars trained on workplace recordings. Lucas: Exactly. So we're in this gray zone where the technology is moving faster than the law. And companies are making bets on what they think they can get away with. Luna: Let's talk about the actual tech for a second. How hard is it to build one of these avatars? Is it something a company could do in-house? Lucas: It's getting easier. You need a decent voice cloning model — something like ElevenLabs or Respeecher — plus a talking-head model that can synchronize lip movements. There are open-source options too, like Wav2Lip. For a company with a decent data science team, they could prototype something in a few weeks. Luna: So the barrier to entry is pretty low. That's a little unsettling. Lucas: It is. And the quality is good enough that in a blind test, employees at that Fortune 500 company could only correctly identify the real person versus the avatar about sixty percent of the time. That's barely above chance. Luna: Sixty percent? So almost half the time, people thought the avatar was the real person. That's… impressive and terrifying. Lucas: Exactly. And the longer the interaction, the harder it becomes to tell. The avatar handles small talk and follow-up questions surprisingly well because it's drawing on a large database of the person's actual responses. Luna: So we're getting to a point where your digital twin could take a meeting for you and no one would know. Lucas: That's the direction some vendors are pitching it. 'Attend meetings virtually without actually attending.' But that raises a whole other set of questions about accountability and agency. Luna: Right — if the avatar makes a commitment or gives bad information, who's responsible? The real person? Lucas: The companies I've talked to are still figuring that out. Most are saying the avatar is just a training tool, not a proxy for attendance. But the technology doesn't inherently have that limitation. Luna: So where does this go? Do you think we'll see industry standards emerge, or is it going to be a Wild West for a while? Lucas: I think we'll see a mix. Some industries — finance, healthcare — will move toward strict guidelines because of regulatory pressure. But in tech and sales, there's going to be a lot of experimentation. The key will be transparency and consent from the start. Luna: And the companies that get that right will have a competitive advantage in talent retention. Because who wants to work somewhere that might turn you into an AI without asking? Lucas: Exactly. Trust is a feature now — maybe the most important one.