Latest / The Adults In The AI Room / Earley AI Podcast - Episode 93: AI Translation, Brand Voice, and Global Content with Olga Beregovaya

Transcript

Earley AI Podcast - Episode 93: AI Translation, Brand Voice, and Global Content with Olga Beregovaya · The Adults In The AI Room

• 0:01
  SPEAKER_01: This is the Early AI Podcast hosted by Seth Early.
• 0:04
  SPEAKER_01: Join us as we delve deep into the passions, expertise, and experiences of thought leaders and practitioners to talk about what's possible with artificial intelligence.
• 0:14
  SPEAKER_01: The Early AI Podcast is sponsored by Early Information Science, your digital transformation journey with design and deployment of innovative technology solutions, as well as Vector.
• 0:26
  SPEAKER_01: Now enjoy the show.
• 0:28
  SPEAKER_03: Welcome to the Early AI Podcast.
• 0:31
  SPEAKER_03: I'm your host, Seth Early, and in each episode we explore how artificial intelligence is changing the way we look at data and business problems and business strategy and business operations and see how things are evolving in this very, very fast-moving space.
• 0:48
  SPEAKER_03: Today we're going to talk about AI translation and global content strategy, a space where the stakes are high, the complexity is underestimated, and the gap between a working demo and a production system at enterprise scale is wider than people realize.
• 1:01
  SPEAKER_03: There are 7,000 plus languages in the world.
• 1:04
  SPEAKER_03: That blows my mind, that statistic.
• 1:06
  SPEAKER_03: And the assumption, the assumptions that work for romance languages fall apart quickly everywhere else.
• 1:12
  SPEAKER_03: Joining me today is Olga Berakovania.
• 1:15
  SPEAKER_03: Did I get that quite right?
• 1:16
  SPEAKER_03: No.
• 1:16
  SPEAKER_02: Almost, almost, almost there.
• 1:18
  SPEAKER_03: Almost there.
• 1:19
  SPEAKER_03: Berakovanya.
• 1:20
  SPEAKER_03: Vaya.
• 1:21
  SPEAKER_03: No end.
• 1:22
  SPEAKER_03: Berakovaya.
• 1:23
  SPEAKER_03: Okay.
• 1:23
  SPEAKER_03: Yes.
• 1:23
  SPEAKER_03: VP of AI at SmartLink, an AI first translation and global content platform.
• 1:27
  SPEAKER_03: Olga brings 25 years of experience in natural language processing and linguistic machine learning, spanning every major evolution in this in the field from rules-based systems through to statistical models, neural translation, and now LLMs.
• 1:40
  SPEAKER_03: She served as CEO of a machine translation company, led language technology at Autodesk for over a decade, and now leads AI at SmartLink.
• 1:48
  SPEAKER_03: Olga, welcome to the show.
• 1:50
  SPEAKER_03: Thanks so much for having me.
• 1:51
  SPEAKER_03: So we start off.
• 1:52
  SPEAKER_03: I like to talk about what people don't understand, the misconceptions and about AI and specifically your space.
• 2:00
  SPEAKER_03: When you talk to executives and technology leaders about AI translation, what do they most consistently get wrong?
• 2:08
  SPEAKER_02: Okay.
• 2:08
  SPEAKER_02: The first thing that's gotten wrong most consistently is looking at the price sheet for commercial provider tokens and just saying, hey, why can I just not pay whatever, let's say$7 per million tokens and call it a day, plug it into my ecosystem.
• 2:26
  SPEAKER_02: So I guess a lot of people just get mesmerized by the price and do not realize the complexities behind.
• 2:31
  SPEAKER_02: And that's actually one of the most frequent conversations that we're having with executives and say C-suite executives, uh senior leadership executives, like, guys, yes, you can totally consume your million tokens at a super low price point, but what exactly are you buying for this money?
• 2:45
  SPEAKER_02: So I guess the simplicity and accessibility and like everything is available right now, and I could just do it myself within a day, that's probably the biggest one.
• 2:53
  SPEAKER_03: And and how do you really uh how do you really illustrate that?
• 2:57
  SPEAKER_03: Like how does that manifest itself?
• 2:59
  SPEAKER_03: And you know, when when is well one of the things that uh we talk about is the need for information architecture when it comes to artificial intelligence.
• 3:07
  SPEAKER_03: But there are instances where you don't need it, right?
• 3:10
  SPEAKER_03: You can you can drop 400 PDFs into uh you know into a SharePoint repository, you can uh run uh you can run uh copilot against it, and you can get some answers.
• 3:22
  SPEAKER_03: And of course, there are use cases like that, right?
• 3:25
  SPEAKER_03: And and my answer to that is well, what would have worked without that uh any curation or any information architecture for copilot would work with full text search because it's it's it's curated content, it's not messy content, it's usually uh a well-maintained application.
• 3:41
  SPEAKER_03: This happened to be about uh standard operating procedures.
• 3:44
  SPEAKER_03: So when so you have to concede that argument that sometimes it's okay to not have IA for your AI.
• 3:49
  SPEAKER_03: When is it okay to do the simple way?
• 3:52
  SPEAKER_03: What it what are the use cases where that argument is is reasonable?
• 3:57
  SPEAKER_02: My message, and that's what I usually share, is just separate branded content versus not branded content.
• 4:05
  SPEAKER_02: In other words, if it's the content that represents your brand, and if it's outbound content that represents your brand and you're super sensitive to your brand tone and voice in the market, then copilot is not a good idea.
• 4:18
  SPEAKER_02: And I'll actually use a specific example.
• 4:20
  SPEAKER_02: So if it can carry any kind of potential brand damage, or if it can carry any liabilities with it.
• 4:26
  SPEAKER_02: And a great example would be, for instance, Sierros or any kind of documents in regulated industries like life sciences, like pharma, right?
• 4:33
  SPEAKER_02: You are you can do it, but why would you if it can potentially bring about a lawsuit?
• 4:38
  SPEAKER_02: And that's the last thing you want.
• 4:39
  SPEAKER_02: So that's the no.
• 4:40
  SPEAKER_03: That's the so the converse of that to say when it does work is when you have more uh pedestrian content that doesn't impact the brand.
• 4:49
  SPEAKER_03: It's not branded content, it's like a disclaimer, maybe I don't know for what the right example is, but but do you have examples of when it would be okay just to use uh a simple uh LLM?
• 5:00
  SPEAKER_02: Thousand percent.
• 5:01
  SPEAKER_02: And you just use the term that I might need to steal from you from this point on, which is pedestrian content.
• 5:05
  SPEAKER_02: I never never heard that one before, but it's a great way of putting it in our industry and language technology industry, we'll say user-generated content.
• 5:13
  SPEAKER_02: Right.
• 5:13
  SPEAKER_02: So UGC, for instance, if it's internal comms and all you need is just process internal emails, don't overthink it.
• 5:20
  SPEAKER_02: Copilot is the way to go, right?
• 5:22
  SPEAKER_02: Or say user feedback, even more so in less specialized industries, the industries that are less dependent on language-specific term, industry-specific terminology, right?
• 5:33
  SPEAKER_02: Like retail, user reviews, opinion portals, go for it.
• 5:37
  SPEAKER_02: Totally go for it, plug it in, and you're golden.
• 5:40
  SPEAKER_03: And that makes sense.
• 5:40
  SPEAKER_03: And when there is risk, when there is a regulated industry, when you do have brand sensitivity, that's when you really need to think about it more carefully.
• 5:48
  SPEAKER_03: Is there anything else that they don't get right?
• 5:50
  SPEAKER_03: Is there anything else that they misunderstand or misconstrue?
• 5:53
  SPEAKER_02: Uh, I think it's like, okay, so the first message was more, I guess, about marketing and business owners, right?
• 5:58
  SPEAKER_02: And you have marketing and if you like if you own your company's business and the geo again, you should be very, you should be very sensitive to your brand voice.
• 6:06
  SPEAKER_02: And I'll just just a little bit of an aside.
• 6:07
  SPEAKER_02: I was talking to a senior, senior level executive from Not of Unfortune 500, not of 100, not a 150, more like Fortune 10 company.
• 6:15
  SPEAKER_02: And they installed one of those co-assistant co-pilot tools.
• 6:18
  SPEAKER_02: I'm not gonna call it out, and he's like, Yeah, it's great, except my company has 300,000 employees, and now my company has 300,000 voices, and that's not necessarily what I was prepared for in different countries.
• 6:31
  SPEAKER_02: So that's one thing that people do not think about, right?
• 6:35
  SPEAKER_02: Again, everybody can totally produce a translation or generate copy, but is it going to if if I have my voice, like let's say both of yours because of Spanish, I have my voice, you have your voice, and that content has gone out.
• 6:47
  SPEAKER_02: That's not necessarily something you want.
• 6:48
  SPEAKER_02: And now the misconception is, and now I'm gonna speak more of engineering and information architecture language.
• 6:56
  SPEAKER_02: First of all, people totally underestimate the importance of data integrity when it comes to in-platform data that they're gonna be using RAG into or some kind of agentic approach into data integrity, both engineering-wise and linguistics-wise.
• 7:11
  SPEAKER_02: And data interoperability is key.
• 7:13
  SPEAKER_02: And I don't think people estimate, really estimate that what comes from Git should really be able to talk to what comes from your content management system, right?
• 7:22
  SPEAKER_02: And AI and large language models specifically, nothing I think is more sensitive to garbage in, garbage out than language technology.
• 7:30
  SPEAKER_02: And the other one, I don't think people I was just talking to our VP of engineering, and he just showed me the log of all the events that happen in the platform to process a chunk of set of text.
• 7:41
  SPEAKER_02: And I think people just assume content in, content out.
• 7:45
  SPEAKER_02: But in reality, for instance, large language models do not play well with special characters.
• 7:49
  SPEAKER_02: They just don't like emojis as much.
• 7:51
  SPEAKER_02: So people don't, I think people don't understand the importance of information architecture and governance and the the pipeline, right?
• 8:00
  SPEAKER_02: Before you can send anything to the model, make sure that the model is not gonna choke on it.
• 8:05
  SPEAKER_02: Very simple.
• 8:05
  SPEAKER_02: Your inline tagging, like good old Wizaby HTML, can completely stall your entire translation process and your entire delivery process.
• 8:13
  SPEAKER_02: So I think complexity, engineering complexity, and data integrity is another one people are not paying enough attention to.
• 8:19
  SPEAKER_03: And and they don't necessarily realize that it's is very important for language translation as it is for retrieval and access.
• 8:28
  SPEAKER_03: And and so uh what you're saying is it's really the the processing pipeline and making sure that it is understandable by whatever uh tool you're using and being able to get quality results out of that.
• 8:41
  SPEAKER_03: And there has to may have to be some pre-processing in order for that to happen.
• 8:44
  SPEAKER_02: Absolutely so.
• 8:45
  SPEAKER_02: Absolutely so.
• 8:46
  SPEAKER_02: And there is another a lot of attempt, attempts, I guess, a lot of experimentation is going on now, and some very successful, marrying the knowledge management, which obviously would be essential for retrieval, marrying LLM, Gen AI, frontier models-based processes with more old school knowledge management, such as knowledge graphs or ontologies.
• 9:06
  SPEAKER_02: And this is actually again talking about the overall architecture space that's equally relevant for generation retrieval, new generation of search, and LLM-based, prompt-based or rag-based or whatever you're doing there processes.
• 9:18
  SPEAKER_02: So uh yeah, I guess that was my answer.
• 9:22
  SPEAKER_03: So uh so yeah, so that that is interesting because I would not have thought that the IA would be as important in this context, but but it is part of the pipeline processing because you still have to do things like chunk it, tag it, and now you have to do that in different languages, and then the meanings of those tags are going to be different, the context will be different, the use cases will you know will vary, and so on.
• 9:46
  SPEAKER_02: And from there, uh it's not only chunking the content, think about the string length, right?
• 9:51
  SPEAKER_02: Like let's say you're generating, producing, generating content or translating content for your UI, right?
• 9:57
  SPEAKER_02: The model can go a rogue.
• 9:58
  SPEAKER_02: The model, even if you tell the model be formal, be like you can put all the constraints you want.
• 10:03
  SPEAKER_02: And before you know it, the model needs, I don't know, 40 more characters to dream things up to its liking.
• 10:09
  SPEAKER_02: Right.
• 10:09
  SPEAKER_02: So there are also constraints, there are a lot of guardrails that you need to put in place that people just don't think about.
• 10:14
  SPEAKER_02: Tokenizer, you just said you need to check this content.
• 10:17
  SPEAKER_02: Like I think even the process of tokenization is underestimated, right?
• 10:21
  SPEAKER_02: Because whatever you're gonna get from bidirectional languages, right?
• 10:25
  SPEAKER_02: RTLs and whatever you're gonna get from ideographic languages, they all would pose their own tokenization from the interesting.
• 10:32
  SPEAKER_03: It's so interesting.
• 10:32
  SPEAKER_03: And there was something that just uh came to mind as you were speaking about that, and now I've forgotten what that was.
• 10:39
  SPEAKER_03: But I think that it's interesting that you know you talk about how you do have to really think through this the pipeline and how that pipeline is going to be different depending upon these languages.
• 10:50
  SPEAKER_03: What when you s when we say there's 7,000 languages in the world, is that is that an accurate statement, or did my uh LLM make that up?
• 10:58
  SPEAKER_02: No, it did not.
• 10:59
  SPEAKER_02: Or at least then all my other sources equally make it up.
• 11:02
  SPEAKER_02: Uh it's uh I think it's 7,200 something or to 7,120.
• 11:06
  SPEAKER_02: Now it's very important that not all of these languages actually have scripts associated with them, right?
• 11:12
  SPEAKER_02: Not all of them have glyphs, not not all of them have a writing system.
• 11:14
  SPEAKER_02: So when we're talking about spoken languages, obviously we need to be very careful separating.
• 11:19
  SPEAKER_02: And that opens another issue in the world of AI, uh language AI, be translation or whatnot.
• 11:25
  SPEAKER_02: What do you do with long-tailed languages, under resource languages, right?
• 11:29
  SPEAKER_02: I mean, your models are predominantly trained on English or more prevailing language content.
• 11:34
  SPEAKER_02: Out of these 77,000, I'd probably say that at least half of them, if not all the half of them, are underserved in the universe, in the universe of natural language processing and AI translation.
• 11:46
  SPEAKER_02: Or again, again, global content.
• 11:48
  SPEAKER_02: Let's narrow it down to translation.
• 11:50
  SPEAKER_03: And what is it that what what is the number of languages that are kind of uh you know, what what what does that tail, what does that long tail look like?
• 11:57
  SPEAKER_03: What does that head and torso look like in terms of the number of languages that are commonly, not commonly, but you know, what does it look like?
• 12:04
  SPEAKER_03: What statistically speaking, how does it break down in terms of the number of languages that require a focus and attention in in this way?
• 12:11
  SPEAKER_02: Okay, the biggest temptation is just quickly launched GPT or Claude on my computer and look it up.
• 12:16
  SPEAKER_02: Uh let me drain things up a little bit.
• 12:18
  SPEAKER_02: So I would say, let's just split it, chunk it into language families.
• 12:22
  SPEAKER_02: And let me start with language families that are appropriately served, right?
• 12:27
  SPEAKER_02: And this is what will give you again, look at the population, availability of writing system, and representation in the digital space, right?
• 12:35
  SPEAKER_02: These would be three factors that would put the languages on either side of the spectrum or on torso in between.
• 12:41
  SPEAKER_02: So the one, and also maybe the way those languages render themselves to automated language processing.
• 12:47
  SPEAKER_02: So I would start with romance languages.
• 12:50
  SPEAKER_02: If I want to do a demo to a customer and put my best foot forward, grab Latin American Spanish.
• 12:56
  SPEAKER_02: Why?
• 12:57
  SPEAKER_02: Huge population, romance language with not super complex grammar, not super complex morphology, not super complex uh syntax.
• 13:04
  SPEAKER_02: So romance languages would be probably on the ultimate, perfectly served.
• 13:09
  SPEAKER_02: Then you would go with your Germanic, German being the most complex one of them.
• 13:13
  SPEAKER_02: Then I would probably from there would proceed to Slavic, then from there I would proceed to Asian, be it the usual suspect, the usual four or Southeast Asian, and then you have things like Finno-Yogaric languages, right?
• 13:25
  SPEAKER_02: Estonian.
• 13:26
  SPEAKER_02: And this is probably the bulk of what's really served by linguistic natural language processing, linguistic community.
• 13:34
  SPEAKER_02: Then you come with your Indic.
• 13:35
  SPEAKER_02: So let's say we just isolated like good 30 languages, right?
• 13:38
  SPEAKER_02: We isolated good 30 top performers.
• 13:41
  SPEAKER_02: Then the next batch is probably going to be, let's say, another 30 performers.
• 13:45
  SPEAKER_02: And that's for instance how we think about it at SmartLink.
• 13:47
  SPEAKER_02: And this is where Indic languages would come in, right?
• 13:49
  SPEAKER_02: A little bit more still thinking.
• 13:52
  SPEAKER_03: So which languages did you say?
• 13:53
  SPEAKER_02: Which Indic, like Indic, Hindi, Hindi, Punjabi, all the surroundings.
• 13:57
  SPEAKER_02: Then you get probably your Turk Turkic.
• 13:58
  SPEAKER_02: But again, think about it from an economic perspective, right?
• 14:01
  SPEAKER_02: It's a it's not a linguistic phenomenon, it's a sociolinguistic phenomenon.
• 14:05
  SPEAKER_02: How much can you actually fetch?
• 14:07
  SPEAKER_02: So I would say that you have probably 150 languages that are well served and well represented.
• 14:13
  SPEAKER_02: And I think I just delivered the breakdown.
• 14:15
  SPEAKER_02: But from there, from there, you're really moving to long tail.
• 14:18
  SPEAKER_02: And we actually work with, like, for instance, African Language Lab, that our their mission is digitizing African languages.
• 14:25
  SPEAKER_02: So from there, just imagine again, geographically, the geos where internet is not necessarily accessible, where kids don't have computers to go to school.
• 14:34
  SPEAKER_02: And then that would plant your like indigenous languages of South America, African languages.
• 14:40
  SPEAKER_02: So I think that that map, when you think about it, is a very, very intuitive map.
• 14:45
  unknown: Yeah.
• 14:45
  SPEAKER_02: It's tied directly to economic phenomena.
• 14:47
  SPEAKER_03: Right, right.
• 14:48
  SPEAKER_03: That's very fascinating.
• 14:49
  SPEAKER_03: And again, didn't think of it in those terms because I'm not uh a linguistic uh expert by any means.
• 14:55
  SPEAKER_03: Well, maybe in English, when it comes to information architecture, I know something about linguistics, but not when it comes to multiple languages.
• 15:02
  SPEAKER_03: By the way, how many languages do you speak?
• 15:04
  SPEAKER_03: Out of curiosity.
• 15:05
  SPEAKER_02: I mean, uh right now I'm in my summer home in Mexico, so that gives me broken Spanish.
• 15:09
  SPEAKER_02: Okay.
• 15:09
  SPEAKER_02: I mean survival Spanish.
• 15:11
  SPEAKER_02: I do speak Danish.
• 15:13
  SPEAKER_02: I mean, I obviously speak Russian, that's my mother tongue.
• 15:15
  SPEAKER_02: I do speak uh just happened geographically because I'm from St.
• 15:19
  SPEAKER_02: Petersburg in Russia, just the proximity to the Nordic countries gave me the entire Nordic package.
• 15:24
  SPEAKER_02: So that would be Danish, Swedish, Norwegian, uh some kind of broken Icelandic.
• 15:29
  SPEAKER_02: I read Old Norse, which my son is very proud of.
• 15:31
  SPEAKER_02: Obviously, had to study Latin, and then from there, I would say probably like reading knowledge, academic knowledge of French and German.
• 15:40
  SPEAKER_02: So that would that would give you a pretty good scan.
• 15:42
  SPEAKER_02: Yeah, and whatever whatever Yiddish my grandparents spoke at home, but that again does these are these are childhood memories.
• 15:48
  SPEAKER_03: So what what did what did that total up to, just out of curiosity?
• 15:52
  SPEAKER_03: Eight.
• 15:52
  SPEAKER_03: Yeah, wow.
• 15:53
  SPEAKER_03: That's amazing.
• 15:54
  SPEAKER_03: That's fantastic.
• 15:55
  SPEAKER_02: One thing I would want to say though, a lot of people, when you say that you're in the linguistics field, and you just said it yourself, like you don't necessarily need a foreign language, you need to understand how languages work.
• 16:04
  SPEAKER_02: And a lot of times people ask us, like, oh, so like myself and my colleagues, how many languages do you speak?
• 16:09
  SPEAKER_02: And it's important that you do speak foreign languages, but if it's more important if you understand the structure of languages, and you understand the structural linguistics.
• 16:17
  SPEAKER_02: Like I don't speak a word of Japanese or a word of Korean, but damn do I know how many difficulties for writing systems introduce for Japanese.
• 16:26
  SPEAKER_02: So I think it's more marrying your understanding of yes, not all languages are like your mother tongue, but also really understanding the broader picture.
• 16:34
  SPEAKER_02: Like RTL.
• 16:35
  SPEAKER_02: RTL is another great example.
• 16:37
  SPEAKER_03: Tell me about RTL in terms of uh right to left, right?
• 16:40
  SPEAKER_02: Um again.
• 16:41
  SPEAKER_03: Oh yeah, yeah, yeah, right.
• 16:42
  SPEAKER_03: Yeah, yeah.
• 16:42
  SPEAKER_02: Or like code switching languages with busy code switching.
• 16:46
  SPEAKER_02: I don't speak any, like I don't speak Hindi, but I do know that because of the history of India, the code switching is very prominent.
• 16:53
  SPEAKER_03: Tell me what code switching is.
• 16:54
  SPEAKER_03: I don't know what that means.
• 16:55
  SPEAKER_02: Uh code switching is every uh computational linguist and natural language processing professional nightmare because it's basically when half the sentence, the language and the script and everything just changes.
• 17:06
  SPEAKER_02: Like here you were you were happily living in the Japanese world or Hindi world, and then suddenly in the middle of the sentence you just get a plug-in from another language, which is also represented like you're, you know, like say uh right to left again, like you have Arabic, Arabic, Arabic, hello, a plug of English alphabet.
• 17:22
  SPEAKER_02: And this is where your entire processing chain, unless you've designed it properly, the entire processing is just go not doing it.
• 17:30
  SPEAKER_03: They just give up, they throw their hands up.
• 17:32
  SPEAKER_03: Not actually.
• 17:34
  SPEAKER_03: So this is also uh it's interesting because it really does it's it's it's it it certainly is is representative of the equity and availability and you know the what what what am I searching for here?
• 17:49
  SPEAKER_03: It it it it means that those populations are underrepresented when when you don't have those abilities to translate those languages, they're just left out, essentially.
• 17:59
  SPEAKER_03: Those people are left out.
• 18:01
  SPEAKER_03: So this is an important perspective from from or important issue from that perspective.
• 18:05
  SPEAKER_02: In the modern time and day, everybody speaks about digital divide, right?
• 18:08
  SPEAKER_02: You open any blog, any podcast, and digital divide is a very prominent concept.
• 18:13
  SPEAKER_02: But I don't think people think enough about linguistic divide and the universe I operate in, the notion of linguistic divide is essential for us.
• 18:22
  SPEAKER_02: Obviously, linguistic divide follows digital divide if you don't have any means of accessing accessing digital data.
• 18:28
  SPEAKER_02: But I think bridging the linguistic divide is, I mean, that's the biggest mission.
• 18:31
  SPEAKER_02: And people do the most fascinating things.
• 18:33
  SPEAKER_02: People go out, they record speech.
• 18:35
  SPEAKER_02: Yeah, the ASR, right?
• 18:37
  SPEAKER_02: The automated speech recognition now is at a place, even for languages that the system does not necessarily understand.
• 18:43
  SPEAKER_02: Well, the ASR is at a place where you can actually collect spoken data, you can actually develop uh a written model training corpora out of spoken data.
• 18:54
  SPEAKER_02: And there's quite quite a bit of quite a bit of that happening as well.
• 18:57
  SPEAKER_03: So that is so what does that mean?
• 18:59
  SPEAKER_03: Uh and uh, you know, on the I mean, again, these languages are disappearing as well, right?
• 19:04
  SPEAKER_03: So does it have some implication around the preservation of these languages in the long term?
• 19:10
  SPEAKER_02: It definitely does, but it's a two-edged sword.
• 19:13
  SPEAKER_02: And yes, indeed.
• 19:14
  SPEAKER_02: Obviously, the more data you collect, the more data you can uh, the more knowledge you can present to people in their own language, the more you help those languages survive, right?
• 19:24
  SPEAKER_02: On one hand.
• 19:25
  unknown: Right.
• 19:25
  SPEAKER_02: On the other hand, think about it, the improvements in the quality of translation can actually lead to a completely different story, which would be might not be the most popular term, but that would be linguistic colonization.
• 19:36
  SPEAKER_02: Yes, where suddenly, even if it's presented to you in your language, the English language or like an English language or other more prominent language-centric phenomena can become overbearing even in your own language, right?
• 19:49
  SPEAKER_02: Your language may survive linguistically, but it's not going to survive conceptually and anthropologically.
• 19:54
  SPEAKER_03: Interesting.
• 19:55
  SPEAKER_03: Very interesting.
• 19:56
  SPEAKER_03: And so that really is a fascinating uh approach.
• 19:58
  SPEAKER_03: Now, we did talk a little bit about regulated industries, financial services, pharma, medical devices, legal.
• 20:05
  SPEAKER_03: You know, what do you think is legitimate in terms of caution and what are people what what's overblown?
• 20:12
  SPEAKER_03: So how do you how do you find the line between you know something that can be alarmist versus something that is uh you know more practical?
• 20:20
  SPEAKER_03: So where where do you how do you see that spectrum?
• 20:22
  SPEAKER_02: Okay, generally uh regulated industries have been much slower at adopting any kind of NLP or even like global content transformation management, just because of the liabilities and the risks.
• 20:35
  SPEAKER_02: And usually, even in the early days of machine translation, heavy machinery would already be covered, right?
• 20:41
  SPEAKER_02: And by the way, in heavy machinery or like if you if your engine says press this lever or pull this liver or do not pull this liver, next thing you know, you could have dropped a whole crane, right, and destroyed the entire construction site.
• 20:53
  SPEAKER_02: But risk, there were much more, those industries were much less risk averse.
• 20:58
  SPEAKER_02: Regulated industries, just because of the compliance issues, are were slow to the party and still are to the party.
• 21:04
  SPEAKER_02: It's InfoSec, it's PII, right?
• 21:07
  SPEAKER_02: Like just the fact of PII leakage.
• 21:09
  SPEAKER_02: There are a lot of legitimate concerns, but the thing is they are all mitigatable.
• 21:15
  SPEAKER_03: Right.
• 21:15
  SPEAKER_03: Okay.
• 21:16
  SPEAKER_02: Right.
• 21:16
  SPEAKER_02: And there are different means of doing it.
• 21:18
  SPEAKER_02: There are different means of doing it.
• 21:20
  SPEAKER_03: So what are some of the mitigation?
• 21:21
  SPEAKER_03: What are some of the steps to mitigation of those risks?
• 21:24
  SPEAKER_02: I just use PII, PHI, right?
• 21:27
  SPEAKER_02: The biggest concern is like, hey, my patient's uh PII or PHI is going to leak.
• 21:32
  SPEAKER_02: And obviously it's just the normal IT precautions, right?
• 21:36
  SPEAKER_02: And obviously making sure that all your connectors, all the APIs speak to APIs in the most secure manner.
• 21:41
  SPEAKER_02: That's one, right?
• 21:42
  SPEAKER_02: And that's where you preserve the leakage.
• 21:44
  SPEAKER_02: If you're using commercial models, obviously make sure that all your SLAs, all your agreements with commercial models are what's called zero trace, right?
• 21:53
  SPEAKER_02: Where they can they cannot like even cache your data, right?
• 21:56
  SPEAKER_02: So not caching the data, making sure that none of the data can be used for.
• 22:00
  SPEAKER_02: training right private keys are essential like you always want to make sure that somehow your instance is private and not share it with anybody uh and like even P A P H A masking paraphrasing there are so many techniques that actually can be put in place to protect your customer data hallucinating it's just typical stuff that you have to do anyway in protecting that data you have to you have to but I think it's blown out of proportion the risks and a lot of times I see the InfoSec questionnaires and I'm like okay well this is 10 years ago this is from five years ago we've really we've really made so much more progress since then and yeah I just gave you the most obvious examples the one area of NLP is actually you mentioned PDFs regulated industries are way ahead of everybody else when it comes to OCR right yes reason being uh if you're a medical I mean you're you first of all they were dealing maybe military right dod and or whatever military sector and regulated industries are dealing with much more handwritten content than anybody else.
• 23:06
  SPEAKER_02: I see that right for a flat PDF that you just like you know you have an archive of 45,000 cabinets all of them filled with PDFs and you need to OCR them.
• 23:16
  SPEAKER_00: Right.
• 23:16
  SPEAKER_02: That's one place and that actually paved way to faster adoption of AI by regulated industries because they were already halfway there.
• 23:26
  SPEAKER_02: Right, right being able to digitize their assets.
• 23:29
  SPEAKER_03: Interesting interesting so when you start thinking about what what what happened when you you've seen so you've seen this industry change through all these different architectural shifts and what changed when LLMs arrived and what did that unlock that was not possible with say you know neural machine translation and other approaches because they I see them on a spectrum but maybe I I'm not understanding it as well.
• 23:56
  SPEAKER_03: So what what is your take on how LLMs changed the landscape?
• 24:01
  SPEAKER_02: Well I would say that I mean first and foremost 19 God I still live in the previous century.
• 24:06
  SPEAKER_02: 2017 right when transformers as we know them attention is all you need or attention is what you need when this famous Google research paper came out right and introduced transformer models give a take as we know them right that was probably the first breakthrough right and that's where neural machine translation deep learning based machine translation took over and again other approaches based linguistic approaches for other use cases based on that.
• 24:29
  SPEAKER_02: So that kind of paved way to where we are now with LLMs.
• 24:33
  SPEAKER_02: Now the constraint was and in knowledge management and in other again other applications, queries the constraint was always that neural machine translation is generally operating within sentence boundaries and all the APIs and all the processing actually it all would send one chunk of text right it's operating it's literally operating between start of sentence end of sentence and I would say the two major breakthroughs with LLMs were one that next generation of transformer models right the whole notion of pre-trained generated transformers which is basically next generation right it killed the two previous the two previous iterations so one second is again the attention mechanism was completely overhauled for what you do and how you actually parse out your content just the number of parameters right one was also an LLM BERT was also an LLM but once you go into those massive foundational models just the number of parameters and the volume of training text unlocked the accessibility to access unlocked access to different domains different languages again right because as much as was mostly English centric you still could have access different languages.
• 25:44
  SPEAKER_02: I would say the biggest and the hugest unlock was context as I said NMT operating neural models generally operating within sentence boundaries and suddenly things like resolving pronouns resolving omonyms resolving ambiguity in terminology all of it became a non-issue right as long as you can operate on a chunk level or even better so document level.
• 26:08
  SPEAKER_02: So that would probably be the biggest unlock that's so interesting.
• 26:11
  SPEAKER_03: Yeah I I had not I had not thought of that.
• 26:14
  SPEAKER_03: So we've we've also kind of talked a little bit about this but you know we we know that it's there's are are there assumptions that translation works well equally well across all languages obviously not because there's underserved languages and but but and that's a a dangerous misconception um what you know where does it where does it break down and and and is it just a matter of these underserved languages like where else does it break down when you start looking at these language families and and what else do people need to think about when it when it comes to the effectiveness of of machine translation?
• 26:49
  SPEAKER_02: Well yeah again it's translation or content translation content generation or anything else I would say I mean yes obviously it's the representation of the training data again the quality of tokenizers is a big issue right if you cannot tokenize your text properly that's another breaking point next breaking point would be just pure linguistic complexity complexity of language right if you look at Finno Greek languages I mean I don't speak Hungarian but if I'm not mistaken we're talking like 16 cases for nouns and I forgot how many conjugations.
• 27:21
  SPEAKER_02: So obviously more grammatically complex languages obviously they introduce uh they introduce additional level of complexity then again let me think what else and again like I would use Japanese as an example the presence of four writing systems obviously that's another one right where you need to pivot between when you need to pivot between different writing systems there.
• 27:39
  SPEAKER_02: Each of them each of language families comes with its own complexities but I would probably call out like chromatic complexity is probably is probably the main one.
• 27:48
  SPEAKER_02: Take German for instance there is another everybody's favorite is German compounds for instance and that would not be an issue for say Slavic languages but German compounds where you can actually append a stem on a stem and a stem and a stem on a stem right before you know it you have like 40 letters in your word and go parse it out and go parse the meaning of it out.
• 28:06
  SPEAKER_03: Right.
• 28:07
  SPEAKER_03: Interesting that's fascinating so one of the things that we sort of touched on uh but not didn't get into but information architecture and its relationship to language translation so when when content coming into the pipeline is fragmented, inconsistently structured using different terminology across systems right that's all that's information an information architecture problem.
• 28:31
  SPEAKER_03: So what does good information architecture look like as a foundation for translation?
• 28:36
  SPEAKER_02: And then what happens when it's missing it's in equal part as I said it's an equal parts and engineering issue right because if you cannot parse out the input properly right if the input cannot reduce itself to a place where a model can handle it properly that would probably be the first breaking point right if you cannot reduce it to like say JSON or there is a standard in our industry which is a flavor XML called XLIF, if you cannot parse your content out properly and feed it into the system, that would be like that's one breaking point.
• 29:09
  SPEAKER_02: I will use for instance subtitles right subtitles back to your initial comment about use cases which are good use cases for like good enough subtitling speeches, subtitling presentations that's a great one.
• 29:21
  SPEAKER_02: Until you start thinking that like a format subtitling format like SRT, it's actually what is it?
• 29:26
  SPEAKER_02: It's a sentence but then it's also chunks within the sentence right human intonation prosody a human made and before you know it you're not dealing with a sentence or segment you're actually dealing with completely random chunks.
• 29:37
  SPEAKER_02: So that part of architecture you actually need to be able to stitch it back together, make it understandable but then chunk it to a place where subtitles actually with toggles and timestamps and everything else where subtitles actually usable for your generated or translated content.
• 29:54
  SPEAKER_02: So these are the things that you need to think through easier content engineering wise suitable for natural language processing whatever direction it takes rendering understanding natural language understanding natural NLU NLG NLP and all of those and this is where again a lot of people very much underestimated as I said like a bunch of nested HTML tags if you have not figured out a way of filtering them out properly that's gonna break things completely.
• 30:19
  SPEAKER_02: So it's data integrity not just from linguistic perspective but it's just pure hygiene and governance in terms of storage your database architecture.
• 30:29
  SPEAKER_02: If you build homegrown systems and just came up with your own standard you're on your own go build it.
• 30:35
  SPEAKER_03: Nobody can be there to help it's so interesting how it all does keep you know boiling down to structure and architecture and you know I just spoke at a a an event held by the conference board they have uh a KM council and one of the things that uh they're dealing with the folks that are leading knowledge management initiatives in these large enterprises is hey we have AI why do we need KM right why do we need knowledge and and at the end of the day you know AI is not going to work if you're you know if your knowledge is not there or if it's junk or if it's you know if it's it's messy or if it's unstructed you know if it's if it's not uh curated if it's not organized if it's not chunked if it's not tagged right it's all that and it's interesting to see that translation is coming down to the same fundamental principles.
• 31:22
  SPEAKER_02: And think about how much your model can look can learn from a simple ontology right if you have a simple ontology with representation of all the dependencies it definitely informs your model in a much better way than just like you know random random soup of unstructured content.
• 31:36
  SPEAKER_02: And that's where I'm saying marrying the two schools of thought is probably the most promising field right now.
• 31:41
  SPEAKER_02: The problem is that if you don't have any foundation build building a knowledge is extremely expensive.
• 31:45
  SPEAKER_02: Right.
• 31:45
  SPEAKER_02: Right even if you do it in the most automated way.
• 31:48
  SPEAKER_03: Well yeah and it can be some of it can be accelerated but there is yeah you're right there's a there's a lot of challenges but but you know there can be some AI tools that will accelerate that but you still need the right structure you still need the right frameworks you still read need the right methodologies and examples and so on.
• 32:04
  SPEAKER_03: So it certainly can accelerate things and so you we're just talking we were just talking about this so you know a a hybrid approach that combines LLMs with structure knowledge with ontologies with small language models rule based systems when does each layer kind of earn its place and how do you decide what that right combination is and again when I say small language models you know people it it even when you're building your ontology and you're building your taxonomies you know that is a it's a subset of language model that is more fine-tuned to your context right your domain in your world right so that it gives additional context so talk a little bit more about how those things work together and how they build.
• 32:47
  SPEAKER_02: Let me think about it.
• 32:48
  SPEAKER_02: So I mean first thing maybe we could get small language models out of the way first because that's the easiest one.
• 32:53
  SPEAKER_02: That's the lowest hanging fruit right when you look at frontier models they are jack of all trades.
• 32:58
  SPEAKER_02: Yeah they can do thousand things so what we when here at SmartLink what we uh experimented quite a bit with and getting a lot of successes with is actually build a purpose built model that is suited specifically for language tasks.
• 33:10
  SPEAKER_02: And luckily being a platform we obviously have enough performance data right and we have enough of actual linguistic data to inform that small model to actually do what it's what what it should be doing.
• 33:22
  SPEAKER_02: It can be customer specific, domain specific or it can just be more generalized but still the expectation is that this model is there for translation.
• 33:30
  SPEAKER_03: Translation so it's specialized to the task but then it can also be subspecialized to the to the domain to the industry to the client world right even if you're in the same industry and you do things differently you name things differently and so on.
• 33:43
  SPEAKER_03: Yeah.
• 33:43
  SPEAKER_02: Yeah you can funnel in I mean you can funnel in as as as much as you wish.
• 33:46
  SPEAKER_02: So I think small language models again they would have their place because they would probably require much less effort in for in terms of getting them focused on what matters and in our case global content transformation.
• 33:59
  SPEAKER_02: Right.
• 33:59
  SPEAKER_02: Because all the dances like all the tribal dances that you need to perform around frontier models they're much more complex because you need to extract a specific task out of a multitask model right and this is where and this is where everything can potentially break or you need to fine tune fine tuning still works although we just had meetings with most of the major frontier model providers we just did a tour around different conferences it looks like fine tuning is still a thing but it's giving RAG now we're going to get to your knowledge management.
• 34:29
  SPEAKER_02: It's giving way to rag or hygenic retrieval which is not like basically next generation of RAG right because fine tuning again it's expensive it can be time consuming but you still need to curate your data if you want to fine tune your smaller model or fine tune your fine model.
• 34:44
  SPEAKER_02: So data curation still has its place.
• 34:46
  SPEAKER_02: But for retrieval and now we've gotten to the retrieval part retrieval is only as good as your linguistic assets are and the structure of your linguistic assets right because if you're giving it examples and it's retrieving garbage you're not getting anywhere.
• 35:01
  SPEAKER_02: So that's where we work with again I'm speaking from smart link perspective because that's what I do day in and day out we absolutely work with our customers on data governance data curation data structure data organization and help them it's a prequel right the prerequisite you need to curate your data right you need to have your data governed you need to have it in a structured fashion you need to have either semantic depend semantic relations or you need to have very clear taxonomy whichever comes first if you have enough metadata obviously you can get by with the taxonomy if you don't have enough metadata you obviously need to build up semantic relations but that's the ultimate foundation yeah data for retrieval or fine tuning you're not gonna get anywhere without it so I would say prerequisite then data manipulation whatever you've been able to do with different prompting techniques or older school like we do for instance we still use quite a bit of elastic search in different applications that we use like keyboard extraction.
• 35:55
  SPEAKER_02: So I think data governance figuring out the right tooling you said in the beginning of this conversation something that really resonated with me.
• 36:02
  SPEAKER_02: Don't do it if you don't need to do it right or I'm paraphrasing you in the most maybe not necessarily most elegant way but I think one of mistakes people make when they implement AI is basically art for art's sake.
• 36:13
  SPEAKER_02: Don't touch what's not broken.
• 36:15
  SPEAKER_02: Right?
• 36:15
  SPEAKER_03: Right, right right simpler is better and if you can get away with it great uh but if your use case demands it and your and your data your validation your metrics show that you you cannot satisfy the use case unless you have these prerequisites that's when you have to really focus on building that foundation I like what you refer to as tribal dances.
• 36:35
  SPEAKER_03: Have the tribal dances you have to go through to deal with these large language models and to subset them out and to get them to specialize and do the things you want right and knowledge knowledge provisioning and methodology provisioning is all part of that to give that context and then those smaller language models which represent your world so that really has application to everything right and they can work in a chord again sometimes you can get whatever you need the basic you can get it from the frontier model and then you augment it with a second step secondly or a small language model which is task specific so and again it's a matter of funneling in funneling into your task.
• 37:13
  SPEAKER_03: Yep great great and when you start thinking about you know an AI first translation workflow how do you integrate uh human review right that's still essential especially with higher risk things how how are you making those decisions based on risk and and how are they uh integrated into the workflow?
• 37:33
  SPEAKER_02: Uh let me come back to you with a question.
• 37:35
  SPEAKER_02: Do you see where major AI developers are spending most money right now?
• 37:39
  SPEAKER_02: I mean okay data centers and GPUs aside what is the second hugest line item for them right and the hugest what what do you think of it?
• 37:46
  SPEAKER_02: Go ahead go ahead you you're you're asking it as a rhetorical question but go ahead with a rhetorical I was curious but anyways uh it's data it's developing human curated data sets right ton of money is poured into because why we've already scrapped everything we could right the data drift and the data does not evolve as fast as the need for the models that are hungry for high quality curated data.
• 38:07
  SPEAKER_02: So the human in the loop is now I mean they're making billions of money just putting human labels and human annotators in place.
• 38:14
  SPEAKER_02: Now think about our universe if you want to ship on brand high quality content on brand high quality you absolutely need a human review and human correction.
• 38:25
  SPEAKER_02: And we have the content sensitivity and risk tolerance matrix which is actually divided by markets and I just did a presentation with you should probably not mention the client but I just did a presentation at a conference there where we spoke about AI positive and AI negative markets.
• 38:40
  SPEAKER_02: So it's the market tolerance and then it's the content type tolerance.
• 38:43
  SPEAKER_02: And then you phase in the human review which can be light human review for spot checking for more glaring errors or you phase in human review where you really have super high sensitivity to the quality of the content.
• 38:56
  SPEAKER_02: Now you can still obviously and huge chunk of our business is just fully automated translation, fully automated AI translation and it works and because we deliver bespoke solutions, as I said we're using RAG, we're using customer specific linguistic assets, it's higher quality than what you can get out of the foundational frontier model.
• 39:15
  SPEAKER_02: Still if you think about it, without at least a little bit of content coming from human review, how are you expecting your models to evolve?
• 39:22
  SPEAKER_02: Right, right if you don't have human reference and that's why it's like for instance we're run we run this tool called LQA agent which is basically automated language quality linguistic quality assurance agent.
• 39:35
  SPEAKER_02: But the best way for the model to learn is to make sure that the model agrees with direct assessment by humans.
• 39:41
  SPEAKER_02: And there you are no humans no direct assessment right yeah go ahead sorry no no it's very very simple right if you want your models to evolve and you if you want your models to learn yeah you obviously need somewhere for these models to learn from and this is where a human review it's always twofold it's guaranteeing the quality to your customers and helping your models evolve.
• 40:01
  SPEAKER_03: Right and it and it's like ground truth for rag right where does ground truth come from it comes from from somebody saying that's a good answer.
• 40:10
  SPEAKER_03: Correct not correct you know you do need to have an LLM check the LLM because you will have model drift right and you'll have uh you know LLMs uh to not not stop following your directions quietly silently and silently fail.
• 40:22
  SPEAKER_03: So you absolutely do have to have that it's it's really interesting.
• 40:25
  SPEAKER_03: I mean I we've see that quite a bit and that's why we need LLMs to judge the output of LLMs but humans to judge the ground truth that goes into the assessment right the ground truth needs to come from somewhere and maybe the volume of the ground truth is shrinking the need right because you need zero like I mean you can get by with like a single shadow or like one shut or like zero shots uh zero shot references.
• 40:47
  SPEAKER_02: So maybe the volume of that ground truth need can shrink but you cannot completely get rid of that.
• 40:52
  SPEAKER_02: Yeah absolutely an interesting paradox I just want to say one one quick thing when it comes to judgment what we find out is humans to get the agreement between human and the and the model is half the problem.
• 41:05
  SPEAKER_02: The real problem is the subjectivity of human reviewers and human annotators not only do they not agree between themselves if a reviewer or an annotator can be in a good mood on Monday and in a bad mood on Wednesday and that same person would judge the same output in a completely different way.
• 41:22
  SPEAKER_02: And one of the things that my team works a lot on okay how do we extract the most objective if objectivity can even be applied to human judgment how do we extract the most objective ground truth set to train the model of what's hypothetically is true.
• 41:37
  SPEAKER_03: Right, right that is fascinating because you're right you know one of the things about you know say machine tagging is they may get it wrong but they get it wrong consistently right humans get it wrong inconsistently or get it right inconsistently and yes there's a lot of subjectivity there and it does depend on did you have your double shot of uh espresso this morning or did you were you too late to uh your regular barista wasn't in I yeah yeah yeah but you'd be surprised actually sometimes the model what we come across and that's why it's important to have second model judged output of first model model can also be inconsistent in its judgment like we have a technique called feedback loop where the model says this was a mistake correct it you go and correct it and then the model goes no no no what did you just do actually it was correct now it is a mistake so to mitigate the model hallucinations and as you said like model can get a mind of its own and start making decisions and stop obeying your destruction your instructions the whole hallucination mitigation and model predictability that's another thing that's essential to get right reasonable output and reasonable judgment out of them.
• 42:39
  SPEAKER_03: It's another metric that you need to look at and and being able to establish all of those different metrics to say where you know where are the boundaries when do you escalate when what how do you measure hallucination how do you measure groundedness citability all of those things are are part of that.
• 42:55
  SPEAKER_03: Yeah absolutely so very good to have you thank you so much for joining us and sharing your 25 years of expertise in this field.
• 43:03
  SPEAKER_03: It was a fascinating discussion.
• 43:05
  SPEAKER_03: Any final thoughts for executives or or technology and business leaders as they look at this space?
• 43:12
  SPEAKER_02: Okay so I was asked the same question two years ago or three years ago and at the time my response was that right now we're like this everybody's in love with LLMs and what they can do.
• 43:23
  SPEAKER_02: And then it was implement and fail 85% of time as we know from the statistics.
• 43:28
  SPEAKER_02: And now I think we're finally in a place of measured deployments.
• 43:32
  SPEAKER_02: So measure right as we say in Russian measure seven times cut cut once so measure understand that you have the real use case understand the use case and as we were talking throughout this conversation do not underestimate the complexities yeah your engineers you think you have the most brilliant engineers which I'm sure you do but do they speak all the languages do they know all the intricacies that we you and I just spoke about not necessarily so RY use case and don't underestimate the complexities.
• 44:02
  SPEAKER_02: Wonderful don't pour your time and money into something that experts can do for you.
• 44:06
  SPEAKER_03: Wonderful well thank you again appreciate you joining us and it really helps to understand the the level of nuance that that goes into something like this and you certainly open my eyes to a lot of those details.
• 44:18
  SPEAKER_03: And to our listeners thank you for tuning in to the Early AI podcast.
• 44:23
  SPEAKER_03: Be sure to subscribe for more conversations about how AI is shaping the future of business.
• 44:29
  SPEAKER_01: Again Golga thank you for your time today thanks so much for having me thank you for joining us on another deep dive into AI innovation.
• 44:37
  SPEAKER_01: Tune in next time when we introduce another industry expert and discuss how to maximize AI in your world the Early AI podcast is sponsored by Early Information Science and Vector.
• 44:49
  SPEAKER_01: To learn more visit early dot com that's earl e y dot com.
• 44:54
  SPEAKER_01: Thank you for tuning in