In detail
1. The failure is fluent, which is what makes it expensive
Olga Beregovaya, VP of AI at Smartling, describes a case that will not show up in a readability check: an Italian researcher receiving text that was smooth Italian, while the phenomena being described were “100% American culture centric.” Her verdict is that “you do lose on factual accuracy and anthropological and phenomenological local phenomena.” (6:54)
Coverage drives it. “the less covered and less represented the languages in the model, the higher the odds are that biases will be introduced.” (4:06) For a company that, in her words, covers “every language under the sun,” that is the whole problem rather than an edge case.
She also pushes back on the assumption that the newest tool wins: “Quite often, neural machine translation would do much better than a large language model would do.” (6:54, 7:16)
2. Two counterintuitive findings from production
Test the prompt language. The instinct is to have a native-speaking linguist write the prompt in their own language. Beregovaya reports the opposite result: “you quickly find out that a carefully crafted prompt in English will render a better result in the target language.” (7:39) Treat that as a finding to check, not a rule. A NAACL 2025 study of multilingual tasks found that translating inputs into English was not universally better: behavior varied across models and tasks, and native-language prompting did better on culture-related ones. That study did not test the language of translation instructions, so check it on your own work: compare English and target-language instructions on the same held-out examples, and have bilingual reviewers judge the output.
Early tokenizers set a floor. Before model quality was even in play, the text had to segment. Describing the early days, she says “if you feed Swahili and all you have at your disposal is an English tokenizer, it’s not gonna get you far. It’s gonna give you the most disastrous results.” (14:05) She welcomes the wider availability of tokenizers for different languages, and later says models are “getting really, really good at being able to vectorize multiple languages.” (18:36) Her phrase for shipping into a language the model cannot reliably vectorize is “upload and pray.” See tokenization for the mechanism.
3. Retrieval is what makes the style guide affordable
Localization carries a large body of reference material: style guides that “can be a 100 pages,” translation memories, dictionaries, do-not-translate lists. Stuffing that into every prompt is, in her words, a “waste of fortune, make no money.” (15:22)
So the pipeline uses retrieval to “only fetch what’s relevant from external sources,” (15:22) which she credits as a game changer. It also narrows the failure surface: grounding output in a curated external source is how she describes “mitigating biases and hallucinations.” The same reasoning appears in what is RAG.
Worth noting what her team optimized, because it was not the model: “queuing, caching, multithreading, and everything else that helps reduce inference time.” (13:02) A perfect prompt that takes five minutes to run, as she puts it, is “obviously not gonna fly.”
4. Where the human actually sits
Beregovaya’s division of labour is explicit: “AI does the heavy lifting, introduces predictability, does a lion chunk of the work, takes us much closer to human parity than ever before. And the human translator or fact checker or validator really compensates for the delta that AI is not able to cover.” (10:44, 11:09)
The economics are why it matters. Her industry baseline is that “On average, a translator can deliver 2,000 words a day.” (11:09) On the gain from AI she is deliberately imprecise, saying only that it is a matter of “magnitude and order of magnitude” (11:09) rather than quoting a multiple.
Fully unattended AI translation does work, but she scopes it to a specific shape of content: shorter shelf life, higher risk tolerance, instant turnaround. Opinion portals and forums are her examples. The interesting design decision is upstream of translating anything, which is routing: does this piece need a human translator, AI with human validation, or machine translation alone?
5. Multimodal collapsed steps that used to cost money
Two changes she flags as real. Localizing images used to mean desktop publishing: extracting text, translating, rebuilding the artwork. Now “all you really need to do is just say reproduce this image in a target language.” (33:03)
And the speech pipeline shortened. Translating spoken audio “took four steps”: recognize, translate, render, synthesize. A multimodal model can now fit that in one, and it has reached ordinary users: Google Meet and the Google Translate app now translate live speech into audio that keeps something of the speaker’s tone and pacing. Her reason for caring is compounding error: “every time, every step you introduce, you introduce potential fine error.” (33:56)
A worked check for meaning
Take a shipping instruction: “Ship no more than 10 boxes. Keep below 5°C. Returns are allowed within 14 days.” Here is what a fluent but faulty translation could say, glossed back into English:
| Source says | Faulty translation says | What changed |
|---|---|---|
| No more than 10 boxes | At least 10 boxes | The maximum became a minimum |
| Below 5°C | Above 5°C | The storage condition reversed |
| Within 14 days | After 14 days | The return window reversed |
Every number survives and every instruction flips, so a check that the numbers match would pass it. Give a qualified reviewer the source, the full document, the terminology list and the intended audience, and have them mark each changed span and what it would cause. That is the approach of Freitag and colleagues’ MQM study: professional translators with the full document in view labeled explicit errors in WMT 2020 systems, and their ranking of those systems differed from the one WMT’s crowd workers produced. An overlap metric such as BLEU can track a fixed test set over time, but it cannot clear instructions like these.
Why it matters
If you are judging AI translation by whether it reads well, you are checking the thing that stopped being the problem. The remaining failures are factual, cultural, and concentrated in exactly the languages your quality checks cover worst. Beregovaya’s condition for the human leaving the loop is specific: “until biases and hallucinations have been completely solved for and until we have even language coverage.” (38:41)
Hear it from the guest
“And the human translator or fact checker or validator really compensates for the delta that AI is not able to cover.”
“If you feed Swahili and all you have at your disposal is an English tokenizer, it's not gonna get you far. It's gonna give you the most disastrous results.”
Quotes lightly edited to remove filler words.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
Concepts in this explainer
TokenizationBLEU ScoreRetrieval-Augmented Generation (RAG)AccuracyGroundingInferenceLarge Language ModelMultimodal AI