Generative AI and Large Language Models (LLMs) are expected to disrupt content services industries and professions in several ways, including putting the ability to generate multilingual content into the hands of content authors, streamlining workflows, and performing translations.   

Welocalize, a pioneer of tech enablement within language services, is monitoring these forces. Language services companies have used AI to translate content for decades and understanding how LLMs compare to that technology will help inform how the multilingual services landscape will evolve.   

In a recent study, the company compared the performance of eight different LLMs and MT (Machine Translation) workflow variants, including a commercial MT system trained by Welocalize and currently in production. The company analyzed the quality of translations for customer support content from English into five target languages, including Arabic, Chinese, Japanese, and Spanish.   

According to Welocalize's findings, the custom NMT (Neural Machine Translation) models outperformed all others, both with ‘pure LLM’ output, and those combining NMT and LLMs. However, it is worth noting that, the output from both the LLM-augmented workflows as well as the ‘pure LLM prompts’ came very close to meeting a high industry standard quality level threshold, sometimes differing by mere tenths of a percentage.    

“It is particularly impressive that more challenging target languages like Arabic, Chinese, and Japanese saw promising results,” comments Elaine O’Curran, Senior AI Program Manager at Welocalize.   

Although LLMs like GPT-4 may not yet quite match the raw translation performance of highly trained NMT engines, they exhibit impressive proximity to achieving similar results.   

As LLMs become fine-tuned and work their way into the corporate IT stack, their ability to achieve desired translation results with lighter prompting and minimal task-specific training will be a compelling alternative.    

“It is easy to imagine a future where LLMs outperform NMT, especially for specific applications, content types, or use cases. We will continue to compare and analyze their performance in the coming months,” adds O’Curran. “It will also be interesting  to see the performance of customized LLMs. Similar to MTs, the idea is to fine-tune the model for a specific context, domain, task, or customer requirement to enhance their ability to provide more accurate translations for different use cases.”    

Moving Multilingual Content Generation Upstream   

The integration of LLMs into content tools and workflows could reconfigure the translation industry. Companies will be able to produce content simultaneously in multiple languages, streamlining their processes and increasing efficiency.   

LLMs represent a force of potential disruption within the translation industry. As they continue to evolve and become more accurate, this will lead to an uptick in automation and push translation and localization upstream in the content supply chain.    

For more information, visit welocalize.com.