It’s no exaggeration to say that the online world is obsessed with ChatGPT. Since launching in November 2022, the AI-powered chatbot has made quite a splash. It’s both impressed and ruffled the feathers of business leaders, educators, and scientists, among others. While concerns exist about its factual accuracy as well as the ethical implications of how it may be used by students, there’s no disputing that it’s propelled the value and use of AI into a larger spotlight than perhaps ever before.

The conversation around ChatGPT has extended to just about any conceivable use case—including machine translation.

Can ChatGPT translate languages?

Yes, ChatGPT can translate content between languages. Just like Google Translate, ChatGPT generates these translations in real time and can translate content even when English is not the source or target language.

The below example shows GPT translating a sentence from English to Spanish, then Spanish to French, then French to German.

ChatGPT can also translate content into multiple languages at once. This essentially automates the process of translating content into multiple languages via Google Translate, which requires toggling the target language each time.

As a result, translating content into multiple languages at once is much more efficient in ChatGPT than it is in Google Translate.

Is ChatGPT a good translator?

ChatGPT’s translation quality has created quite a stir in the language and localization community. Most recently, Tencent compared the quality of translations from ChatGPT against leading machine translation technologies including Google Translate and DeepL. What this initial study found is that ChatGPT performs at a level on par with these other translators, particularly when translating between European languages for which there is a significant amount of data.

However, like other publicly available translation engines, ChatGPT struggles when tasked with low-resource languages, for which the corpus of data to reference is smaller. Further, the analysis concluded that ChatGPT’s handling of informal or user-generated content, such as that seen in Reddit comments, left much to be desired.

Ultimately, the findings boil down to an unsurprising conclusion: when trained on similarly massive datasets, an NLP tool like ChatGPT can produce roughly the same quality of translations as other such NLP-based translation engines.

Whether or not any of this makes ChatGPT a “good” translator is more complicated, however, and boils down to the intended use case of its translations.

Are ChatGPT translations reliable for use in business or customer service?

The use of real-time translation technology in a business capacity has proven to be extremely beneficial for organizations with global audiences. Rather than hiring a full team of fluent speakers to cover every possible language that customers may speak, translation technology enables global brands to use their existing team of speakers, whether they be monolingual (e.g., English-only) or limited to a set of languages, to communicate with customers in virtually any language. This reduces the cost of providing multilingual support while boosting customer satisfaction—a win-win.

Given these benefits, businesses are asking: can ChatGPT be used to fulfill this role?

Our answer is: no—not without additional technology layered on top of it to provide context. 

To explain, here is an example of a real customer support chat that a business in the sports betting industry received, and how ChatGPT translated it.

ChatGPT’s translation is almost incomprehensible, particularly in the setting of sports betting. For a customer support agent who is potentially managing multiple chats at once, such a confusing translation may completely disrupt their workflow or require additional assistance from another support team member, leading to inefficiencies across the team.

Let’s break down why this message was so difficult for ChatGPT to translate:

1. Misspellings:

There are two distinct misspellings in the customer’s query. “Ganafor” should be “ganador,” meaning winner; while “iltima” should be “última,” meaning last. ChatGPT was able to understand “iltima” anyway and translated it as “final,” but was stumped by “ganafor” and translated it as “winery.”

2. Colloquialisms:

The Spanish-speaking customer used two instances of Spanish slang. Referring to the French Open, the customer said “RG” — an acronym for “Roland-Garros,” the name by which the French Open goes in most languages other than English. In the translation, “rg” is nowhere to be seen, so even someone who is familiar enough with the industry to recognize that “RG” means French Open wouldn’t be able to make that connection. Then, we see the customer also used the term “rafa” to refer to Rafael Nadal. ChatGPT did at least translate this to Rafael, which would likely help the agent know that the customer is referring to Nadal.

3. Multiple languages:

For an extra challenge, the customer threw in an English word at the end of their Spanish message: “free.” Parsing the use of multiple languages in content can be a challenge for human translators, so it’s no surprise that it could also trip up AI. Here ChatGPT mistranslated this usage of “free” as “fee.” 

So what is the customer actually trying to say? Interpreted properly, the customer’s message is: “I have redeemed the promo code for the French Open winner with Rafael Nadal and you have not given me the latest free [reward].”

Here is how Language I/O’s technology, which takes into account brand- and industry-specific jargon, translated the sentence:

Because Language I/O technology has a glossary in place to pick up on phrases that demand context, this translation is much more clear about what the customer is trying to accomplish.

In this scenario, ChatGPT suffers from the same issues as other MT engines. Without added context, the engine doesn’t know how to handle “tricky” terminology such as slang and misspellings. Considering that messages from customers are often riddled with improper grammar and slang, as well as brand- or industry-specific jargon and acronyms, relying on a translation engine that can’t determine context is going to lead customer service conversations down the wrong path.

It’s important to note that this isn’t strictly a ChatGPT issue. Any other generative AI tool—such as Google Bard—will fall short in the same ways.

The Best Alternative to ChatGPT for Customer Service Translation

Because ChatGPT’s translations lack the necessary context required for successful use in customer service conversations, customer-centric organizations should find a solution that adds context. 

At Language I/O, we have built our technology to do just that. By aggregating the world’s leading translation engines—which, like ChatGPT or Google Bard, are trained on massive amounts of data—we are able to generate the best-in-class translation for each language pair and customer use case. We then add critical context via glossary imposition, which ensures that acronyms, misspellings, slang, and industry- or brand-specific terminology are handled properly. 

ChatGPT’s technology is revolutionary, and the use of AI in this manner will only continue to increase in relevance. But without the ability to properly parse high-context user queries, global enterprises will be better served by technology that takes into account the nuances of conversational text.

To learn more about how Language I/O can give your support team the ability to provide multilingual support at a reduced cost, reach out to us or request a demo.

Heather Morgan Shoemaker

CEO of Language I/O

With an extensive background in product and code globalization, Heather founded Language I/O in 2011. She is the mastermind behind Language I/O’s core technology, which eliminates the need to train a neural machine translation engine by dynamically selecting the best NMT engine to translate a given piece of content, and imposing company-specific terminology onto the translation.