Human Translation Outranked by Machine Translation? Not Quite

Human Translation Outranked by Machine Translation? Not Quite

Human Translation Outranked by Machine Translation? Not Quite 7360 4912 Sergio Guillén

Why Human Translation Is Still Your Company’s Best Option

Machine translation is more accurate than ever before, and the future of MT looks bright. But human translators have benefits which machines simply can’t match. While the debate over the efficiency of man compared to machine can be fascinating, machines probably have a while to go before reaching human parity.

How are Machine and Human Translators Compared?

The goal for machine translation is parity with humans. Microsoft detailed this concept in their research paper, “Achieving Human Parity on Automatic Chinese to English News Translation.”

To measure parity, Microsoft had bilingual evaluators compare errors found in human and machine translated writing. If “no statistically significant difference” was found between the two scores, the machine was said to have achieved human parity.

Has Human/Machine Parity Actually Been Achieved?

After Microsoft’s tests, machine translation ended up looking pretty good. But other researchers say to hold on.

Samuel Laubli, Dr. Rick Sennrich, and Dr. Martin Volk published “Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation.”

Their main point was that parity isn’t measured correctly. Current MT research standards call for sentence comparison.

In the Microsoft tests, the human evaluators only saw single sentences from test documents. They’d rate each sentence on a scale of zero to 100 for fluency and adequacy.

But Laubli, Sennrich, and Volk argue this single sentence rating has a lot of problems. Certain translation errors would be undetectable. Plus, human and machine translations were never directly compared.

How to Rate Neural Machine Translation

The researchers then detailed what they feel is a better ranking system. Instead of scoring sentences in isolation, their model compared full documents.

Plus, they also used pairwise ranking, which the Microsoft tests did not. With pairwise ranking, the human scorers saw two documents at the same time. One was translated by a human and the other by machine. Scorers chose the better of each pair.

The results of the test were overwhelmingly in favor of human translation. Humans were able to translate full documents with much more cohesion and fluency. However, there was no particular difference between people and machines when scoring for adequacy at the sentence level.

Were Microsoft’s Results Purposefully Misleading?

In an interview with Slator, the researches declined to blame Microsoft. “It’s not their fault,” said Laubli. “The procedure they used is standard practice in the MI community.”

Instead, the researchers say the current best practice of evaluating sentences needs to change. Machine translation has improved significantly over the past few years. Evaluating MT’s abilities based on individual sentence scoring is now out-of-date.

However, Laubli did acknowledge the title of Microsoft’s paper was “a bit bold.” When comparing isolated sentences, MT did as well as humans. But parity has hardly been achieved for full document translation.

Basically, machine translation has become too good for single sentence evaluation. MT will typically be just fine in a sentence-to-sentence comparison. But this scoring method creates invisible translation errors because scorers don’t see how those individual sentences perform in the context of a full document.

If researchers (especially industry leaders like Microsoft) say translation parity has been achieved when it hasn’t, the damage can be far-reaching.

First, funding for MT research could decline substantially. After all, why spend money if the problem has been solved?

Additionally, companies might be putting more trust into MT documents than they should. Human translation services can start to seem irrelevant if machines are supposedly just as accurate, only faster and more affordable.

The Importance of Human Translation

Machine translation is improving dramatically. But if you want to succeed internationally, you’ll need qualified human translators on your side.

According to expert translator Leila Razmjou, high-quality professional translation utilizes socio-cultural awareness. Expressions, idioms and other specific vocabulary must be understood in both languages. While machines are notoriously weak in these areas, human translators thrive.

But machine translation still plays an important role. When dealing with an avalanche of content, machines can provide accurate translations with fast turnaround time.

However, MT needs human help. A professional human translator can help decide which content can be translated by machine and which needs a more personal touch.

A machine can translate every sentence accurately but still completely miss the main point of the document. Qualified human translators play a vital role in ensuring your messaging will connect with locals.

While sentence-to-sentence parity is an impressive technological achievement, it has limited use in the worlds of business and marketing. Your potential customers need to see the big picture, which means accurate translation at the document level.

Professional human translation is still the best option for any company entering a foreign market. When selecting a translation service, the people will make a far bigger difference than any software—and that’s true in any language.

If you’re interested in learning more, check out this checklist for buying translation services.

Sergio Guillén

As the CEO of Accelingo, I believe in a globalized world where businesses flawlessly share their message across different cultures. I strive to empower startups to scale up globally and take over the world by providing compelling multilingual content that preserves their brand’s message and identity while increasing their reputation and visibility among global clients and partners. If you’re an ambitious startup hungry for international success, get in touch with me today and let’s find out how your startup can benefit from my expertise!

All stories by: Sergio Guillén