Sarah, an executive assistant at a consumer electronics company, was tasked with managing the translation of product manuals into Chinese. She hired a reputable agency with strong credentials—but when the translated materials reached her product team, a team leader flagged quality issues after just a few paragraphs. The agency, however, found no apparent errors and asked for specifics. The team leader could only express general dissatisfaction. Two friends Sarah consulted gave conflicting opinions: one praised the accuracy, the other said it felt “translated.”
This scenario is not unique. Many professionals managing translation projects face similar challenges in evaluating translation quality. How can we assess translation quality more accurately and effectively? This guide will explore this question and provide practical methods for evaluation.
What is translation quality assessment?
Translation quality assessment (TQA) is the systematic process of evaluating translated content against defined criteria to determine its accuracy, fluency, and fitness for purpose. It goes beyond simple proofreading—TQA examines whether the translation faithfully conveys the source meaning, reads naturally in the target language, uses correct terminology, and meets the specific requirements of its intended audience and use case.
In practice, TQA typically involves qualified evaluators reviewing translations using a structured framework—such as the LISA QA model or MQM—that categorizes errors by type and severity, then calculates a quality score against predefined thresholds. This structured approach replaces subjective opinions (like the conflicting feedback Sarah received) with measurable, repeatable results.
Key translation quality assessment criteria
Before diving into the practical steps of conducting a translation quality assessment, it helps to understand the core criteria against which translations are evaluated. While specific frameworks like LISA QA and MQM define their own detailed error taxonomies, most translation quality assessment criteria fall into five fundamental categories:
- Accuracy: Does the translation faithfully convey the meaning, intent, and factual content of the source text? This includes correct word choice, terminology, and faithful representation of numbers, names, and data.
- Fluency: Does the translation read naturally in the target language? A fluent translation feels as though it was originally written in that language, with proper grammar, syntax, and idiomatic expression.
- Consistency: Are terminology and stylistic choices uniform throughout the translation? Inconsistencies—especially in technical or brand-specific terms—erode trust and cause confusion.
- Grammar and spelling: Is the translation free of grammatical, spelling, and punctuation errors in the target language?
- Adaptability: Has the translation been appropriately adapted for the target market’s cultural context, conventions, and expectations? This is particularly important for marketing and consumer-facing content.
These criteria form the foundation of any translation quality assessment model. In the sections below, we will first walk through the best practices for setting up a TQA process, then explore how established models like LISA QA operationalize these criteria into structured, scorable frameworks.
Best practices for translation quality assessment
Drawing from years of industry experience, we can identify three crucial elements for successful translation quality assessment: selecting appropriate evaluators, establishing proper quality metrics and criteria, and managing the evaluation process systematically.
1. Selecting qualified evaluators
The selection of evaluators forms the foundation for accurate and objective translation assessment. Ideal evaluators should possess the following key qualifications:
Language proficiency
- Native-level fluency in the target language
- Strong understanding of the source language
- Excellence in target language expression to evaluate fluency, accuracy, and naturalness
Subject matter expertise
- Deep knowledge of the relevant industry or field
- Understanding of technical terminology and conventions
- Ability to verify technical accuracy and appropriate context
Evaluation experience
- Track record in translation quality assessment
- Capability to provide constructive feedback
- Understanding of common translation challenges and solutions
Objectivity
- Impartial approach to assessment
- No conflicts of interest
- Ability to separate personal preferences from quality standards
Impact of inappropriate evaluators
Referring to Sarah’s case, we can see how inappropriate evaluators can complicate the evaluation process:
- While knowledgeable about the product, the product team leader lacked the linguistic expertise to provide specific feedback.
- Though fluent in both languages, the two friends lacked industry-specific expertise and assessment experience.
Selecting the right evaluators
When selecting evaluators, consider these two primary approaches:
Recruit experienced quality checkers: Conduct thorough reviews of candidates’ professional backgrounds. Implement practical assessment tests to gauge their evaluation capabilities.
Partner with professional translation agencies: This approach is ideal for multilingual projects where internal recruitment may be challenging. A professional agency can offer scalable solutions for projects with tight deadlines and ensure consistent quality across multiple languages through established processes.
The choice between these approaches depends on factors such as the project’s scope, timeline, budget, and available resources. Sometimes, a combination of both methods may yield the most effective results.
2. Establishing translation quality metrics and criteria
Establishing appropriate quality metrics and criteria is essential for achieving high-quality translation outputs. These metrics and criteria act as objective tools, providing clear guidelines for both translators and evaluators to adhere to. Without clearly defined metrics and criteria, the evaluation process can become subjective and yield inconsistent results.
Adopting mature industry quality models
For those new to translation management, selecting a proven translation quality assessment model is advisable. Referencing industry-recognized standards such as LISA QA, MQM, or TAUS DQF can be beneficial. These mature models help avoid common pitfalls and save time and costs associated with trial and error, enhancing evaluation efficiency.
In-depth investigation
To effectively implement translation quality models, a thorough understanding of each error category defined in the model and the severity levels of different errors is necessary. It is also important to understand the methods of quality score calculation and recognize the model’s limitations. A deep investigation allows for smooth model customization to better fit specific needs.
Customization of the model
Every organization’s translation needs are unique, necessitating customizing general quality models to specific requirements. Based on the translation content’s characteristics and the audience’s particular needs, it is important to adjust and optimize error categories, severities, and scoring methods. This customization helps develop translation metrics and criteria that are fully adapted to your projects.
In the next chapter, we will focus on systematically establishing translation metrics and criteria to enhance translation projects’ quality and effectiveness.
3. Effective management of the evaluation process
Effective management of the translation evaluation process is key to long-term success. Proper management ensures that evaluation activities are carried out smoothly and efficiently. Without it, evaluations may become superficial, failing to accurately reflect the quality of translations or even conceal underlying problems. Here’s how to implement scientific and effective management.
Practical process management
Onboarding period
During the initial phase of cooperation between translation service providers and clients, the risk of translation quality issues is heightened due to unfamiliarity with each other’s workflows, quality criteria, and communication methods. Common issues include:
- Translation providers may not understand the client’s content and quality standards.
- Clients may not fully grasp the capabilities or processes of the translation providers.
- Inefficient communication can lead to misunderstandings or incorrect assumptions.
A robust onboarding process is essential to mitigate these risks and enhance communication and cooperation. Both parties should dedicate more resources to understanding each other, thoroughly analyzing requirements, discussing potential issues, and implementing pilot translation projects. Experience has shown that this significantly reduces translation quality risks.
Other key processes
- Communication and feedback mechanisms: Establish effective communication channels between evaluators and translators to discuss key quality issues, explore quality improvement measures, and analyze the source text collaboratively.
- Rebuttal and arbitration: Provide translators with an opportunity to challenge evaluation results they disagree with, require evaluators to review disputed errors, and involve objective third parties for arbitration when necessary.
- Continuous improvement: Utilize evaluation results to refine translation processes and quality control measures. Analyze quality assessment statistics to identify trends and implement preventive measures to avoid significant quality issues.
Process management is a higher-level management form typically based on established evaluation practices. While not the main focus of this article, readers interested in this topic can look forward to future articles or contact us directly for more information.
It’s important to clarify that merely choosing a reputable translation company does not guarantee issue-free quality. Even top translation service providers face quality challenges; however, their advantage lies in their process management capabilities. Effective service providers swiftly address and resolve quality issues and continuously enhance their translation quality, whereas less adept providers may struggle with recurring unresolved quality problems.
Establishing quality metrics and criteria
As highlighted, establishing translation quality metrics and criteria is a complex aspect of translation quality assessment. This article aims to provide detailed guidance on effectively setting up these metrics and criteria. The process typically involves three key steps: selecting an appropriate model, customization, and gradual improvement.
Selecting an appropriate model
Let’s first look at widely used quality models in the industry.
What is the LISA QA model?
The LISA QA model is one of the most widely used translation quality assessment models in the localization industry. Introduced in the 1990s by the Localization Industry Standards Association (LISA), it remains a foundational framework for evaluating translation quality—even though LISA itself ceased operations in 2011.
Its most current version, LISA QA 3.1, is recommended for organizations in the initial stages of establishing translation metrics. The model is celebrated for its clarity and ease of customization, making it an accessible starting point for those new to this field.
This model offers a systematic framework for error assessment, designed to evaluate quality translations through standardized methods. It introduces a structured approach to classifying error categories and assessing their severity, enabling quantitative evaluation and calculation of translation quality. This structured approach ensures that quality assessments are consistent and replicable, providing reliable metrics for ongoing improvement.
Error category
The LISA QA model categorizes translation errors into seven categories. Each found error will be given an error category.- Mistranslation
- Accuracy
- Terminology
- Language
- Style
- Consistency
- Country-specific errors
- Translating “fiscal year” as “calendar year” is categorized as a Terminology error.
- Translating “approximately $500,000” as “half a million dollars” when precision is required falls under an Accuracy issue.
- Translating “Fortune 500” as “Fortune Five Hundred” when the numerical format should be preserved is a style error.
Error severity Errors are further classified by their severity into three levels:
- Critical: This may cause misunderstanding or seriously affect user operations, such as the “fiscal year vs. calendar year” example, which is considered critical.
- Major: Affects understanding but doesn’t lead to complete misinterpretation, like the “approximately $500,000 vs. half a million dollars” example.
- Minor: Minor impact, often involving stylistic choices or non-critical text errors, such as the “Fortune 500 vs. Fortune Five Hundred” example.
Scoring method
In the LISA QA framework, each severity level is assigned a specific point value:- Critical: 10 points
- Major: 5 points
- Minor: 1 point
Assessment process
Evaluators conduct a thorough review or a sample check of the translation content, marking each found error by its category and severity. Each error is then scored accordingly. The total error score is divided by the total assessed word count evaluated to determine the error rate. The final assessment result is compared against the target value set in the quality criteria.
For instance, in a review of 2,000 words, finding three minor errors and two major errors results in a score of 3×1+2×5=13, leading to an error rate of 13÷2000=0.65%. If the target error rate is set at 0.5%, this assessment would fail; if the target is 1%, it would pass.
Other models
- Standardized error classification and severity grading.
- Real-time quality tracking and feedback.
- Seamless integration with translation tools.
Unlike the models above, ISO 17100 does not directly assess the quality of a finished translation. Instead, it is an international standard that defines the requirements for the translation process itself—covering translator qualifications, revision procedures, and project management workflows. Organizations that hold ISO 17100 certification demonstrate that they follow a structured process designed to produce high-quality translations. While ISO 17100 does not replace a translation quality assessment model like LISA QA or MQM, it serves as a complementary quality signal: it governs how translations are produced, while TQA models evaluate the result.
As machine translation (MT) becomes more prevalent, organizations also need methods to evaluate MT output. Two approaches are commonly used. Automatic metrics like BLEU (Bilingual Evaluation Understudy) compare MT output against human reference translations to produce a numerical score—useful for benchmarking MT engines at scale.
Machine Translation Quality Estimation (MTQE) goes a step further by predicting translation quality without a reference translation, using machine learning to flag problem segments before human review. While these methods are designed for MT rather than human translation, understanding them is increasingly relevant for teams that use MT with human post-editing (MTPE) workflows.
Customization
After selecting the appropriate model, the next step involves making targeted customizations to ensure the model aligns perfectly with your specific content requirements. Below are customization suggestions tailored for several common content types:
| Content type | Characteristics | Assessment focus |
|---|---|---|
| Technical documents |
Contains specialized terminology in fields such as engineering, medicine, IT, and others. Translation accuracy is critical; even minor errors can lead to significant issues. It often includes visual elements like charts and tables requiring precise alignment and formatting. |
Terminology accuracy: Ensure technical terms are accurately translated and consistently used throughout the document. Comprehensibility: Despite the technical complexity, the translation should be easily understandable to the target audience, including professionals and laypersons. Layout and formatting: Translated visual elements must align with the originals, maintaining proper formatting and alignment. |
| Marketing materials |
Aimed at attracting and engaging audiences, often employing appealing phrases, slogans, and emotional language. Content is designed to resonate with the cultural nuances and preferences of the target market. Maintaining consistency in brand tone and style across languages is essential. |
Cultural appropriateness: Ensure the translation is culturally adapted without compromising the original intent, style, or humor. Creativity and appeal: Maintain the creative elements of the original content to ensure effectiveness in the target language. Brand consistency: The translation must reflect the brand's voice and uphold the established brand image. |
| Legal documents |
It contains language that could have legal consequences, such as contracts, agreements, and regulations. Language is typically formal and must convey the exact meaning of the source. Legal systems can vary significantly between countries, which must be considered in translation. |
Precision and literal translation: Translations must be exact, often favoring a literal approach to ensure the legal meaning is not altered. Consistency: Key legal terms and phrases must be consistently translated to preserve their intended impact and meaning. Compliance with local legal norms: Translators must thoroughly understand local legal terminology and concepts to guarantee legal accuracy in the target jurisdiction. |
Error category
When adapting a translation quality model, identify which error categories from the original model should be retained, removed, or merged, and determine what new categories must be added. Provide a clear description for each category to assist evaluators in correctly classifying translation errors.
For example, after identification and analysis, you might have the following types:
| Error type | Description |
|---|---|
| Accuracy | Examine whether the translation is accurate in word choice and terminology and remains faithful to the source meaning. |
| Grammar and spelling | Check for spelling or grammatical mistakes within the translation. |
| Fluency | Assesses whether the translation reads naturally and meets the expectations of target language users. |
| Consistency | Ensures that terminology and stylistic choices remain uniform throughout the translation. |
| Adaptability | Evaluates whether the translation appropriately considers the target market's culture and context. |
Error severity
Decide which severity levels to keep, adjust, or introduce. Define clear scores for each error severity level to guide the evaluation process. For example:
| Severity level | Description | Score |
|---|---|---|
| Critical | Severe translation errors that mislead users and potentially damage reputation. | 5 |
| Major | Mistranslations, omissions, or erroneous terminology that cause logical problems or unclear meanings. | 2 |
| Minor | Issues with readability, imprecise translations, or deviations from the source with minimal impact. | 0.5 |
| Not an error | Issues originating from the source text or cases where better alternatives exist but the original translation is still correct. | 0 |
Special scores
For errors that require additional attention, special scores can be assigned outside the regular scoring system, particularly emphasizing critical aspects. Such as accuracy:
| Error Type | Severity | Score |
|---|---|---|
| Accuracy | Critical Error | 5 |
| Accuracy | Major Error | 3 |
| Accuracy | Minor Error | 1 |
This approach emphasizes the importance of accuracy, encouraging translators to focus intensely on this aspect.
Evaluation criteria
Define target benchmark scores to establish pass/fail thresholds. This will categorize the results into distinct performance levels based on industry standards. For example:
| Score | Result | Pass/Fail |
|---|---|---|
| Error rate <0.3% | Excellent | Pass |
| 0.3% < Error rate ≤ 0.5% | Good | Pass |
| 0.5% < Error rate ≤ 1% | Acceptable | Pass |
| 1% < Error rate ≤ 2% | Unacceptable | Fail |
| Error rate > 2% | Rejected | Fail |
Implementing these customized metrics and criteria in an Excel template file simplifies the quality assessment process, making it more efficient and standardized for routine use in translation quality evaluations.
Gradual improvement
After the customization phase, the initial quality metrics and criteria drafts should be presented to evaluators for review. It’s equally important to involve the translation team to gather their insights. This approach underscores the dynamic nature of the process, which necessitates continuous updates and refinements based on feedback. Subsequently, the refined standards can proceed to the trial phase.
These metrics and criteria are tested in actual translation quality reviews during the trial period to evaluate their effectiveness and practicality. It is crucial to actively solicit and incorporate regular user feedback during this phase, as their insights are invaluable for continuous improvement.
It is understood that quality metrics and criteria are never perfect; they need to evolve as organizational quality requirements change. Through ongoing feedback and iterative improvements, these standards will constantly adapt to meet changing needs and serve as practical tools for driving continuous enhancements in translation quality.
Key takeaways
- Start with the right evaluators. Qualified reviewers with both linguistic expertise and subject matter knowledge are the foundation of any reliable translation quality assessment.
- Use a proven model. Frameworks like LISA QA, MQM, and TAUS DQF provide structured, repeatable methods for evaluating translation quality—avoid reinventing the wheel.
- Define clear criteria and thresholds. Customize error categories, severity levels, and pass/fail benchmarks to match your specific content type and audience.
- Treat quality assessment as iterative. Metrics and criteria should evolve through feedback and real-world application—no framework is perfect on day one.
- Process matters as much as measurement. Effective onboarding, communication channels, and continuous improvement practices are what separate organizations that consistently improve translation quality from those that struggle with recurring issues.
If you encounter any challenges during the translation quality assessment process, do not hesitate to reach out. Our goal is to support you in maintaining and improving standards to ensure high-quality translation outcomes.