Automated Quality Assurance at Janus: How We Validate AI Before It Works for You

Machine translation, even built on the most modern LLM models, must not only sound smart, but also be objectively qualitative. At Janus Worldwide, we are confident that artificial intelligence is not magic, but a tool whose results can be measure, compare and improve.

To achieve this, we have implemented into our AI ecosystem automatic quality assessment system (Automatic QA), fully integrated with the CAT environment memoQ. This system allows analysis of accuracy and usefulness in real time translation generated by LLM and comparing it with the reference (final) version. Let’s take a look at how this process works and why it matters for both the Janus team and our clients.

Automatic QA process

1. Data transfer from memoQ to the AI Ecosystem
   The source translation segments are exported from the memoQ project.
   They are fed into a language model (LLM), which performs machine translation.
2. Machine Translation Processing in the AI Ecosystem
   Process: Source segments → LLM → Machine translated segments.
   The generated segments are stored in the AI Ecosystem for further analysis.
3. Comparison with the final version
   Once the project is completed, the final (human-edited) version of the translation is uploaded to the AI Ecosystem.
   Machine and final translations are automatically compared by segments to assess quality.

Why is this important?

• Automation of quality assessment without the participation of editors.
• An objective comparison of different LLM models and providers, based on actual results, not subjective feelings.
• Saving time and resources: QA does not need to be done manually.
• Rapid feedback for R&D: if a model shows a decline in quality, this is immediately visible, and you can quickly adjust the prompt, model, or glossaries.

What does the client get?

• Transparency: you see real numbers and indicators, not just assurances of high quality.
• Comparative analysis: you can objectively compare which model or approach works best for your needs.
• Savings: the closer the machine translation is to the final version, the less time is required for editing, which means the cost of the project is lower.
• Improving quality without manual costs: the system itself identifies weak points, allowing them to be corrected before they become a problem.

AI translation is not an experiment. It is part of the scalable, professional infrastructure of Janus Worldwide. With our automated QA system, we transform technology into a controlled and measured instrument, which helps you not only translate faster, but also guarantee consistent quality in all languages and markets.

At Janus, AI works in the client’s best interests: transparently, efficiently, and with proven results.