Does any LLM know everything about everything? Let’s face it – if a model hasn’t seen a term used in a certain specialised context in the training data it’s hardly likely to come up with a correct translation. Let’s take the Dutch word “heuvel”. If you’ve done the Duolingo Dutch course you’ll probably know that “heuvel” means “hill”, and the Dutch word for “hills” is “heuvelen”. But Duolingo won’t teach you that “heuvelen” also means “hump shunting”. Now, you may not know what “hump shunting” is, but I know as I spent many years translating railway engineering documents. What about ChatGPT? Does it know the meaning of “heuvelen”? Well, it tells us that “heuvelen” means “to form into hills”. Doesn’t quite fit in the railway engineering context, does it?
Getting the right translation of specialised terms has always been a challenge for translators. My younger self used to travel 60-odd miles on the train to London to visit specialist libraries in search of the right terminology. In Neural Machine Translation fine-tuning has proved to be to be the most effective way of getting a model to use correct terminology as the model learns context, and fine-tuning LLMs is certainly a viable way of injecting specialist knowledge into an LLM trained on general data. But is fine-tuning a practical solution for a user when faced with an urgent translation task in a highly specialised field on which their chosen LLM has probably not been trained? When reliable glossaries are available in digital format, Retrieval Augmented Generation (RAG) can be an effective tool for feeding an LLM with specialist terminology to be used in a translation task.
The Local AI Translator includes a “RAG Manager” which provides a process for taking glossaries and files to be translated and building a single glossary which covers all the documents for translation. This filtering capability is useful since it is unnecessary to extract, for example, all the terms related to the “railway universe” to translate a series of studies on the best design for automated level crossings. I have experimented with embeddings in educational software and our Local AI Legal Assistant relies on ChromaDB to analyze legal documents, but for technical translation I prefer to use a glossary based on verified pairs of terms and phrases or even the entries in a translation memory. The process shown in the attached video clip produces “prompt_data.json” This file is then loaded in the “load glossary” function to produce the following prompt:
prompt = f””” You are a translation assistant. Use the following glossary to translate specific terms in the provided text from {source_lang} to {target_lang}. Replace each term or phrase in the text with its corresponding translation from the glossary, preserving the sentence structure. If a term or phrase is not in the glossary, translate it using your internal resources. Do not provide explanations or comment on the translation. Glossary (JSON format): {glossary_str}”.
Sticking to the railway engineering field, an example of a Dutch-English glossary submitted to an LLM within a prompt is:
{“glossary”: {
“groepsrailrem”: “group retarder”,
“heuvel”: “hump”,
“heuvelloc”: “humping locomotive”,
“heuvellocomotieven”: “humping locomotives”,
“heuvelproces”: “hump shunting process”,
“heuvelprocesleider”: “hump yard master”,
“heuvelsysteem”: “hump shunting system”,
“heuveltop”: “crest of the hump”,
“hoofdrailrem”: “main retarder”, …
With most LLMs I have tested, it is sufficient to include the glossary in this format within the translation prompt as a {custom_prompt} for the model to know what to do, i.e. apply the terminology in the translation.
I am aware that this is a simplistic approach, an example of naive RAG. As I have mentioned above I used an embeddings based approach for the Local AI Legal Assistant which can perform analysis of complex legal documents. However, the purpose of the Local AI Translator is to provide a tool for generating LLM-based translations offline on relatively low-cost entry-level computers which would not be powerful enough to run applications like ChromaDB without serious latency issues.
This essential RAG function is offered to the users of the Local AI Translator as a straightfoward approach for ensuring greater terminological accuracy in their machine translations.
For more information on the Local AI Translator visit https://localai.world. For the video clip on Essential RAG for Translation, see
https://www.youtube.com/watch?v=S0ZzRPcm1Ts&ab_channel=LocalAIWorld
