Category: Blog

How a simple Ollama front-end grew into a professional-grade, fully offline translation environment

Not long ago, working with a large language model for translation meant opening a terminal, typing a prompt, waiting, and copying the output by hand. The arrival of Ollama made it straightforward to run powerful open-source LLMs on a personal computer, but interacting with them remained a largely technical affair. The Local AI Translator, developed by Terence Lewis with support from Philip Staiger  and available at LocalAI.World, set out to change that — and the distance it has travelled since its first release tells a compelling story about what offline AI tools can become when they are built with the practising translator firmly in mind.

The Starting Point: An Ollama Wrapper with Ambition

The earliest version of the application was, in essence, a graphical wrapper around Ollama — a way of sending text to a locally running LLM and receiving a translation back, without needing to touch a command line. That was already genuinely useful. A translator could type or paste a passage, choose a model, and get a rendered result in a clean desktop window. No API key, no subscription, no data sent to a remote server.

The privacy dimension was central from the outset. Sensitive corporate documents, legal texts, medical records — content that organisations are rightly reluctant to upload to cloud services — could now be handled entirely on the user’s own machine. The application runs on any Windows 11 PC with 16 GB of RAM, with no GPU required, making it accessible to professionals on standard hardware.

Growing the Toolbox: From Single Prompt to Full Environment

As the application matured, a row of feature buttons began to define its identity. Model Manager, Prompt Manager, RAG Manager, Back-translate, Batch Translation, and Tools — each button represents a layer of capability added in response to the real demands of translation work.

Model Manager and Model Evaluator

The Model Manager allows users to download, switch, and delete LLMs directly from the GUI, with direct access to the Ollama model library. Alongside it sits a Model Evaluator — a genuinely important feature given the proliferation of models claiming multilingual capability. The evaluator lets users benchmark a series of LLMs against each other for a specific language pair using standard metrics, so they can make an informed choice rather than guessing which model will perform best for their use case. Each evaluated translation can be saved with its model name for later comparison.

Prompt Manager and RAG Manager

The Prompt Manager gives translators something that cloud tools rarely offer: the ability to write a proper brief for the model. Custom prompts can range from a handful of terminological preferences to comprehensive style guides or extensive domain-specific glossaries. The RAG Manager goes further, enabling Retrieval Augmented Generation in three modes: a flat glossary import, a knowledge graph built from a source text and an existing translation memory, and import of an existing knowledge graph in JSON, TTL, or GML format. A simple checkbox toggles RAG on and off, so the user can compare results with and without augmentation in real time.

File Translation and Batch Processing

Beyond screen input, the application can translate text files at sentence, paragraph, or document level — a meaningful distinction, since paragraph and document-level processing gives the model the broader context it needs for consistent terminology and register. MS Office files, LibreOffice files, and PDFs are all supported. A Batch Translation module handles multiple files simultaneously, across any mix of formats and language pairs.

The Tools Suite

A Tools menu provides ten additional utilities: a Model Supported Language Checker, an AI Chat box for ad-hoc research and glossary generation, an Image Analyser capable of intelligent OCR and image reasoning, a Bitext Creator, an XLIFF Generator, a Transcriber for audio and video, a Rewriter, a Proofreading and Correction tool, a Terminology Extractor, and a Summarizer. These utilities are designed to orbit the central translation task, supporting it from multiple angles rather than functioning as standalone products.

The CAT Tool Paradigm: Interactive Mode

The most significant step in the application’s evolution has been the introduction of Interactive Mode — the point at which it stopped being a translation generator and became a translation environment. Activating Interactive Mode brings up a segmented grid that will feel immediately familiar to anyone who has worked in Trados, MemoQ, or any other CAT (Computer-Assisted Translation) tool.

The grid displays source and target segments side by side, at sentence or paragraph level as the user chooses. The LLM generates a proposal for each segment, but the translator remains firmly in control: any proposal can be edited directly in the target cell, or discarded entirely and replaced with a human translation. The model can even be changed mid-session, allowing the user to call on a different LLM for a particularly difficult segment without abandoning the current project.

Three Proposals, Three Models: A New Kind of Quality Control

One of the most distinctive features of the Interactive Mode is the ability to present the translator with multiple proposals for each segment. In one configuration, a single model generates three alternative translations, giving the user a choice of renderings and a basis for selecting or synthesising the best option. In another, three different LLMs each contribute their own proposal for the same source segment.

This multi-model approach is a genuine innovation. Different LLMs have different strengths — in fluency, in terminological precision, in handling idiomatic language — and presenting their outputs side by side turns model variation from a problem into a resource. The translator can pick the proposal that best fits the context, or use the spread of suggestions as a prompt for their own formulation. It is, in effect, a form of AI-assisted quality control built into the translation workflow itself.

CAT-Compatible Output: Bridging the Offline and Professional Worlds

A translation produced in Interactive Mode does not have to stay within the application. Once the translator is satisfied with the segmented bilingual content, it can be saved in several formats designed for onward use in professional workflows.

Output formats include:

• XLIFF — the XML Localisation Interchange File Format, the lingua franca of the translation industry, importable directly into Trados, MemoQ, Wordfast, and most other professional CAT environments
• TMX — Translation Memory eXchange format, allowing the completed bilingual content to be added to an existing translation memory in any compatible tool
• DOCX — as a standard or bilingual Word document
• ODT — for LibreOffice compatibility
• TSV and plain text for lightweight downstream processing

The XLIFF and TMX outputs are particularly significant. They mean that a translator can use the Local AI Translator to produce a first-pass bilingual draft entirely offline, then import it into a commercial CAT tool for terminology checking, quality assurance, or client delivery in a familiar format. The offline and professional worlds, often seen as separate, become directly interoperable.

Accessibility and the Global South

The application’s developers have been explicit about one of its core motivations: making professional-grade AI translation tools available beyond the markets where cloud services are cheap and connectivity is reliable. For researchers, educators, and translators in countries where API costs denominated in US dollars represent a significant burden, where internet access is intermittent, or where data sovereignty concerns make cloud processing inadvisable, a one-time purchase of €149 that runs entirely locally is a materially different proposition from a monthly subscription to a remote service.

Once a model has been downloaded, the application requires no connection at all. That single characteristic — complete offline operation — is not a minor technical detail. For a significant portion of the world’s translators, it is the difference between access and exclusion.

Conclusion

The Local AI Translator has travelled a long way from its origins as a graphical front-end for Ollama. It now offers a translation environment that competes meaningfully with commercial CAT tools in its core workflow, while adding capabilities — multi-model proposal comparison, knowledge graph RAG, integrated back-translation, offline audio and video transcription — that most commercial tools do not provide at any price.

Its development reflects a broader truth about offline AI tools: the constraints of local deployment, rather than limiting what is possible, can focus development on what actually matters to the people doing the work. By keeping the translator at the centre of the process — in control of every segment, able to choose between proposals, able to bring their own judgement to bear at every point — the Local AI Translator has arrived at something more interesting than a translation machine. It is a translation workbench.

What is neural machine translation?  Well, according to Wikipedia, “Neural machine translation (NMT) is an approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modeling entire sentences in a single integrated model”.  The stand-out phrase here is “predict the likelihood”. As a young translator, I was always under the impression that when I translated sentence A into sentence B I had to be certain that sentence B conveyed the meaning expressed in sentence A.  If I ever told my project manager that my translation was likely to convey the meaning of the original,  I would probably soon have found myself looking for a new job!

In the early days of machine translation, the translation was produced through the application of a very large number of rules. The greater the complexity of the languages involved, the more granular were the rules that governed the translation process.  In theory, if all the necessary rules were applied and all the words in the source text were contained in a bilingual custom dictionary and in a bilingual general dictionary,  you could be reasonably confident that the rule based MT system would produce an appropriate, if somewhat wooden, translation.  

Neural machine translation does not deploy a huge number of hand-crafted rules. Instead the rules enabling the model to translate from A to B, or to predict an output from an input sequence of tokens, are learned by the model from the data itself.  Having sufficient “clean data” in domains for which the neural MT system is used is half the battle when it comes to building confidence in securing an accurate translation. Of course, the model will be unable to generalise from sequences of tokens it has seen nowhere during training.  The inability of a model trained solely on the bible and other religious texts to translate simple sentences like “My child is sick, I need to see a doctor” underscores the indispensability of data pertaining to the domains or fields of human experience for which we wish to apply the model.  For developers working with “low resource” languages a lack of real-world data poses a challenge which is being met with a variety of innovative approaches.

Over the years since the appearance of rule-based MT in the early 1950s,  various metrics have been developed to measure the accuracy of machine translation systems, the best-known being the Bilingual Evaluation Understudy (BLEU) algorithm, which is probably the main starting point for developers seeking to establish just how good their systems are.   When I was a schoolboy our knowledge of Latin and Greek was put to the test by having us translate “unseen” passages from the works of classical authors.  Nobody could memorize the translations of every classical author so the test challenged us to generalise from our experience of the works of the authors on our syllabus and make a fair fist of rendering our text into English. We were expected to exercise creativity to deal with the odd unknown word in our text.   A well-trained neural machine translation model that has not “over-fitted” or simply memorized the training data will produce a varyingly successful translation of the unseen test set, as evidenced by whatever is commonly accepted as a good BLEU score.  Byte-pair encoding and other sub-word techniques will reduce the number of unknown words.  Automatic evaluation metrics such as BLEU, NIST, METEOR, WER, PER, GTM, TER and CDER help researchers and developers to determine how successfully their model has been trained.  Taking an NMT model into production in domains for which it has been trained is therefore definitely not a leap in the dark.  Then  professional translators sometimes make mistakes, and translation software makes different kinds of mistakes.  A critical eye is always needed, however the translation is produced.  To go back to our original question “Can we have confidence in neural machine translation?”,  the answer is that with reservations we probably can.

Communication is the key to successful business relationships, especially nowadays when the reality of the working world and the globalized professional market has forced a metamorphosis and language skills can no longer be taken lightly. Post-covid, physical location (city, country, or even continent) is no longer a constraint. Your employees  are in contact with  foreign colleagues and customers daily, both virtually and in person. Therefore the company’s energy should be focused on the realization, implementation, and success of the project rather than on the efforts invested and time wasted to ensure a good understanding and overcome language barriers.  

Advanced AI based language technology is a key tool for achieving these aims.   Its implementation can take the form of remote interpreting, machine translation (in the cloud or “on premises”),  enterprise chat translation or translation on a handheld device. To be effective, it is essential to diagnose the language strengths and weaknesses within the organization. A language technology audit  allows an organization to identify those areas where language technology  can assist employees in their communication with foreign colleagues and customers.  These assessments look at all a  company’s activities which involve oral or written interactions with foreign customers, partners and colleagues and highlight those areas where employees could be assisted by the use of AI based language technology. Requirements for language technology will vary from one business to the next.  A customer support organisation may benefit from a system that directly translates communications in a chat environment.  A company that needs to scan huge volumes  of data in many languages every day will be looking for a machine translation system that can process millions of words in an hour.  A scientific research organisation would be well served by a neural machine  translation system designed to translate scientific documents in a specific branch of science to a “near human quality” standard.  Requirements vary and there is an  exciting range of language technology tools to meet these requirements.  Our language technology audit is designed to identify such requirements within your organisation. You can purchase a voucher for a language technology audit in our “Knowledge Shop”.

Downloading pretrained Hugging Face translation models, fine-tuning them with new datasets and conversion to OpenNMT’s CTranslate2 inference engine – that seems to be the most cost- and energy-effective way to build new models for low resource  language pairs where gathering data is a true treasure hunt. I’ve just fine-trained the Opus-MT Oromo-English pair. Oromo is a Cushitic language spoken by about 30 million people in Ethiopia, Kenya, Somalia and Egypt, and is the third largest language in Africa. Despite the large number of speakers, there are very few bilingual written materials in Oromo and English. I managed to pull together some three thousand new sentences from human-translated documents and fine-tuned the Opus-MT pair in both directions. This fine-tuned model has been converted into the CTranslate2 format and is now available on my free translation site at http://nmtgateway.com. The results still leave much to be desired, but the fine-tuned model could be useful at a very basic level. For the other language widely spoken in Ethiopia – Amharic, the official language with some 25 million speakers -, I managed to gather around one million sentence pairs from a variety of sources and trained models with the OpenNMT-tf framework. Again, at the level of simple sentences, like “The army delivers clean water to all the villages in the region”, the English-Amharic model generates useful if not perfect translations, and it makes a good job of a health-related sentence like “The government is introducing measures to stop the spread of the virus”. The Opus-MT Oromo<>English models were trained on the (limited) Opus data. As I found with my Tagalog<>English experiments last year, we seem to need around one million sentence pairs to get usable translations of simple sentences. The “zero-shot” road is one on which I have yet to travel!