LibreTranslate/LTEngine: Local AI-Powered Machine Translation with LLM Compatibility 🚀

LTEngine is a free and open-source local AI machine translation API built with Rust, designed to be self-hosted and compatible with LibreTranslate. It utilizes large language models (LLMs) via llama.cpp, offering higher translation quality but requiring more memory compared to lightweight models in LibreTranslate. While LTEngine can run solely on a CPU, using an accelerator like CUDA, Metal (on macOS), or Vulkan significantly enhances performance. The largest model, gemma3-27b, is compatible with consumer GPUs like the RTX 3090. Developed in active progress, LTEngine supports any GGUF language model and allows custom models through specified paths. Translation requests can be sent via standard POST requests, with responses including translated text or detected languages. Contributions are encouraged, and the project operates under the GNU Affero General Public License v3. Resources and setup instructions for developers are provided on its GitHub repository.

Source link