A multilingual RAG chatbot answers questions in the language your customer actually types or speaks, grounded in your own documents, even when those documents exist only in English. For Indian businesses that means handling Hindi, Tamil, Kannada, Marathi, Bengali and the rest, plus the romanized, code-mixed messages most people really send, without the answers drifting from your source content.
Why Indian-language support is harder than adding a translate button
A customer in Pune might write "mera refund kab aayega, order 4471 ka" in Latin script. Another in Chennai types in Tamil script, then switches to English for the product name. A third sends a voice note. Running these through a generic English pipeline produces poor retrieval, because the query and the documents no longer share vocabulary, and the reply often comes back in stiff English the customer did not ask for.
Three realities shape the design:
- Code-mixing is the norm. Hinglish, Tanglish and similar blends mix grammar and vocabulary from two languages in one sentence.
- Romanized text is common. Many users type Indian languages in Latin script, with inconsistent spelling.
- Source content is lopsided. Policies, manuals and FAQs are usually written in English, sometimes with a few translated pages.
If you already know the languages and channels you need, share them with our team and we can map the architecture to your user base.
Three architecture patterns for a multilingual RAG chatbot
There is no single right design. The choice depends on your languages, your corpus and how much latency the channel tolerates.
Pattern 1: Translate in, translate out
Detect the language, translate the query to English, retrieve and generate in English, then translate the answer back. It is simple and lets you reuse an English pipeline. The downside is two extra hops of latency and translation errors that compound, especially for domain terms and names.
Pattern 2: Cross-lingual retrieval
Use a multilingual embedding model, such as BGE-M3 or multilingual E5, that maps a Hindi question and an English passage close together in the same vector space. Retrieval works directly on the original query, and a capable LLM writes the answer in the user's language. This usually gives the most natural replies, but quality varies by language, so it must be measured rather than assumed.
Pattern 3: Hybrid with a normalization layer
Many production systems combine both. A normalization step transliterates romanized input into native script or produces an English paraphrase, and retrieval runs over both representations before reranking. More moving parts, but more robust for messy real-world input.
Indian-language building blocks worth knowing
- Open translation models from AI4Bharat, such as IndicTrans2, which cover the scheduled Indian languages.
- The government's Bhashini platform, which exposes translation, speech recognition and text-to-speech services for Indian languages.
- Commercial LLMs and Indian-built model families that are increasingly competent in Indic scripts; test the ones you shortlist on your own questions.
Content and retrieval decisions that matter
Retrieval quality decides whether the chatbot is useful, regardless of language.
Keep a glossary of terms that must not be translated
Product names, plan names, scheme names and legal terms should stay fixed. A maintained glossary, injected into prompts and used to protect tokens during translation, avoids answers that rename your flagship product in Hindi.
Decide whether to translate the corpus
Pre-translating your top documents into the two or three busiest languages can improve retrieval for those languages. It also creates a maintenance burden every time the English source changes, so tie translations to the source version and regenerate them automatically.
Answer in the user's language, cite the original
Show citations that link to the original source passage, even if it is in English. Support agents and auditors need to verify the answer, and paraphrased translations are hard to check.
Testing quality language by language
A chatbot that performs well in English can quietly underperform in Marathi or Odia. Retrieval and generation quality can vary sharply by language and by model, especially for lower-resource languages, so averaging everything into one score hides problems.
A sound evaluation plan includes:
- A test set per language, written by fluent speakers using real customer phrasing, including romanized and code-mixed variants.
- Separate scoring for retrieval (did the right passage come back?) and answer quality (correct, faithful, natural in that language).
- Human review by native speakers for tone and politeness, since automated judges are weaker in low-resource languages.
- Regression runs every time you change a model, prompt or embedding, gated in CI.
Our article on testing AI agents before launch covers how to turn these checks into release gates.
Channels: WhatsApp, voice and web
Channel shapes the build as much as language does. WhatsApp is where many Indian customers already talk to businesses, so message length limits, template rules and media handling all come into play; see our guide to WhatsApp chatbot development in India. Voice adds speech recognition and synthesis, with accent and background noise to consider. A web widget gives you the most layout control for citations and escalation buttons.
Whatever the channel, design a clean handoff to a human agent that passes along the conversation, the detected language and the retrieved sources, so the customer does not have to repeat themselves.
Questions to put to any vendor before you sign
When you speak with vendors, ask them to demonstrate their system on your own sample questions in at least two Indian languages, including a code-mixed one. Ask how they measure quality per language, how they protect personal data under India's DPDP Act, where conversation logs are stored and who on their side can access them. Ask whether you will own the prompts, glossaries and evaluation sets at the end.
iJurug Soft has been building software from Bangalore since 2018, and our LLM and RAG engineering services cover retrieval design, evaluation, channel integration and long-term support with senior engineers on each engagement.
Frequently asked questions
Which Indian languages should we launch with?
Start with the languages that appear most in your current support tickets, call recordings or app settings. Launching two or three languages well is better than launching ten with uneven quality.
Do our documents need to be translated first?
Not always. Cross-lingual retrieval can answer Indian-language questions from English documents. Translating your highest-traffic content can still help retrieval for the busiest languages, so test both approaches.
Can a multilingual RAG chatbot handle Hinglish?
Yes, if it is designed and tested for it. Normalization, multilingual embeddings and test sets that include romanized, code-mixed queries are what make Hinglish work reliably.
Can it run on our own infrastructure?
Yes. Open-weight LLMs, embedding models and translation models can be self-hosted when data residency or privacy rules require it, with some trade-offs in operational effort.
Tell us which languages, channels and documents your chatbot needs to cover, and we will propose an architecture and a phased plan. Reach us through the contact form or write to info@ijurugsoft.com.