Skip to content

IndicTrans2 ONNX Export — Codebase Interview Dossier

Repository: Hari31416/indictrans2-onnx-export (public)
Inspected checkout: repos/indictrans2-onnx-export (cloned 2026-07-09)
HEAD: 23cf57d (2026-07-07)
Consumers: indictrans2-onnx-browser-demo, local-voice-chat
Stack: Python 3.11+ · PyTorch · ONNX Runtime · uv · Make · browser-lab · Vite dashboard

Reading order

  1. Evidence Map
  2. Product & System Overview
  3. Backend Deep Dive — export/quantize/upload scripts
  4. Frontend Deep Dive — browser-lab + dashboard
  5. Data Model & Storage
  6. Infra & Local Ops
  7. Security & Safety
  8. Testing & Quality — parity as the gate
  9. Unique Engineering Highlights
  10. Interview Prep
  11. Risks & Next Steps

Note: Chapter 02-agentic-ai-architecture is omitted — classical NMT export pipeline.

Preview

make -C projects/indictrans2-onnx-export serve
make -C projects/indictrans2-onnx-export consolidate

Executive summary

Python/Make pipeline that converts AI4Bharat IndicTrans2 PyTorch models into browser-ready ONNX (encoder + decoder + decoder_with_past), optimizes bundle size (tied-weight dedup, shared decoder sidecars), builds fast tokenizers, validates ≥99% token-exact parity on 3,300 golden fixtures, produces fp16/int8/q4f16 variants, and uploads to Hugging Face. Distilled 200M/320M and full 1B directions are supported. A browser-lab tests local scratch/ bundles; a React dashboard visualizes benchmarks.

System map

graph TD
  classDef default fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc
  classDef highlight fill:#065f46,stroke:#34d399,stroke-width:2px,color:#f0fdf4

  PT["AI4Bharat PyTorch"] --> EXP["Manual ONNX export"]:::highlight
  EXP --> OPT["Bundle optimize"]
  OPT --> TOK["Fast tokenizers"]
  TOK --> VAL["Parity ≥99%"]:::highlight
  VAL --> Q["fp16 / int8 / q4f16"]
  Q --> HF["HF Hub ONNX repos"]
  HF --> Demo["browser-demo / voice-chat"]
  OPT --> Lab["browser-lab scratch/"]

  linkStyle default stroke:#64748b,stroke-width:2px

Top things to know

  1. Optimum cannot export custom IndicTrans — manual torch.onnx.export + wrappers.
  2. Three graphs + shared decoder_shared.onnx.data after optimization.
  3. Parity on golden fixtures is the quality gate before publish.
  4. v2 scripts handle 1B config dims and 2GB protobuf limits.
  5. Quantization accuracy varies sharply by size (1B more robust than 200M Q4).
  6. Heavy artifacts live in gitignored scratch/ and on HF.
  7. Dashboard CI deploys docs/benchmarks only — no export CI.

Most impressive engineering

Mask-preserving ONNX wrappers; tied-weight / shared-sidecar size cuts; SPM→Fairseq dict remap; FP16 bias fix; Q4F16 EP-aware settings; 1B export path.