03. Backend Deep Dive

No server backend. Inference “backend” is translator.js.

Responsibilities

Function	Role
MODEL_CONFIGS	Maps UI keys → HF repo ids / directions
fetchWithCache / fetchWithProgress	Cache Storage + byte progress
loadModel	Meta, tokenizers, sidecars, 3 ORT sessions
unloadModel	Drop in-memory sessions (cache kept)
translate	Preprocess → encode → greedy decode → postprocess

Session layout

1. encoder_model.onnx
2. decoder_model.onnx (first step)
3. decoder_with_past_model.onnx (KV-cache steps)

externalData attaches .onnx.data weight sidecars (critical for WASM MountedFiles). Special case: en-indic-200m + q4f16 uses decoder_shared.onnx.data only.

Decode loop

• FLORES language tags + separate tokenization workaround for BPE leading space
• Encoder once → decoder → loop decoder_with_past with past_key_values
• Greedy argmax; max 128 new tokens; EOS stop
• Vocab clamps via tokenizer_meta sizes / unk_id
• Optional transliteration back to target script

Providers

executionProviders: [selected, 'wasm'] fallback. WASM paths pointed at jsDelivr ORT 1.21.0. numThreads temporarily set to 1 while fetching sidecars.

Interview Q&A

Q: Why not Transformers.js generate()?
A: Need explicit control over dual tokenizers, sidecars, and IndicTrans2 FLORES tagging with exported graph names.