Skip to content

03. Backend Deep Dive

Layout and entrypoints

Piece Path Role
CLI / script entry main.py Thin wrapper around uvicorn
App factory app/main.py create_app(), lifespan, CORS, /health
Settings app/config.py pydantic-settings from .env
OCR router app/api/ocr.py All /ocr/* endpoints
Schemas app/schemas/ocr.py Job + metadata models
Pipeline service app/services/pipeline.py Model cache, run_image / run_pdf
Core OCR paddleocr_pipeline.py Layout, parallel predict, MD/HTML render
VLM clients app/services/vlm_client.py PaddleVLMClient, LiteLLMVLMClient
Celery app/tasks/celery_app.py, ocr_task.py Worker + run_ocr_job

Python requirement: >=3.13 (pyproject.toml).

Application lifecycle

On startup (app/main.py lifespan):

  1. Ping Redis (settings.redis_url); warn if unreachable
  2. Optionally warmup_models(...) when prewarm_models is true
  3. Mount CORS from ALLOWED_HOSTS (* → allow all origins)
  4. Include OCR router; expose GET /health

API surface

Prefix /ocr (app/api/ocr.py):

Method Path Behavior
GET /jobs List job dirs ∩ Redis status
POST /jobs Upload + enqueue (202)
DELETE /jobs/{job_id} Delete disk + Redis
GET /jobs/{job_id}/status Poll progress
GET /jobs/{job_id}/download/{fmt} md | html | zip
GET /jobs/{job_id}/metadata All pages
GET /jobs/{job_id}/metadata/{page} One page (1-based)

Submit parameters

  • file (multipart) — required
  • vlm_mode — optional override of env VLM_MODE
  • dpi — 72–600, default 200
  • max_pages — optional
  • output_format — md | html | both

Upload allowlist: png, jpg, jpeg, tiff, tif, bmp, webp, pdf.

Celery configuration

broker/backend = REDIS_URL with logical DB rewritten to /1
queue          = ocr
prefetch       = worker_prefetch_multiplier=4  # comment says "one task"

Worker Docker CMD uses --pool solo (Dockerfile.celery). Prewarm hooks on worker_process_init (celery_app.py).

Job task branching

graph TD
  classDef default fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc
  classDef highlight fill:#065f46,stroke:#34d399,stroke-width:2px,color:#f0fdf4
  classDef warning fill:#78350f,stroke:#fbbf24,stroke-width:2px,color:#fffbeb

  Start["run_ocr_job"]:::highlight
  Mode{"vlm_mode"}
  N["NvidiaOCRExtractor"]
  S["SarvamExtractor"]
  V["VLMExtractor<br/>whole page"]
  L["get_models → layout + VLM"]
  PDF["run_pdf / run_image"]
  Out["Write MD/HTML + metadata"]
  Done["Redis status=done"]:::highlight
  Fail["Redis status=failed"]:::warning

  Start --> Mode
  Mode -->|"nvidia + key"| N
  Mode -->|"sarvam + key"| S
  Mode -->|vlm| V
  Mode -->|layout default| L
  L --> PDF
  N --> Out
  S --> Out
  V --> Out
  PDF --> Out
  Out --> Done
  Start -.->|exception| Fail

  linkStyle default stroke:#64748b,stroke-width:2px

Evidence: app/tasks/ocr_task.py early returns for nvidia/sarvam/vlm before layout path.

Layout + VLM pipeline (default)

  1. get_models (pipeline.py):
  2. Paddle model name (blank / paddleocr-vl* / pp-vl* / pp-docvlm*) → build_pipeline + PaddleVLMClient
  3. Else → PP-DocLayoutV3 + LiteLLMVLMClient (requires non-empty VLM_MODEL)
  4. PDF pages rasterized with PyMuPDF at configured DPI
  5. Layout predict yields labeled boxes
  6. Labels in IMAGE_LABELS (image, figure, seal) → embed base64 crop, skip VLM
  7. Text blocks transcribed in parallel:
  8. LiteLLM: asyncio + semaphore (vlm_async_concurrency, default 8)
  9. Paddle: ThreadPoolExecutor (vlm_max_workers)
  10. Prompts from LABEL_TO_PROMPT (table/chart/formula) or "OCR:"; optional custom file for LiteLLM only
  11. results_to_markdown / results_to_html; PDF stitch with page markers

Core helpers live in paddleocr_pipeline.py; service layer adds timing and Redis progress callbacks.

Provider extractors

Raw VLM (app/services/vlm/extractor.py)

Whole-page PNGs at 150 DPI, Jinja prompt app/prompts/ocr_vlm/vlm_ocr_prompt.jinja2, LiteLLM via app/core/llm/client.py.

NVIDIA (app/services/nvidia/extractor.py)

  • Model: nvidia/nemoretriever-parse
  • 300 DPI JPEG with size backoff under ~3.5 MB base64
  • Max 2 parallel pages
  • LaTeX tabular → Markdown helper
  • Gemma fallback on truncation/context errors (Confirmed in module docstring and helpers)

Sarvam (app/services/sarvam/extractor.py)

Async Sarvam job lifecycle; downloads md/html/json/zip; JSON normalized to PageMetadata in ocr_task.py.

Configuration highlights

From app/config.py / .env.example:

Setting Purpose
REDIS_URL Job state (DB 0) and base for Celery URL rewrite
VLM_MODE Default provider mode
VLM_MODEL LiteLLM model or Paddle pattern
VLM_PADDLE_SERVER_URL Local Paddle VL OpenAI-compatible server
NVIDIA_OCR_* / SARVAM_* Third-party credentials and timeouts
VLM_ASYNC_CONCURRENCY / VLM_MAX_WORKERS Parallelism knobs
PREWARM_MODELS Startup model load
JOBS_DIR Artifact root
ALLOWED_HOSTS CORS origins

Error handling and logging

  • Task exceptions → Redis status=failed + error string, then re-raise for Celery FAILURE
  • API uses HTTPException for 404/409/422
  • Logging via standard logging in tasks/pipeline; startup still uses print in app/main.py

Backend interview Q&A

Q: Why Celery instead of BackgroundTasks?
A: OCR is multi-minute GPU/CPU work; Celery isolates workers, enables independent scale, and survives API restarts. Documented in ADR 1.

Q: How do you avoid cold-start latency?
A: Process-level model singletons in pipeline.get_models plus optional prewarm on API lifespan and Celery worker_process_init.

Q: How does LiteLLM fit?
A: Non-Paddle VLM_MODEL values go through LiteLLMVLMClient, so OpenAI/NIM/Gemini/local vLLM share one transcription interface without changing layout logic.