Skip to content

09. Unique Engineering Highlights

Focus on distinctive work, not ordinary FastAPI/Celery plumbing.

1. Layout-aware multimodal OCR

Problem: Classic OCR loses structure on tables, formulas, and figures.
How: PP-DocLayoutV3 detects labeled regions; text blocks go to a VLM with label-specific prompts; figures/seals embed as base64 in HTML/MD.
Why interesting: Combines detection + generative transcription instead of a single end-to-end black box.
Tradeoff: Two-model pipeline is heavier than page-level VLM-only.
Evidence: paddleocr_pipeline.py (LABEL_TO_PROMPT, IMAGE_LABELS), app/services/pipeline.py.
Interview line: "We OCR blocks, not just pages, and we treat figures as first-class embeds."

2. One job API, four provider backends

Problem: Teams want to A/B local Paddle, cloud VLMs, NVIDIA Parse, and Sarvam without rewriting clients.
How: Shared submit/status/download surface; Celery branches on vlm_mode.
Why interesting: Product flexibility without fracturing the REST contract.
Tradeoff: Metadata richness differs by provider (layout path is richest).
Evidence: app/tasks/ocr_task.py, frontend VLM_MODES.
Interview line: "Clients speak jobs; workers speak providers."

3. Pluggable VLMClient + LiteLLM escape hatch

Problem: Vision model landscape changes monthly.
How: Abstract VLMClient; is_paddle_model routes blank/Paddle names locally; everything else goes through LiteLLM.
Why interesting: Swap OpenAI/NIM/Gemini/vLLM by env var.
Tradeoff: Debugging spans LiteLLM provider quirks.
Evidence: app/services/vlm_client.py, ADR 5.
Interview line: "The pipeline depends on an interface, not a vendor."

4. Process-level model cache and prewarm

Problem: Loading PP-DocLayoutV3 / VL models per job is multi-second cold start.
How: Module singletons in get_models; optional warmup on API lifespan and Celery worker_process_init.
Why interesting: Makes async workers practical for interactive dashboards.
Tradeoff: Memory held per worker; model changes need process recycle.
Evidence: pipeline.py, celery_app.py, app/main.py, ADR 2.

5. Parallel per-block transcription with live progress

Problem: Dozens of blocks per page make sequential VLM calls too slow.
How: Thread pool (Paddle) or asyncio semaphore (LiteLLM); callbacks increment Redis progress.
Why interesting: Latency optimization tied directly to UX progress bars.
Tradeoff: Provider rate limits; concurrency knobs must be tuned.
Evidence: pipeline.py, ADR 3, frontend progress calculation.

6. NVIDIA extractor resilience

Problem: High-DPI page images blow payload limits; APIs truncate or error.
How: JPEG quality backoff under size cap, max 2 parallel pages, LaTeX→MD tables, Gemma fallback on truncation/context errors.
Why interesting: Production-minded failure handling around a third-party OCR NIM.
Tradeoff: Complexity localized in one large module (~500 LOC).
Evidence: app/services/nvidia/extractor.py.

7. Sarvam artifact normalization

Problem: External OCR returns proprietary JSON/zip layouts.
How: Download artifacts; map Sarvam blocks/coordinates into shared PageMetadata.
Why interesting: Keeps the metadata API stable across providers.
Tradeoff: Some fields (timings, image blocks) are filled with defaults.
Evidence: _normalize_sarvam_metadata in ocr_task.py.

8. Dual-use core pipeline module

Problem: Want CLI experimentation without the full service.
How: paddleocr_pipeline.py is importable and has a main() CLI path; service wraps it.
Why interesting: Separates algorithm from orchestration.
Tradeoff: sys.path insert in pipeline.py to import repo-root module.
Evidence: paddleocr_pipeline.py, app/services/pipeline.py.

9. First-class project documentation

Problem: OCR systems are hard to onboard.
How: In-repo MkDocs covering architecture, providers, ADRs, API reference.
Why interesting: Unusual completeness for a 0.1.0 service.
Tradeoff: Docs can drift (e.g. stale AGENTS.md infra mentions).
Evidence: docs/, mkdocs.yml, make docs-serve.

What is ordinary (do not oversell)

  • FastAPI routers and Pydantic models
  • Celery + Redis as a generic async pattern
  • shadcn/Tailwind dashboard chrome
  • Extension allowlists on uploads