08. Testing & Quality¶
Layout and commands¶
| Item | Detail |
|---|---|
| Runner | pytest (pyproject.toml [tool.pytest.ini_options]) |
| Async | asyncio_mode = auto |
| Dev deps | pytest, pytest-asyncio, httpx, pytest-mock |
| Command | make test (project Makefile) |
Test files¶
| File | Focus |
|---|---|
tests/conftest.py |
Session mocks for Redis/Celery; temp JOBS_DIR; FastAPI TestClient |
tests/test_api.py |
Health, submit, reject bad type, status, list, delete, download 409/200/zip, metadata |
tests/test_tasks.py |
Image job writes md/html/meta; failure → failed; output_format=md only |
tests/test_pipeline_prompt.py |
Custom prompt for LiteLLM vs label prompts for non-LiteLLM |
What is well tested¶
- HTTP contract for job lifecycle with fakes
- Storage + Redis hash interactions in API tests
- Task success/failure status transitions with mocked pipeline
- Prompt selection policy for custom OCR prompts
What remains risky / untested¶
| Gap | Why it matters |
|---|---|
| Live Paddle layout / VL | Core accuracy path |
| NVIDIA / Sarvam / raw VLM extractors | Provider-specific failure modes |
| Multi-page PDF integration | Stitching and progress semantics |
| Frontend | No Vitest/Playwright suite found |
| Docker compose smoke | Not covered by pytest |
| Upload size limits | None to test |
Fixtures and mocks¶
conftest.py patches Celery app construction and Redis so API tests do not need a live broker. Tasks tests call run_ocr_job with patched svc.run_image / settings.
CI / quality gates¶
Unknown in this inspection: no .github/workflows reviewed in the file inventory (none listed by rg --files at clone). Local gates: make test, make fmt (black + isort).
Stale test note¶
test_submit_job_config_is_env_only name suggests env-only config, but the live API accepts vlm_mode query override (Confirmed in app/api/ocr.py). Treat the test name/comment as potentially outdated relative to product behavior.
Quality interview Q&A¶
Q: How do you test without GPUs?
A: Mock pipeline / VLM clients; assert Redis fields and files on a temp JOBS_DIR.
Q: What would you add first?
A: Contract tests for each vlm_mode branch with recorded fixtures, plus a Playwright smoke for upload→done, and server-side upload limit tests once limits exist.