Backend Deep Dive

FastAPI lifecycle

app.main initializes Qdrant collections, optional summary collections, PostgreSQL, seeded admin/core projects, optional startup document sync, Redis memory, and reranking. It closes memory and database resources on shutdown. Routers cover auth, chatbot, ingestion, retrieval, query, download, document, project, bulk import, tasks, history, and concordance.

Async/background boundaries

Celery workers handle ingestion tasks while the API serves streaming and query endpoints. The backend combines async database sessions, object storage, vector search, keyword search, and LLM calls. This makes startup configuration and dependency health important operational concerns.

Authentication

The database is the account source of truth. Login returns a signed JWT stored by the shared frontend under vanijya_access_token; both Analyst and RAG APIs verify it with the shared secret.

Notable modules

Document services include Docling, PyMuPDF, pdfplumber, OCR, hierarchy extraction, specialized tabular chunking, summaries, clustering, and filesystem sync. Retrieval services cover Qdrant, Meilisearch, reranking, query optimization, and multi-hop.