Data Model and Storage

Storage topology

flowchart LR
  classDef default fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc
  Postgres["PostgreSQL users/projects/history"] --> API["RAG API"]
  MinIO["MinIO originals/extracted files"] --> API
  Redis["Redis conversation window"] --> API
  Qdrant["Qdrant embeddings/chunks"] --> API
  Meili["Meilisearch BM25 index"] --> API
  API --> Answer["Grounded answer + sources"]
  linkStyle default stroke:#64748b,stroke-width:2px

PostgreSQL owns users, projects, document metadata, history, and suggestions. MinIO owns originals and extracted hierarchy artifacts. Qdrant owns dense chunks; Meilisearch owns sparse keyword indexes; Redis owns expiring conversational context.

Ingestion lifecycle

Upload → metadata row → object persistence → extraction → hierarchical/chunked text → embeddings and keyword index → optional clustered summaries → suggested questions. Re-ingestion and filesystem sync are important consistency paths.

Isolation

Projects scope documents and use cases. Private dossier, HS, and concordance project IDs create explicit boundaries. The codebase should ensure project ownership and index filters are applied consistently across metadata, Qdrant, Meilisearch, downloads, and prompts.