Data Model and Storage

Core entities

The README identifies User, Officer, Grievance, GrievanceCategory, chat sessions, and RAG chunks as core records. Grievances carry category, mandatory fields, priority, SLA/TAT, status, assignment, and timestamps. RAG chunks carry source document, role, text, and embedding.

erDiagram
  %%{init: {"theme":"base","themeVariables":{"primaryColor":"#1e293b","primaryTextColor":"#f8fafc","primaryBorderColor":"#38bdf8","lineColor":"#64748b","attributeBackgroundColor":"#1e293b","attributeTextColor":"#f8fafc"}}}%%
  USER ||--o{ GRIEVANCE : submits
  OFFICER ||--o{ GRIEVANCE : handles
  GRIEVANCE_CATEGORY ||--o{ GRIEVANCE : classifies
  DOCUMENT ||--o{ RAG_CHUNK : contains
  USER {
    uuid id
    string role
  }
  GRIEVANCE {
    string grievance_id
    string status
    string priority
    date sla_deadline
  }
  RAG_CHUNK {
    string access_role
    vector embedding
  }

Lifecycle

Plain documents are chunked, embedded, and upserted into pgvector with access roles. Querying first determines allowed roles, then performs similarity search, builds grounded context, calls the LLM, and validates returned sources. Attachments and ATR PDFs live in GCS; metadata remains in application records.

Redis is intentionally ephemeral session storage; PostgreSQL is the consistency boundary for cases and vector metadata.

Artifact lifecycle

flowchart TD
  classDef default fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc
  classDef highlight fill:#065f46,stroke:#34d399,stroke-width:2px,color:#f0fdf4
  Upload["Citizen or officer upload"]:::highlight --> Parse["Parse text/PDF/image"]
  Parse --> Session["Chat attachment metadata"]
  Upload --> GCS["GCS object key"]
  Officer["Officer resolves case"] --> PDF["ReportLab ATR PDF"]
  PDF --> GCS
  Doc["Knowledge-base document"] --> Chunk["Normalize and chunk"]
  Chunk --> Embed["Gemini embedding"]
  Embed --> PG["rag_chunks + pgvector"]
  linkStyle default stroke:#64748b,stroke-width:2px

Chat uploads use an object key under attachments/{session_id}/... and store the key plus filename in session attachments. The raw object and extracted text therefore have different ownership and retention considerations. ATR generation creates a branded PDF and stores it separately from the grievance fields.

Consistency and idempotency

RAG ingestion accepts an optional document ID and caps chunks at 200. Re-ingestion behavior is mediated by the vector store and should be verified for duplicate prevention. Grievance finalization should be idempotent on session ID because a client retry after an SSE disconnect could otherwise create duplicate cases.

Retention questions

The repository does not establish retention periods for raw citizen documents, chat messages, generated PDFs, or embeddings. A production policy should define deletion, legal hold, backup, and access-log requirements, especially because embeddings can preserve sensitive information even after a source file is removed.