02. Agentic AI Architecture

Browser RAG is not a multi-agent tool-using system in production. It is a RAG orchestration loop with optional LLM query rewriting, hybrid retrieval, and streamed local generation. Tool-calling plumbing exists but is disabled on the product path.

Orchestration model

graph TD
  classDef default fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc
  classDef highlight fill:#065f46,stroke:#34d399,stroke-width:2px,color:#f0fdf4
  classDef warning fill:#78350f,stroke:#fbbf24,stroke-width:2px,color:#fffbeb

  Q["User query + history"]:::highlight
  RW["rewriteQueryForRetrieval<br/>maxTokens 128"]
  RQ["Standalone retrieval query"]
  HY["retrieveChunks<br/>embed + vector + keyword + RRF"]
  CTX["Build Source N context"]
  GEN["streamLLMWithToolLoop<br/>thinking on, tools off"]:::highlight
  OUT["text / thinking / citations / debug"]
  EMPTY["No relevant information..."]:::warning

  Q --> RW --> RQ --> HY
  HY -->|hits| CTX --> GEN --> OUT
  HY -->|empty| EMPTY

  linkStyle default stroke:#64748b,stroke-width:2px

Entry point: generateRAGAnswer in src/rag/orchestrator.ts.

Model provider boundaries

Layer	Provider	Evidence
Embeddings	Transformers.js in embedding.worker	src/rag/embedding-runtime.ts, src/workers/embedding.worker.ts
LLM — Transformers.js	@browser-ai/transformers-js / HF ONNX	use-qwen35, catalog transformers-js
LLM — WebLLM	@mlc-ai/web-llm / @browser-ai/web-llm	use-webllm
LLM — Gemma kernel	Bundled gemma-4-e2b.js	use-gemma4
LLM — LFM kernel	Bundled lfm2_5.js	use-lfm2

Unified API: loadLLMVariant / streamLLMWithToolLoop / LLMEngineAdapter in src/llm/llm-runtime.ts.

Defaults (src/llm/llm-models.ts): iOS → WebLLM qwen-0.5b; else Transformers.js qwen35-0.8b.

Prompts

Stage	Prompt role
Rewrite	System: produce a single standalone search query; user: history + latest message
Answer	System: answer only from context; cite [1]/[2]; include Context Excerpts
Tools	Prompt augmenters exist but return empty strings when tools disabled

History for rewrite/answer is capped at MAX_HISTORY_TURNS = 8.

Tools and tool execution

Confirmed: RAG calls streamLLMWithToolLoop with toolsEnabled: false.

Confirmed stubs in llm-runtime.ts:

• MAX_TOOL_ROUNDS = 0, MAX_TOOL_CALLS_PER_ROUND = 0
• executeToolCalls → []
• buildToolPromptPrefix / buildToolPromptSection → ''

Parsers and engine-features.ts still describe calculator/time-style capabilities — Inferred leftover from a broader Browser AI toolkit, not active in this app’s ask path.

Representative request sequence

%%{init: {
  "theme": "base",
  "themeVariables": {
    "primaryColor": "#1e293b",
    "primaryTextColor": "#f8fafc",
    "primaryBorderColor": "#38bdf8",
    "lineColor": "#64748b",
    "actorBackground": "#1e293b",
    "actorBorder": "#38bdf8",
    "actorTextColor": "#f8fafc",
    "noteBorderColor": "#fbbf24",
    "noteBkgColor": "#78350f",
    "noteTextColor": "#fffbeb"
  }
}}%%
sequenceDiagram
  participant Orch as Orchestrator
  participant LLM as Engine Adapter
  participant Ret as Retrieval
  participant UI as Chat UI

  Orch->>LLM: Rewrite stream (thinking/tools off)
  LLM-->>Orch: rewritten query text
  Orch-->>UI: retrieval_query
  Orch->>Ret: retrieveChunks
  Ret-->>Orch: results + RetrievalDebugInfo
  Orch-->>UI: debug, citations
  Orch->>LLM: Answer stream (thinking on, tools off)
  loop tokens
    LLM-->>Orch: text_delta / thinking_delta
    Orch-->>UI: forward events
  end
  Orch-->>UI: done

Memory, state, resumability

Concern	Behavior
In-chat memory	React state for messages; last 8 turns passed into orchestrator
Persistence	Completed Q/A written to query_history (chat route)
Checkpoints	None for mid-stream generation
Abort	abortSignal checked during rewrite/stream
Embedding lock	Project stores embedding_model_id; retrieval filters by it

Run lifecycle (document indexing)

stateDiagram-v2
  [*] --> pending: insert document
  pending --> processing: indexDocument start
  processing --> completed: chunks + embeddings committed
  processing --> failed: extract/embed/DB error
  failed --> processing: retry from stored file
  completed --> [*]

Statuses live on documents.status (pending|processing|completed|failed). Schema also defines index_jobs, but no application writes were found.

Streaming and debug events

RAGAnswerChunk types: text_delta, thinking_delta, citations, retrieval_query, debug, done, error.

RagDebugInfo includes user vs rewritten query, history turn count, and full RetrievalDebugInfo (semantic/keyword/fused hits + stage timings). UI: src/components/chat/retrieval-debug-panel.tsx.

Human-in-the-loop

No approval gates for tool use (tools off). User controls: model selection, project settings, abort, document filter, retry failed indexes, backup/restore.

Failure handling

Failure	Handling
Rewrite fails	Fall back to original query
No citations	Fixed “No relevant information…” text; no LLM call
Generation error	Yield { type: 'error', error }
Index failure	documents.status = failed + error_message; retry from IDB file bytes

Interview Q&A

Q: Is this an agent?
A: Product path is a deterministic RAG pipeline with one optional LLM rewrite step, not a tool-calling agent loop. Tool infrastructure is stubbed.

Q: How do you keep multi-turn retrieval accurate?
A: rewriteQueryForRetrieval resolves pronouns into a standalone search query before hybrid search, while the answer turn still sees full recent history.

Q: Why hybrid + RRF?
A: Vector search misses exact tokens; keyword OR + ts_rank recovers lexical hits; RRF merges ranked lists without score calibration (k=60).