Skip to content

Grug Reasoning Fine-Tune — Codebase Interview Dossier

Repository: Hari31416/qwen-grug-finetune (public)
Inspected checkout: repos/qwen-grug-finetune (cloned 2026-07-09)
HEAD: 0f12bc6 (2026-07-07)
Artifacts: hari31416/qwen-grug-finetune on Hugging Face
Stack: Python 3.11+ · MLX / mlx-lm · LoRA SFT · OpenAI-compatible compression API · React 19 visualizer · Vite · pnpm

Reading order

  1. Evidence Map
  2. Product & System Overview
  3. Backend Deep Dive — pipeline scripts
  4. Frontend Deep Dive — experiment visualizer
  5. Data Model & Storage
  6. Infra & Local Ops
  7. Security & Safety
  8. Testing & Quality
  9. Unique Engineering Highlights
  10. Interview Prep
  11. Risks & Next Steps

Note: Chapter 02-agentic-ai-architecture is omitted. This is a supervised LoRA style-transfer experiment with a one-shot LLM compression step in data prep — not an agent loop.

Preview and export

make -C projects/qwen-grug-finetune serve
make -C projects/qwen-grug-finetune consolidate

Executive summary

Teach a small DeepSeek-R1 distill model to write telegraphic “Grug” chain-of-thought inside <think> blocks: drop filler, keep logic, cut tokens and latency. The pipeline samples general reasoning prompts, generates raw CoT on MLX, compresses traces with an external LLM guided by style_guide.md, validates with a Grug score, formats regularized SFT JSONL, trains LoRA via mlx_lm.lora, and evaluates on GSM8K.

Published Iteration 2 (regularized, 1.5B 4-bit): large thinking-token and latency wins with an accuracy “alignment tax” on GSM8K. A React visualizer explores traces and metrics locally or from HF. Current config.yaml points at 7B while committed reports and README still describe 1.5B runs.

System map

graph TD
  classDef default fill:#1e293b,stroke:#38bdf8,stroke-width:2px,color:#f8fafc
  classDef highlight fill:#065f46,stroke:#34d399,stroke-width:2px,color:#f0fdf4
  classDef warning fill:#78350f,stroke:#fbbf24,stroke-width:2px,color:#fffbeb

  S["sample_sft_prompts"] --> G["generate_traces MLX"]
  G --> C["compress_traces API"]:::highlight
  C --> V["validate + grug_score"]
  V --> F["format_data regularization"]
  F --> T["train.py mlx_lm.lora"]:::highlight
  T --> E["eval.py GSM8K"]
  E --> P["plot + report/"]
  P --> HF["sync_hf Hub"]
  P --> UI["visualize/ SPA"]

  linkStyle default stroke:#64748b,stroke-width:2px

Top things to know

  1. Goal is token-efficient CoT style, not a new reasoning algorithm.
  2. Apple Silicon + MLX is a hard requirement for train/generate/eval.
  3. Compression uses style_guide.md as the system prompt document.
  4. SFT regularization (negatives + prompt dropout) fixed prompt regurgitation.
  5. Eval is GSM8K-focused in code; ARC is planned in docs.
  6. data/ and adapters/ are gitignored — pull from HF to reproduce.
  7. Visualizer is client-only; GitHub Pages deploy workflow exists.
  8. Watch the 1.5B docs vs 7B config.yaml drift.

Most impressive engineering

Grug score + logic validators; SFT regularization against leakage; end-to-end MLX LoRA loop with best-val checkpoint promotion; experiment visualizer.

Report coverage

File Role
00 Evidence
01 Overview
03 Scripts
04 Visualizer
05 Artifacts
06 Ops
07 Safety
08 Quality
09 Highlights
10 Interview
11 Risks