01. Product & System Overview

What it is

A research/engineering experiment: fine-tune DeepSeek-R1-Distill-Qwen (historically 1.5B 4-bit, config now 7B) on Apple Silicon so its reasoning traces become Grug-style — telegraphic, low-filler CoT that still solves the task.

Not a product SaaS. Users are the experimenter and anyone reproducing from GitHub + Hugging Face.

Primary workflow

%%{init: {
  "theme": "base",
  "themeVariables": {
    "primaryColor": "#1e293b",
    "primaryTextColor": "#f8fafc",
    "primaryBorderColor": "#38bdf8",
    "lineColor": "#64748b",
    "actorBackground": "#1e293b",
    "actorBorder": "#38bdf8",
    "actorTextColor": "#f8fafc",
    "noteBorderColor": "#fbbf24",
    "noteBkgColor": "#78350f",
    "noteTextColor": "#fffbeb"
  }
}}%%
sequenceDiagram
  participant Exp as Experimenter
  participant MLX as MLX model
  participant API as Compression LLM
  participant LoRA as mlx_lm.lora
  participant GSM as GSM8K eval

  Exp->>MLX: Generate raw CoT on sampled prompts
  MLX-->>Exp: Keep correct traces only
  Exp->>API: Compress thinking via style_guide
  API-->>Exp: Grug traces
  Exp->>Exp: Validate + format SFT JSONL
  Exp->>LoRA: Train adapters
  Exp->>GSM: Baseline vs fine-tuned

Headline results (Iteration 2 regularized)

From committed report/REPORT.md (1.5B base):

Theme	Outcome
Efficiency	Large cuts in thinking tokens and latency
Format	High compliance with answer-after-</think>
Accuracy	GSM8K drop vs base (“alignment tax”)
Leakage	Regularization reduced system-prompt regurgitation vs earlier runs

Exact percentages are in the report tables; cite report/REPORT.md in interviews rather than memorizing every delta.

Subsystems

Piece	Role
scripts/	Full data + train + eval pipeline
config.yaml / lora_config.yaml	Model paths and LoRA HPs
style_guide.md	Compression style contract
report/	Committed iteration write-ups
visualize/	Static explorer for traces/metrics
HF Hub	Distribution of gitignored heavy artifacts

Technical bets

Bet	Tradeoff
Style-transfer SFT instead of RL	Simpler; risk of accuracy tax and leakage
External LLM for compression	Quality depends on API model/cost
MLX on Mac	Fast local loop; not CUDA-portable
General reasoning mix → GSM8K eval	Tests transfer; may underfit math

Interview one-liner

"I built an MLX LoRA pipeline that teaches a small R1 distill model to think in compressed Grug CoT, with validators and SFT regularization so it stays terse without regurgitating the style prompt — measured on GSM8K token, latency, and accuracy tradeoffs."