11. Open Questions & Risks

Open questions

1. Is config.yaml 7B the next committed experiment or an incomplete switch?
2. Which compressor model produced the published traces?
3. Will ARC-Challenge eval land in eval.py?
4. Should transformers be a direct dependency for grug_score.py?
5. Are it-1 narrative details about GSM8K train data fully reconciled with later docs?

Risks

Risk	Notes
Reproducibility	Needs HF download + Mac + API credits
Config/docs drift	7B vs 1.5B confusion for newcomers
Accuracy tax	Limits “free lunch” claims
No Python CI	Regressions in validators possible
API cost	Full recompress of thousands of traces

Practical next steps

1. Align README/config.yaml model size or document dual targets clearly
2. Add pytest for grug_score and validators
3. Pin compressor model id in report metadata
4. Complete 7B run and publish report/it-3/
5. Keep visualizer deploy green on Pages

Hub publishing note

This dossier is public-source and listed in projects/publish.txt for the code-dossiers hub.