2.9 KiB
Quintus Documentation
Quintus-1.7B is a compact assistant built from the Qwen3-1.7B-Base architecture. The project uses online full-vocabulary knowledge distillation from a Qwen3-8B teacher, followed by targeted SFT for instruction style, identity grounding, and generation stability.
This documentation summarizes the public architecture, training decisions, evaluation controls, and release artifacts for the showcase branch.
Reading Order
- Architecture: End-to-end pipeline, modules, data flow, and training phases.
- Experiment Timeline: How the project moved from offline top-k KD to final online full-vocabulary KD.
- Training Playbook: Practical training choices, memory rules, packing, kernels, and checkpointing.
- Pipeline Hardening: Silent-failure classes and the safeguards added around artifacts, provenance, and runtime.
- Evaluation Methodology: Benchmark controls, parser traps, raw/chat comparisons, and qualitative evaluation rules.
- Engineering Insights: Condensed technical lessons and design decisions.
- Benchmarks: Verified evaluation results and interpretation.
- Weight Audit: Structural checkpoint verification and what the audit means.
- Hugging Face Model Card: Release-page text for the public model card.
Project Summary
The core thesis is simple: a small base model can absorb useful reasoning behavior from a larger instruction model if the distillation signal is dense enough and the evaluation controls are fair.
The project initially explored sparse offline top-k distillation, but that approach hit a ceiling because the student only saw a tiny fraction of the teacher vocabulary distribution. The final pipeline pivots to online KD, where teacher and student are run together and the student receives the teacher's full-vocabulary probability distribution during training.
After KD, a small SFT stage teaches the model how to expose that knowledge in a conversational interface. This separation matters: KD transfers capability; SFT and later preference training improve behavior, style, and confidence calibration.
Repository Map
configs/ Training configuration and DeepSpeed template.
src/ Online KD, data loading, losses, checkpointing, and packing.
sft/ Post-KD supervised fine-tuning, chat, and consolidated evaluation runner.
weight_audit/ Checkpoint structure and weight-divergence audit.
docs/ Public architecture, training, evaluation, and release notes.
Main Public Artifact
The final model weights are available at: Quintus
The Colab quickstart is available at: Colab Quick Chat