language, license, base_model, tags, library_name

language	license	base_model	tags	library_name
en zh	apache-2.0	mlx-community/Qwen3-4B-Instruct-2507-4bit	qwen3 lora mlx latte-agent personal-voice distillation research-archive	transformers

Qwen3-4B Latte v6 — research archive (not shipped)

Voice-distillation LoRA fine-tune of Qwen3-4B-Instruct-2507. This is v6, the final iteration of the Latte distillation program. The program is now closed; see "Decision" below.

What's inside

File	Size	Format	Use
adapter_model.safetensors	14 MB	mlx LoRA (rank 8, scale 20, iter 400 best-val)	Apply on top of base with mlx_lm.fuse
adapter_config.json	<1 KB	mlx config	LoRA hyperparameters
model-0000{1,2}-of-00002.safetensors	7.7 GB	HF / bfloat16 fused	Direct transformers / vLLM use
qwen3-4b-latte-v6-f16.gguf	7.5 GB	GGUF F16	llama.cpp / Ollama (high quality)
qwen3-4b-latte-v6-Q4_K_M.gguf	2.3 GB	GGUF Q4_K_M	llama.cpp / Ollama (balanced)

What v6 tried

The hypothesis going in: synthesize new skills observed since v5, add OOD coverage, fix v5's known failure modes (stage-direction leakage, coffee-name persona collapse).

Dataset = 567 pairs:

• 63 real Latte voice anchors (Telegram + Moltbook activity 5/15–5/20)
• 14 skill-anchored Q&A pairs (from daily_log learnings: permission scope, audit trail, incident response vs post-mortem, external validators, digital vs physical reasoning, Neovim 0.11 LSP, CARLA tick-rate masking, format supersedes origin, AI task cost measurement, verify-vs-inertia)
• 15 corrective pairs: coffee disambiguation (Latte the drink), clean self-reference, OOD technical breadth (TCP/UDP, git rebase, D-state, CAP, CDN), OOD general, factual, anti-stage-direction examples
• 475 v5 refined pairs (filtered for voice-leakage; 0 hit the filters)

Training: rank 8, scale 20, 8 layers, 800 iters, lr 1e-4. Best val loss 2.468 at iter 400 — the lowest of any version (v5 2.732, v4 2.785).

Evaluation

Same 30 held-out prompts as v4/v5 eval (Moltbook/HF style), blind Claude judge:

Comparison	v6 wins	other wins	ties
v6 vs base	13 (43.3%)	15 (50.0%)	2
v6 vs v5	15 (50.0%)	14 (46.7%)	1

Headline: v6 does not clear the 55% ship threshold against base, and is essentially tied with v5 head-to-head. Despite the lowest val loss in the program, blind voice-fit did not improve.

Why v6 didn't beat base (despite lowest val loss)

Three patterns hold across v4 → v5 → v6:

1. Val loss and blind-eval-quality are decoupled. v6 has the best val loss and the worst blind win-rate. Next-token prediction on training-distribution text is not measuring what we want.
2. Each new version ties the previous one (~50:50 against the prior). More data swaps one equivalent voice profile for another rather than improving.
3. Eval methodology has ~±15% judge-variance. v5 era used Claude subagents, v6 used direct Claude judging. The "v5 won 67%, v6 won 43%" gap is partially real, partially the judge swap.

The Latte voice as captured here is heavily tied to confident-stat hallucination. Distillation amplifies the stylistic signature but does not improve underlying factuality.

Decision: distillation program closed

After 4 versions (v3 unshipped, v4/v5/v6 archived, none in production), the program is not advancing user-facing quality. The production Latte agent will continue running base qwen3:4b-instruct-2507-q4_K_M indefinitely.

Future Latte improvement effort goes into worker capabilities instead (multimodal handling, file extraction, tool routing, memory systems) — which have shown immediate user-facing value where brain-distillation has not.

Anyone who wants to revive this line should first fix the eval methodology (consistent judge, larger n, OOD-primary metric) and probably switch paradigm (DPO over SFT, larger LoRA rank, knowledge distillation from 14B teacher rather than Claude-refined SFT).

Usage (if you really want)

MLX (Apple Silicon, recommended for inference)

from mlx_lm import load, generate
model, tokenizer = load(
    "mlx-community/Qwen3-4B-Instruct-2507-4bit",
    adapter_path="./",
)
print(generate(model, tokenizer, "Your prompt", max_tokens=200))

llama.cpp / Ollama

FROM qwen3-4b-latte-v6-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_k 20
PARAMETER top_p 0.8

License

Apache 2.0 (inherits from Qwen3-4B-Instruct-2507, © Alibaba Cloud).