---
language: [en, zh]
license: apache-2.0
base_model: mlx-community/Qwen3-4B-Instruct-2507-4bit
tags:
  - qwen3
  - lora
  - mlx
  - latte-agent
  - personal-voice
  - distillation
  - research-archive
library_name: transformers
---

# Qwen3-4B Latte v6 — research archive (not shipped)

Voice-distillation LoRA fine-tune of `Qwen3-4B-Instruct-2507`. This is **v6**,
the final iteration of the Latte distillation program. The program is now
closed; see "Decision" below.

## What's inside

| File | Size | Format | Use |
|---|---|---|---|
| `adapter_model.safetensors` | 14 MB | mlx LoRA (rank 8, scale 20, iter 400 best-val) | Apply on top of base with `mlx_lm.fuse` |
| `adapter_config.json` | <1 KB | mlx config | LoRA hyperparameters |
| `model-0000{1,2}-of-00002.safetensors` | 7.7 GB | HF / bfloat16 fused | Direct transformers / vLLM use |
| `qwen3-4b-latte-v6-f16.gguf` | 7.5 GB | GGUF F16 | llama.cpp / Ollama (high quality) |
| `qwen3-4b-latte-v6-Q4_K_M.gguf` | 2.3 GB | GGUF Q4_K_M | llama.cpp / Ollama (balanced) |

## What v6 tried

The hypothesis going in: **synthesize new skills observed since v5, add OOD
coverage, fix v5's known failure modes** (stage-direction leakage, coffee-name
persona collapse).

Dataset = 567 pairs:
- 63 **real Latte voice anchors** (Telegram + Moltbook activity 5/15–5/20)
- 14 **skill-anchored Q&A pairs** (from daily_log learnings: permission scope,
  audit trail, incident response vs post-mortem, external validators, digital
  vs physical reasoning, Neovim 0.11 LSP, CARLA tick-rate masking, format
  supersedes origin, AI task cost measurement, verify-vs-inertia)
- 15 **corrective pairs**: coffee disambiguation (Latte the drink), clean
  self-reference, OOD technical breadth (TCP/UDP, git rebase, D-state, CAP,
  CDN), OOD general, factual, anti-stage-direction examples
- 475 v5 refined pairs (filtered for voice-leakage; 0 hit the filters)

Training: rank 8, scale 20, 8 layers, 800 iters, lr 1e-4. **Best val loss
2.468 at iter 400** — the lowest of any version (v5 2.732, v4 2.785).

## Evaluation

Same 30 held-out prompts as v4/v5 eval (Moltbook/HF style), blind Claude judge:

| Comparison | v6 wins | other wins | ties |
|---|---|---|---|
| v6 vs base | 13 (43.3%) | 15 (50.0%) | 2 |
| v6 vs v5  | 15 (50.0%) | 14 (46.7%) | 1 |

**Headline:** v6 **does not** clear the 55% ship threshold against base, and
is essentially tied with v5 head-to-head. Despite the lowest val loss in the
program, blind voice-fit did not improve.

## Why v6 didn't beat base (despite lowest val loss)

Three patterns hold across v4 → v5 → v6:
1. **Val loss and blind-eval-quality are decoupled.** v6 has the best val loss
   and the worst blind win-rate. Next-token prediction on training-distribution
   text is not measuring what we want.
2. **Each new version ties the previous one** (~50:50 against the prior). More
   data swaps one equivalent voice profile for another rather than improving.
3. **Eval methodology has ~±15% judge-variance.** v5 era used Claude subagents,
   v6 used direct Claude judging. The "v5 won 67%, v6 won 43%" gap is partially
   real, partially the judge swap.

The Latte voice as captured here is heavily tied to confident-stat hallucination.
Distillation amplifies the stylistic signature but does not improve underlying
factuality.

## Decision: distillation program closed

After 4 versions (v3 unshipped, v4/v5/v6 archived, none in production), the
program is **not advancing user-facing quality**. The production Latte agent
will continue running base `qwen3:4b-instruct-2507-q4_K_M` indefinitely.

Future Latte improvement effort goes into **worker capabilities** instead
(multimodal handling, file extraction, tool routing, memory systems) — which
have shown immediate user-facing value where brain-distillation has not.

Anyone who wants to revive this line should first fix the eval methodology
(consistent judge, larger n, OOD-primary metric) and probably switch
paradigm (DPO over SFT, larger LoRA rank, knowledge distillation from 14B
teacher rather than Claude-refined SFT).

## Usage (if you really want)

### MLX (Apple Silicon, recommended for inference)
```python
from mlx_lm import load, generate
model, tokenizer = load(
    "mlx-community/Qwen3-4B-Instruct-2507-4bit",
    adapter_path="./",
)
print(generate(model, tokenizer, "Your prompt", max_tokens=200))
```

### llama.cpp / Ollama
```
FROM qwen3-4b-latte-v6-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_k 20
PARAMETER top_p 0.8
```

## License

Apache 2.0 (inherits from Qwen3-4B-Instruct-2507, © Alibaba Cloud).