初始化项目，由ModelHub XC社区提供模型
Model: latte-agent/qwen3-4b-latte-v6 Source: Original Platform
This commit is contained in:
ModelHub XC
118
README.md
Normal file
118
README.md
Normal file
@@ -0,0 +1,118 @@
|
||||
---
|
||||
language: [en, zh]
|
||||
license: apache-2.0
|
||||
base_model: mlx-community/Qwen3-4B-Instruct-2507-4bit
|
||||
tags:
|
||||
- qwen3
|
||||
- lora
|
||||
- mlx
|
||||
- latte-agent
|
||||
- personal-voice
|
||||
- distillation
|
||||
- research-archive
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# Qwen3-4B Latte v6 — research archive (not shipped)
|
||||
|
||||
Voice-distillation LoRA fine-tune of `Qwen3-4B-Instruct-2507`. This is **v6**,
|
||||
the final iteration of the Latte distillation program. The program is now
|
||||
closed; see "Decision" below.
|
||||
|
||||
## What's inside
|
||||
|
||||
| File | Size | Format | Use |
|
||||
|---|---|---|---|
|
||||
| `adapter_model.safetensors` | 14 MB | mlx LoRA (rank 8, scale 20, iter 400 best-val) | Apply on top of base with `mlx_lm.fuse` |
|
||||
| `adapter_config.json` | <1 KB | mlx config | LoRA hyperparameters |
|
||||
| `model-0000{1,2}-of-00002.safetensors` | 7.7 GB | HF / bfloat16 fused | Direct transformers / vLLM use |
|
||||
| `qwen3-4b-latte-v6-f16.gguf` | 7.5 GB | GGUF F16 | llama.cpp / Ollama (high quality) |
|
||||
| `qwen3-4b-latte-v6-Q4_K_M.gguf` | 2.3 GB | GGUF Q4_K_M | llama.cpp / Ollama (balanced) |
|
||||
|
||||
## What v6 tried
|
||||
|
||||
The hypothesis going in: **synthesize new skills observed since v5, add OOD
|
||||
coverage, fix v5's known failure modes** (stage-direction leakage, coffee-name
|
||||
persona collapse).
|
||||
|
||||
Dataset = 567 pairs:
|
||||
- 63 **real Latte voice anchors** (Telegram + Moltbook activity 5/15–5/20)
|
||||
- 14 **skill-anchored Q&A pairs** (from daily_log learnings: permission scope,
|
||||
audit trail, incident response vs post-mortem, external validators, digital
|
||||
vs physical reasoning, Neovim 0.11 LSP, CARLA tick-rate masking, format
|
||||
supersedes origin, AI task cost measurement, verify-vs-inertia)
|
||||
- 15 **corrective pairs**: coffee disambiguation (Latte the drink), clean
|
||||
self-reference, OOD technical breadth (TCP/UDP, git rebase, D-state, CAP,
|
||||
CDN), OOD general, factual, anti-stage-direction examples
|
||||
- 475 v5 refined pairs (filtered for voice-leakage; 0 hit the filters)
|
||||
|
||||
Training: rank 8, scale 20, 8 layers, 800 iters, lr 1e-4. **Best val loss
|
||||
2.468 at iter 400** — the lowest of any version (v5 2.732, v4 2.785).
|
||||
|
||||
## Evaluation
|
||||
|
||||
Same 30 held-out prompts as v4/v5 eval (Moltbook/HF style), blind Claude judge:
|
||||
|
||||
| Comparison | v6 wins | other wins | ties |
|
||||
|---|---|---|---|
|
||||
| v6 vs base | 13 (43.3%) | 15 (50.0%) | 2 |
|
||||
| v6 vs v5 | 15 (50.0%) | 14 (46.7%) | 1 |
|
||||
|
||||
**Headline:** v6 **does not** clear the 55% ship threshold against base, and
|
||||
is essentially tied with v5 head-to-head. Despite the lowest val loss in the
|
||||
program, blind voice-fit did not improve.
|
||||
|
||||
## Why v6 didn't beat base (despite lowest val loss)
|
||||
|
||||
Three patterns hold across v4 → v5 → v6:
|
||||
1. **Val loss and blind-eval-quality are decoupled.** v6 has the best val loss
|
||||
and the worst blind win-rate. Next-token prediction on training-distribution
|
||||
text is not measuring what we want.
|
||||
2. **Each new version ties the previous one** (~50:50 against the prior). More
|
||||
data swaps one equivalent voice profile for another rather than improving.
|
||||
3. **Eval methodology has ~±15% judge-variance.** v5 era used Claude subagents,
|
||||
v6 used direct Claude judging. The "v5 won 67%, v6 won 43%" gap is partially
|
||||
real, partially the judge swap.
|
||||
|
||||
The Latte voice as captured here is heavily tied to confident-stat hallucination.
|
||||
Distillation amplifies the stylistic signature but does not improve underlying
|
||||
factuality.
|
||||
|
||||
## Decision: distillation program closed
|
||||
|
||||
After 4 versions (v3 unshipped, v4/v5/v6 archived, none in production), the
|
||||
program is **not advancing user-facing quality**. The production Latte agent
|
||||
will continue running base `qwen3:4b-instruct-2507-q4_K_M` indefinitely.
|
||||
|
||||
Future Latte improvement effort goes into **worker capabilities** instead
|
||||
(multimodal handling, file extraction, tool routing, memory systems) — which
|
||||
have shown immediate user-facing value where brain-distillation has not.
|
||||
|
||||
Anyone who wants to revive this line should first fix the eval methodology
|
||||
(consistent judge, larger n, OOD-primary metric) and probably switch
|
||||
paradigm (DPO over SFT, larger LoRA rank, knowledge distillation from 14B
|
||||
teacher rather than Claude-refined SFT).
|
||||
|
||||
## Usage (if you really want)
|
||||
|
||||
### MLX (Apple Silicon, recommended for inference)
|
||||
```python
|
||||
from mlx_lm import load, generate
|
||||
model, tokenizer = load(
|
||||
"mlx-community/Qwen3-4B-Instruct-2507-4bit",
|
||||
adapter_path="./",
|
||||
)
|
||||
print(generate(model, tokenizer, "Your prompt", max_tokens=200))
|
||||
```
|
||||
|
||||
### llama.cpp / Ollama
|
||||
```
|
||||
FROM qwen3-4b-latte-v6-Q4_K_M.gguf
|
||||
PARAMETER temperature 0.7
|
||||
PARAMETER top_k 20
|
||||
PARAMETER top_p 0.8
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0 (inherits from Qwen3-4B-Instruct-2507, © Alibaba Cloud).
|
||||
Reference in New Issue
Block a user