初始化项目,由ModelHub XC社区提供模型
Model: lewisdog/qwen3-1.7b-cogs-ingest Source: Original Platform
This commit is contained in:
82
README.md
Normal file
82
README.md
Normal file
@@ -0,0 +1,82 @@
|
||||
---
|
||||
base_model: Qwen/Qwen3-1.7B
|
||||
license: apache-2.0
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
tags:
|
||||
- qwen3
|
||||
- cogs
|
||||
- json
|
||||
- structured-output
|
||||
- mlx
|
||||
---
|
||||
|
||||
# qwen3-1.7b-cogs-ingest
|
||||
|
||||
**Merged** full model (Qwen/Qwen3-1.7B + LoRA, weights fused, bf16) — the student
|
||||
model for the [cogs](https://github.com/trunksio/cogs) `ingest` pipeline. Use
|
||||
this repo when you want a standalone model to serve or convert; the standalone
|
||||
LoRA adapter is at
|
||||
[`lewisdog/qwen3-1.7b-cogs-ingest-lora`](https://huggingface.co/lewisdog/qwen3-1.7b-cogs-ingest-lora).
|
||||
|
||||
It replaces a large teacher LLM as a local, OpenAI-compatible provider for four
|
||||
structured-output tasks, emitting **compact JSON exactly as the cogs runtime
|
||||
parses it**:
|
||||
|
||||
| task | required top-level JSON keys |
|
||||
|-----------------|---------------------------------------------|
|
||||
| `extract` | `summary`, `key_claims` (+ quotes/entities) |
|
||||
| `suggest_links` | `linked_claims` |
|
||||
| `page_update` | `topic`, `section_md`, `relevant` |
|
||||
| `contradiction` | `findings` |
|
||||
|
||||
## Apple silicon (MLX / omlx)
|
||||
|
||||
```sh
|
||||
# converts + 4-bit quantizes into an MLX model dir
|
||||
python3 -m mlx_lm.convert --hf-path lewisdog/qwen3-1.7b-cogs-ingest -q --mlx-path qwen3-cogs-ingest-mlx
|
||||
# then drop into ~/.omlx/models and point cogs [llm] at it
|
||||
```
|
||||
|
||||
## ⚠️ Serving: avoid pure greedy decoding
|
||||
|
||||
The model learned the schema well, but under **pure greedy (temperature 0, no
|
||||
penalty)** it degenerates on the long list-valued tasks (`extract`,
|
||||
`suggest_links`): it keeps appending list items and never emits `<|im_end|>`,
|
||||
leaving valid-but-**unterminated** JSON. Fix with either:
|
||||
|
||||
- `repetition_penalty ≈ 1.1` (keeps determinism), **or**
|
||||
- Qwen non-thinking sampling: `temperature=0.7, top_p=0.8, top_k=20`
|
||||
(this model's bundled `generation_config.json` already samples at temp 0.6,
|
||||
which sidesteps the issue — just don't override it to temp-0 greedy).
|
||||
|
||||
With either, a 5-sample / 4-task JSON parse eval is **5/5 (100%) schema-exact**
|
||||
(vs. 3/5 at plain greedy). Use `enable_thinking=False` and stop on `<|im_end|>`.
|
||||
|
||||
## Transformers usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"lewisdog/qwen3-1.7b-cogs-ingest", dtype="bfloat16", device_map="auto"
|
||||
).eval()
|
||||
tok = AutoTokenizer.from_pretrained("lewisdog/qwen3-1.7b-cogs-ingest")
|
||||
|
||||
enc = tok.apply_chat_template(
|
||||
messages, add_generation_prompt=True, enable_thinking=False,
|
||||
return_tensors="pt", return_dict=True,
|
||||
).to(model.device)
|
||||
out = model.generate(**enc, max_new_tokens=2048, do_sample=False,
|
||||
repetition_penalty=1.1, pad_token_id=tok.pad_token_id)
|
||||
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Training
|
||||
|
||||
LoRA SFT (TRL) on 1,921 `cogs distill` chat pairs, 2 epochs, effective batch 16,
|
||||
max_seq 8192, lr 1e-4 cosine, bf16, on an NVIDIA DGX Spark (GB10).
|
||||
Train loss 2.55 → 1.41; eval loss 1.125 → 1.118 (still decreasing); eval
|
||||
token-accuracy 0.756. Full details: adapter repo card + `RESULTS.md`.
|
||||
|
||||
- PEFT 0.19.1 · TRL 1.7.1 · Transformers 5.13.0 · PyTorch 2.12.1+cu130
|
||||
Reference in New Issue
Block a user