---
base_model: Qwen/Qwen3-1.7B
license: apache-2.0
pipeline_tag: text-generation
library_name: transformers
tags:
- qwen3
- cogs
- json
- structured-output
- mlx
---

# qwen3-1.7b-cogs-ingest

**Merged** full model (Qwen/Qwen3-1.7B + LoRA, weights fused, bf16) — the student
model for the [cogs](https://github.com/trunksio/cogs) `ingest` pipeline. Use
this repo when you want a standalone model to serve or convert; the standalone
LoRA adapter is at
[`lewisdog/qwen3-1.7b-cogs-ingest-lora`](https://huggingface.co/lewisdog/qwen3-1.7b-cogs-ingest-lora).

It replaces a large teacher LLM as a local, OpenAI-compatible provider for four
structured-output tasks, emitting **compact JSON exactly as the cogs runtime
parses it**:

| task            | required top-level JSON keys                |
|-----------------|---------------------------------------------|
| `extract`       | `summary`, `key_claims` (+ quotes/entities) |
| `suggest_links` | `linked_claims`                             |
| `page_update`   | `topic`, `section_md`, `relevant`           |
| `contradiction` | `findings`                                  |

## Apple silicon (MLX / omlx)

```sh
# converts + 4-bit quantizes into an MLX model dir
python3 -m mlx_lm.convert --hf-path lewisdog/qwen3-1.7b-cogs-ingest -q --mlx-path qwen3-cogs-ingest-mlx
# then drop into ~/.omlx/models and point cogs [llm] at it
```

## ⚠️ Serving: avoid pure greedy decoding

The model learned the schema well, but under **pure greedy (temperature 0, no
penalty)** it degenerates on the long list-valued tasks (`extract`,
`suggest_links`): it keeps appending list items and never emits `<|im_end|>`,
leaving valid-but-**unterminated** JSON. Fix with either:

- `repetition_penalty ≈ 1.1` (keeps determinism), **or**
- Qwen non-thinking sampling: `temperature=0.7, top_p=0.8, top_k=20`
  (this model's bundled `generation_config.json` already samples at temp 0.6,
  which sidesteps the issue — just don't override it to temp-0 greedy).

With either, a 5-sample / 4-task JSON parse eval is **5/5 (100%) schema-exact**
(vs. 3/5 at plain greedy). Use `enable_thinking=False` and stop on `<|im_end|>`.

## Transformers usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "lewisdog/qwen3-1.7b-cogs-ingest", dtype="bfloat16", device_map="auto"
).eval()
tok = AutoTokenizer.from_pretrained("lewisdog/qwen3-1.7b-cogs-ingest")

enc = tok.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True,
).to(model.device)
out = model.generate(**enc, max_new_tokens=2048, do_sample=False,
                     repetition_penalty=1.1, pad_token_id=tok.pad_token_id)
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Training

LoRA SFT (TRL) on 1,921 `cogs distill` chat pairs, 2 epochs, effective batch 16,
max_seq 8192, lr 1e-4 cosine, bf16, on an NVIDIA DGX Spark (GB10).
Train loss 2.55 → 1.41; eval loss 1.125 → 1.118 (still decreasing); eval
token-accuracy 0.756. Full details: adapter repo card + `RESULTS.md`.

- PEFT 0.19.1 · TRL 1.7.1 · Transformers 5.13.0 · PyTorch 2.12.1+cu130