---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
library_name: transformers
pipeline_tag: text-generation
tags:
- logical-reasoning
- sft
- qwen2.5
---

# Qwen2.5-7B-Instruct — ProofDAG SFT

Full fine-tune của **Qwen/Qwen2.5-7B-Instruct** trên dataset ProofDAG (True / False / Uncertain).


## Training

| | |
|---|---|
| Data | 5640 train / 330 val (multi-turn chat) |
| Hardware | 8× L40 (FSDP FULL_SHARD, bf16) |
| Global batch | 128, max_len 4096 |
| LR | 1e-6 cosine, warmup 0.03 |
| Epochs | 3 (132 steps, 6h 48m) |
| Final train / eval loss | 0.207 / 0.251 |

## Quick start

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

mid = "NhatCuong22/qwen2.5-7b-proofdag-sft"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype="bfloat16", device_map="auto")

messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": "Premises:\n1. If it rains, the ground is wet.\n2. It rains.\n\nProposed conclusion: The ground is wet."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))
```

License: Apache-2.0