Qwen3-1.7B-UltraChat-SFT/README.md

---
library_name: transformers
tags:
- sft
license: apache-2.0
datasets:
- HuggingFaceH4/ultrachat_200k
base_model:
- Qwen/Qwen3-1.7B-Base
pipeline_tag: text-generation
---

# Qwen3-1.7B UltraChat SFT
This model is a supervised fine-tuned variant of [Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base). The original model is a pretrained language model that is not instruction-tuned for conversational tasks. To enable instruction-following and dialogue capabilities, it was fine-tuned on the UltraChat-200k dataset, resulting in a helpful conversational assistant while retaining the base model's general language understanding and knowledge. This model does not support thinking, vision or tool calling yet.

Fine-tuning was performed using LoRA adapters, with additional training applied to the token embedding layer (`embed_tokens`) and the language modeling head (`lm_head`). These components were optimized using a lower learning rate than the LoRA parameters, allowing for gradual adaptation of the vocabulary representations and output distribution. This was particularly beneficial for the special tokens, whose embedding vectors in the base model were initially identical, enabling them to learn meaningful and distinct representations during fine-tuning.

View the better direct preference optimized version of this model: [Qwen3-1.7B Chat](https://huggingface.co/ayushshah/Qwen3-1.7B-Chat)<br>
View the [GitHub](https://github.com/AyushShahh/Qwen3-1.7B-Post-Training/tree/main) repo for Post-training pipeline.

## Model Overview
**Qwen3-1.7B-UltraChat-SFT** has following features:
- **Type**: Causal Language Models
- **Training Stage**: Pretraining & Post-training
- **Number of Parameters** 1.7B
- **Number of Paramaters (Non-Embedding)**: 1.4B
- **Number of Layers**: 28
- **Number of Attention Heads (GQA)**: 16 for Q and 8 for KV
- **Context Length**: 32,768

### Training
- **Method**: Supervised Fine-Tuning (SFT)
- **Framework**: [Unsloth](https://unsloth.ai/)
- **Fine-Tuning**: LoRA
- **Precision**: BF16
- **Environment**: Windows Subsystem for Linux (WSL)
- **GPU**: RTX 4090 24 GB, using FlashAttention 2, xFormers and Triton
- **Context Length**: 2048 tokens
- **Dataset**: [UltraChat 200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k). Conversations longer than the context window were trimmed before training.

### Training Configuration

| Parameter | Value |
|-----------|-------|
| Epochs | 1 |
| Batch Size | 4 |
| Gradient Accumulation | 8 |
| Effective Batch Size | 32 |
| Optimizer | AdamW Fused |
| Learning rate | 2.5e-4 |
| Embedding Learning Rate | 5e-5 |
| LoRA Rank | 32 |
| LoRA Alpha | 64 |
| Scheduler | Cosine |
| Warmup | 3% |
| Gradient Checkpointing | False |
| Packing | True |

LoRA is applied tp `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj` and `down_proj`.

## Evaluation

The evaluation was performed using EleutherAI's [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness/).

| Benchmark | Score |
|-----------|-------|
| MMLU-Pro | 37.91 |
| GSM8K | 71.27 |
| HellaSwag | 62.28 |
| TruthfulQA <sub>mc2</sub> | 51.62 |
| ARC Challenge | 44.97 |
| IFEval <sub>strict prompt</sub> | 33.09 |

## Usage
You can either use **Unsloth** or **transformers**.

### Unsloth (Recommended)
Install Unsloth following these [docs](https://unsloth.ai/docs/get-started/install/pip-install#unsloth-core). Install [Flash Attention](https://github.com/dao-ailab/flash-attention) if possible. Check if Triton and xFormers are installed.

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "ayushshah/Qwen3-1.7B-UltraChat-SFT",
    max_seq_length = 2048,
    dtype = "bfloat16",
    load_in_4bit = False,
)

FastLanguageModel.for_inference(model)

prompt = "Explain gradient descent simply."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt",
).to(model.device)

outputs = model.generate(
    input_ids = inputs,
    max_new_tokens = 2048,
)

input_length = inputs.shape[1]
new_tokens = outputs[0][input_length:]

raw_text = tokenizer.decode(new_tokens, skip_special_tokens=True)
print(raw_text)
```

### transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("ayushshah/Qwen3-1.7B-UltraChat-SFT")
model = AutoModelForCausalLM.from_pretrained(
    "ayushshah/Qwen3-1.7B-UltraChat-SFT",
    torch_dtype="auto",
    device_map="auto"
)

prompt = "Explain gradient descent simply."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt",
).to(model.device)

outputs = model.generate(
    input_ids = inputs["input_ids"],
    max_new_tokens = 2048,
)

input_length = inputs["input_ids"].shape[1]
new_tokens = outputs[0][input_length:]

raw_text = tokenizer.decode(new_tokens, skip_special_tokens=True)
print(raw_text)
```


## Limitations
This model provides the base model with conversational and instruction-following capabilities through supervised fine-tuning. Responses are generally concise and direct, making it a strong foundation for further preference alignment (e.g., DPO, RLHF, PPO) or domain-specific fine-tuning. While suitable for general instruction-following, responses may occasionally lack the depth, natural conversational flow, or helpfulness expected from preference-aligned chat models.

## Citation
```
@misc{qwen3technicalreport,
      title={Qwen3 Technical Report}, 
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388}, 
}
```