---
base_model: Qwen/Qwen3-4B-Instruct-2507
datasets:
- u-10bei/structured_data_with_cot_dataset_512_v5
- daichira/structured-5k-mix-sft
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- lora
- merged
- structured-output
---

# Qwen3-4B-Instruct-2507-sft_166 (merged LoRA, multi-stage SFT)

This repository provides a **merged full model** fine-tuned from **Qwen/Qwen3-4B-Instruct-2507** using **LoRA**.

**Important:** This repository does **NOT** provide separate LoRA adapter weights.
It contains **merged model weights only** (the adapter is not uploaded).

## Training Objective

This model is fine-tuned to improve **structured output accuracy**
(JSON / YAML / XML / TOML / CSV).

Note:
- This README focuses on the competition-required *model card structure*.
- If you need more training/implementation details, please refer to your training logs or scripts.

## Training Configuration

- Base model: Qwen/Qwen3-4B-Instruct-2507
- Method: LoRA (adapters merged after training)
- Max sequence length: 512
- Epochs: 2
- LoRA: r=1, alpha=2

### Multi-stage SFT (2 stages)

**Stage 1 (YAML-focused SFT)**
- Data: “to YAML” subset only (from the sources listed below)
- Learning rate: 2.0e-4

**Stage 2 (XML-focused SFT)**
- Data: “to XML” subset only (from the sources listed below)
- Learning rate: 1.1e-4

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "n4/Qwen3-4B-Instruct-2507-sft_166"
tok = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, device_map="auto", trust_remote_code=True)

user_query = "Please output the following information in JSON format: Name=naisy, Age=714"
messages = [{"role": "user", "content": user_query}]
prompt = tok.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
gen_ids = out[0][inputs["input_ids"].shape[1]:]
text = tok.decode(gen_ids, skip_special_tokens=True)
print(text)
```

## Sources & Terms (IMPORTANT)

Training data:
- u-10bei/structured_data_with_cot_dataset_512_v5
- daichira/structured-5k-mix-sft

Dataset License:
- u-10bei/structured_data_with_cot_dataset_512_v5: MIT License
- daichira/structured-5k-mix-sft: CC-BY-4.0

Compliance:
Users must comply with:
- the dataset licenses above (including attribution requirements for CC-BY-4.0), and
- the base model's original terms of use (apache-2.0).

## Limitations

- Structured outputs may still fail under very long, deeply nested, or ambiguous schemas.
- Validate outputs (e.g., JSON parse / XML validation) in downstream use cases.