初始化项目,由ModelHub XC社区提供模型
Model: n4/Qwen3-4B-Instruct-2507-sft_166 Source: Original Platform
This commit is contained in:
93
README.md
Normal file
93
README.md
Normal file
@@ -0,0 +1,93 @@
|
||||
---
|
||||
base_model: Qwen/Qwen3-4B-Instruct-2507
|
||||
datasets:
|
||||
- u-10bei/structured_data_with_cot_dataset_512_v5
|
||||
- daichira/structured-5k-mix-sft
|
||||
language:
|
||||
- en
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- lora
|
||||
- merged
|
||||
- structured-output
|
||||
---
|
||||
|
||||
# Qwen3-4B-Instruct-2507-sft_166 (merged LoRA, multi-stage SFT)
|
||||
|
||||
This repository provides a **merged full model** fine-tuned from **Qwen/Qwen3-4B-Instruct-2507** using **LoRA**.
|
||||
|
||||
**Important:** This repository does **NOT** provide separate LoRA adapter weights.
|
||||
It contains **merged model weights only** (the adapter is not uploaded).
|
||||
|
||||
## Training Objective
|
||||
|
||||
This model is fine-tuned to improve **structured output accuracy**
|
||||
(JSON / YAML / XML / TOML / CSV).
|
||||
|
||||
Note:
|
||||
- This README focuses on the competition-required *model card structure*.
|
||||
- If you need more training/implementation details, please refer to your training logs or scripts.
|
||||
|
||||
## Training Configuration
|
||||
|
||||
- Base model: Qwen/Qwen3-4B-Instruct-2507
|
||||
- Method: LoRA (adapters merged after training)
|
||||
- Max sequence length: 512
|
||||
- Epochs: 2
|
||||
- LoRA: r=1, alpha=2
|
||||
|
||||
### Multi-stage SFT (2 stages)
|
||||
|
||||
**Stage 1 (YAML-focused SFT)**
|
||||
- Data: “to YAML” subset only (from the sources listed below)
|
||||
- Learning rate: 2.0e-4
|
||||
|
||||
**Stage 2 (XML-focused SFT)**
|
||||
- Data: “to XML” subset only (from the sources listed below)
|
||||
- Learning rate: 1.1e-4
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
repo_id = "n4/Qwen3-4B-Instruct-2507-sft_166"
|
||||
tok = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(repo_id, device_map="auto", trust_remote_code=True)
|
||||
|
||||
user_query = "Please output the following information in JSON format: Name=naisy, Age=714"
|
||||
messages = [{"role": "user", "content": user_query}]
|
||||
prompt = tok.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
)
|
||||
|
||||
inputs = tok(prompt, return_tensors="pt").to(model.device)
|
||||
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
|
||||
gen_ids = out[0][inputs["input_ids"].shape[1]:]
|
||||
text = tok.decode(gen_ids, skip_special_tokens=True)
|
||||
print(text)
|
||||
```
|
||||
|
||||
## Sources & Terms (IMPORTANT)
|
||||
|
||||
Training data:
|
||||
- u-10bei/structured_data_with_cot_dataset_512_v5
|
||||
- daichira/structured-5k-mix-sft
|
||||
|
||||
Dataset License:
|
||||
- u-10bei/structured_data_with_cot_dataset_512_v5: MIT License
|
||||
- daichira/structured-5k-mix-sft: CC-BY-4.0
|
||||
|
||||
Compliance:
|
||||
Users must comply with:
|
||||
- the dataset licenses above (including attribution requirements for CC-BY-4.0), and
|
||||
- the base model's original terms of use (apache-2.0).
|
||||
|
||||
## Limitations
|
||||
|
||||
- Structured outputs may still fail under very long, deeply nested, or ambiguous schemas.
|
||||
- Validate outputs (e.g., JSON parse / XML validation) in downstream use cases.
|
||||
Reference in New Issue
Block a user