初始化项目,由ModelHub XC社区提供模型
Model: IIGroup/X-Coder-SFT-Qwen3-8B Source: Original Platform
This commit is contained in:
100
README.md
Normal file
100
README.md
Normal file
@@ -0,0 +1,100 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model:
|
||||
- Qwen/Qwen3-8B-Base
|
||||
datasets:
|
||||
- IIGroup/X-Coder-SFT-376k
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- code
|
||||
- sft
|
||||
- competitive-programming
|
||||
---
|
||||
|
||||
# X-Coder-SFT-Qwen3-8B
|
||||
|
||||
X-Coder-SFT-Qwen3-8B is a code generation model fine-tuned on fully synthetic instruction data, designed for competitive programming tasks. It serves as the foundation for subsequent RLVR training.
|
||||
|
||||
## Model Description
|
||||
|
||||
- **Base Model**: [Qwen/Qwen3-8B-Base](https://huggingface.co/Qwen/Qwen3-8B-Base)
|
||||
- **Training Method**: Supervised Fine-Tuning (SFT)
|
||||
- **Training Data**: [IIGroup/X-Coder-SFT-376k](https://huggingface.co/datasets/IIGroup/X-Coder-SFT-376k)
|
||||
- **Parameters**: 8B
|
||||
|
||||
## Training
|
||||
|
||||
This model was trained using [ms-swift](https://github.com/modelscope/ms-swift). For training details and code, please refer to the [X-Coder GitHub repository](https://github.com/JieWu02/X-Coder).
|
||||
|
||||
### Training Hyperparameters
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| Base Model | Qwen/Qwen3-8B-Base |
|
||||
| Training Type | Full Parameter |
|
||||
| Epochs | 8 |
|
||||
| Global Batch Size | 128 |
|
||||
| Learning Rate | 5e-5 |
|
||||
| Max Grad Norm | 1.0 |
|
||||
| Max Length | 32768 |
|
||||
| Torch Dtype | bfloat16 |
|
||||
| DeepSpeed | Zero3 Offload (80GB VRAM) / Zero2 (142GB VRAM) |
|
||||
| Packing | True (2x faster training, slightly worse performance) |
|
||||
|
||||
## Performance on LiveCodeBench v5.
|
||||
|
||||

|
||||
|
||||
## Recommended Inference Parameters
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| temperature | 0.6 |
|
||||
| top_p | 0.95 |
|
||||
| top_k | 20 (or -1 to disable) |
|
||||
| max_new_tokens | 32768 |
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_name = "IIGroup/X-Coder-SFT-Qwen3-8B"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
|
||||
|
||||
prompt = "Write a Python function to solve the two sum problem."
|
||||
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=32768,
|
||||
temperature=0.6,
|
||||
top_p=0.95,
|
||||
top_k=20,
|
||||
do_sample=True
|
||||
)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Related Models
|
||||
|
||||
- **RL Model**: [IIGroup/X-Coder-RL-Qwen3-8B](https://huggingface.co/IIGroup/X-Coder-RL-Qwen3-8B) - RLVR trained version achieving 64.0 on LiveCodeBench
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{wu2026xcoderadvancingcompetitiveprogramming,
|
||||
title={X-Coder: Advancing Competitive Programming with Fully Synthetic Tasks, Solutions, and Tests},
|
||||
author={Jie Wu and Haoling Li and Xin Zhang and Jiani Guo and Jane Luo and Steven Liu and Yangyu Huang and Ruihang Chu and Scarlett Li and Yujiu Yang},
|
||||
year={2026},
|
||||
eprint={2601.06953},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CL},
|
||||
url={https://arxiv.org/abs/2601.06953},
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the Apache License 2.0.
|
||||
Reference in New Issue
Block a user