初始化项目，由ModelHub XC社区提供模型
Model: ryzdfm/qwen2.5-coder-3b-claude_opus_4.6-distilled Source: Original Platform
This commit is contained in:
ModelHub XC
170
README.md
Normal file
170
README.md
Normal file
@@ -0,0 +1,170 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- qwen2.5
|
||||
- qwen2.5-coder
|
||||
- unsloth
|
||||
- reasoning
|
||||
- chain-of-thought
|
||||
- coding
|
||||
- distillation
|
||||
- claude
|
||||
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
|
||||
datasets:
|
||||
- nohurry/Opus-4.6-Reasoning-3000x-filtered
|
||||
- TeichAI/claude-4.5-opus-high-reasoning-250x
|
||||
- Jackrong/Qwen3.5-reasoning-700x
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# 🌟 Qwen2.5-Coder-3B — Claude Opus 4.6 Reasoning Distilled
|
||||
|
||||
A compact, fast, locally-runnable coding model fine-tuned on top of **Qwen2.5-Coder-3B-Instruct** using high-quality reasoning trajectories distilled from **Claude 4.6 Opus**. Designed to run efficiently on consumer hardware with as little as **4GB VRAM** at ~88 tokens/sec.
|
||||
|
||||
---
|
||||
|
||||
## 💡 Model Introduction
|
||||
|
||||
**Qwen2.5-Coder-3B-Claude-Opus-4.6-Distilled** combines the strong code generation foundation of Qwen2.5-Coder with the structured, step-by-step reasoning style of Claude 4.6 Opus. Through Supervised Fine-Tuning (SFT) with LoRA, the model learns to think through problems carefully inside `<think>` tags before delivering precise, well-structured answers.
|
||||
|
||||
Unlike larger distilled models, this 3B model is built for **real local inference** — fast, private, and fits comfortably in 4GB VRAM.
|
||||
|
||||
---
|
||||
|
||||
## 🧠 Reasoning Style
|
||||
|
||||
The model adopts Claude Opus's structured reasoning pattern:
|
||||
|
||||
```
|
||||
<think>
|
||||
Let me analyze this carefully.
|
||||
|
||||
1. Identify the core objective.
|
||||
2. Break down into subcomponents.
|
||||
3. Consider edge cases and constraints.
|
||||
4. Formulate and verify the solution.
|
||||
</think>
|
||||
|
||||
[Final clean answer here]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗺️ Training Pipeline
|
||||
|
||||
```
|
||||
Base Model (Qwen/Qwen2.5-Coder-3B-Instruct)
|
||||
│
|
||||
▼
|
||||
Supervised Fine-Tuning (SFT) + LoRA (r=16)
|
||||
│ • 3,209 high-quality Claude reasoning samples
|
||||
│ • Unsloth 2x faster training
|
||||
│ • 1 epoch on T4 GPU (~46 mins)
|
||||
│ • Final loss: 0.88
|
||||
▼
|
||||
Qwen2.5-Coder-3B-Claude-Opus-4.6-Distilled
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Training Details
|
||||
|
||||
| Parameter | Value |
|
||||
|---|---|
|
||||
| Base Model | Qwen/Qwen2.5-Coder-3B-Instruct |
|
||||
| Framework | Unsloth 2026.3 |
|
||||
| LoRA rank | 16 |
|
||||
| LoRA alpha | 16 |
|
||||
| Trainable params | 29,933,568 (0.96%) |
|
||||
| Batch size | 16 (4 × 4 grad accum) |
|
||||
| Learning rate | 2e-4 |
|
||||
| Epochs | 1 |
|
||||
| Max seq length | 4096 |
|
||||
| Final train loss | 0.88 |
|
||||
| GPU | Tesla T4 (16GB) |
|
||||
| Training time | ~46 mins |
|
||||
|
||||
---
|
||||
|
||||
## 📚 Datasets Used
|
||||
|
||||
| Dataset | Samples | Purpose |
|
||||
|---|---|---|
|
||||
| [nohurry/Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | 2,326 | Claude 4.6 Opus reasoning trajectories |
|
||||
| [TeichAI/claude-4.5-opus-high-reasoning-250x](https://huggingface.co/datasets/TeichAI/claude-4.5-opus-high-reasoning-250x) | 250 | High-intensity structured reasoning |
|
||||
| [Jackrong/Qwen3.5-reasoning-700x](https://huggingface.co/datasets/Jackrong/Qwen3.5-reasoning-700x) | 633 | Step-by-step reasoning diversity |
|
||||
| **Total** | **3,209** | |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Running Locally
|
||||
|
||||
### Via Ollama (easiest)
|
||||
```bash
|
||||
ollama run hf.co/ryzdfm/qwen2.5-coder-3b-claude_opus_4.6-distilled
|
||||
```
|
||||
|
||||
### Via llama.cpp (for GPU acceleration)
|
||||
```bash
|
||||
./llama-cli.exe \
|
||||
-m qwen2.5-coder-3b-claude_opus_4.6-distilled.Q4_K_M.gguf \
|
||||
-ngl 99 \
|
||||
--flash-attn on \
|
||||
--jinja \
|
||||
-cnv \
|
||||
--repeat-penalty 1.1 \
|
||||
-p "You are a helpful assistant that thinks step by step."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🌟 Core Capabilities
|
||||
|
||||
- **Structured Reasoning** — thinks through problems step by step in `<think>` blocks before answering
|
||||
- **Code Generation** — built on Qwen2.5-Coder, strong at Python, JavaScript, algorithms
|
||||
- **Math & Logic** — correctly solves multi-step problems with verification
|
||||
- **Fast Local Inference** — 88 t/s on RTX 3050 4GB, fully GPU-accelerated
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Hardware Requirements
|
||||
|
||||
| Quantization | VRAM | Speed (RTX 3050) |
|
||||
|---|---|---|
|
||||
| Q4_K_M (this file) | ~2.1 GB | ~88 t/s |
|
||||
| Q3_K_M | ~1.7 GB | ~95 t/s |
|
||||
| Q8_0 | ~3.3 GB | ~70 t/s |
|
||||
|
||||
Runs comfortably on **4GB VRAM** laptops and desktops.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Limitations
|
||||
|
||||
- **3B scale** — will struggle with very long multi-file code generation or complex system design
|
||||
- **1 epoch training** — reasoning style is distilled but not as deep as larger models
|
||||
- **Hallucination risk** — like all LLMs, may produce incorrect facts; always verify outputs
|
||||
|
||||
---
|
||||
|
||||
## 🙏 Acknowledgements
|
||||
|
||||
- [Unsloth AI](https://unsloth.ai/) for making fine-tuning accessible on consumer hardware
|
||||
- [nohurry](https://huggingface.co/nohurry), [TeichAI](https://huggingface.co/TeichAI), and [Jackrong](https://huggingface.co/Jackrong) for the high-quality distillation datasets
|
||||
- Qwen team for the excellent Qwen2.5-Coder base model
|
||||
|
||||
---
|
||||
|
||||
## 📖 Citation
|
||||
|
||||
```bibtex
|
||||
@misc{ryzdfm_qwen25coder_claude_distilled,
|
||||
title = {Qwen2.5-Coder-3B Claude Opus 4.6 Reasoning Distilled},
|
||||
author = {ryzdfm},
|
||||
year = {2026},
|
||||
publisher = {Hugging Face},
|
||||
howpublished = {\url{https://huggingface.co/ryzdfm/qwen2.5-coder-3b-claude_opus_4.6-distilled}}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user