初始化项目,由ModelHub XC社区提供模型
Model: GetSoloTech/Qwen3-Code-Reasoning-4B-GGUF Source: Original Platform
This commit is contained in:
41
.gitattributes
vendored
Normal file
41
.gitattributes
vendored
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen3-Code-Reasoning-4B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen3-Code-Reasoning-4B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen3-Code-Reasoning-4B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen3-Code-Reasoning-4B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen3-Code-Reasoning-4B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Qwen3-Code-Reasoning-4B.f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Qwen3-Code-Reasoning-4B.Q3_K_M.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e7e26754634c8e8a0cd00c53dd60bf675f8c710b957405f47e1914f83ed207c3
|
||||||
|
size 2075618656
|
||||||
3
Qwen3-Code-Reasoning-4B.Q4_K_M.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:83a25bb48598e4b6c9f0e7dd5892e003a1c52a0ae644df8c9e0008c9dda62d8c
|
||||||
|
size 2497281376
|
||||||
3
Qwen3-Code-Reasoning-4B.Q5_K_M.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e77c9bad036097853cf7a81471e718e31a3261e6faff9de48fc3b39bb6cfb645
|
||||||
|
size 2889514336
|
||||||
3
Qwen3-Code-Reasoning-4B.Q6_K.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2b5d5661f3370fdb82b55a70aa04b437b75e679acc5cf1a0cff5d1a9903b09db
|
||||||
|
size 3306261856
|
||||||
3
Qwen3-Code-Reasoning-4B.Q8_0.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ca068d56c17ff84d7d2ebb6000378894b4647f3d4461ebeff7826674047c6ba4
|
||||||
|
size 4280405856
|
||||||
3
Qwen3-Code-Reasoning-4B.f16.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:224c20fd3ca6891b463618e086c1ee9201a5a3223ea6bd4df2381cc8acf47652
|
||||||
|
size 8051285856
|
||||||
180
README.md
Normal file
180
README.md
Normal file
@@ -0,0 +1,180 @@
|
|||||||
|
---
|
||||||
|
license: apache-2.0
|
||||||
|
datasets:
|
||||||
|
- GetSoloTech/Code-Reasoning
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
base_model:
|
||||||
|
- GetSoloTech/Qwen3-Code-Reasoning-4B
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
tags:
|
||||||
|
- coding
|
||||||
|
- reasoning
|
||||||
|
- problem-solving
|
||||||
|
- algorithms
|
||||||
|
- python
|
||||||
|
- c++
|
||||||
|
---
|
||||||
|
|
||||||
|
# GetSoloTech/Qwen3-Code-Reasoning-4B-GGUF
|
||||||
|
|
||||||
|
This is the GGUF quantized version of the [Qwen3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Qwen3-Code-Reasoning-4B) model, specifically optimized for competitive programming and code reasoning tasks. This model has been trained on the high-quality Code-Reasoning dataset to enhance its capabilities in solving complex programming problems with detailed reasoning.
|
||||||
|
|
||||||
|
|
||||||
|
## 🚀 Key Features
|
||||||
|
|
||||||
|
* **Enhanced Code Reasoning**: Specifically trained on competitive programming problems
|
||||||
|
* **Thinking Capabilities**: Inherits the advanced reasoning capabilities from the base model
|
||||||
|
* **High-Quality Solutions**: Trained on solutions with ≥85% test case pass rates
|
||||||
|
* **Structured Output**: Optimized for generating well-reasoned programming solutions
|
||||||
|
* **Efficient Inference**: GGUF format enables fast inference on CPU and GPU
|
||||||
|
* **Multiple Quantization Levels**: Available in various precision levels for different hardware requirements
|
||||||
|
|
||||||
|
### Dataset Statistics
|
||||||
|
|
||||||
|
* **Split**: Python
|
||||||
|
* **Source**: High-quality competitive programming problems from TACO, APPS, CodeContests, and Codeforces
|
||||||
|
* **Quality Filter**: Only correctly solved problems with ≥85% test case pass rates
|
||||||
|
|
||||||
|
## 🔧 Usage
|
||||||
|
|
||||||
|
### Using with llama.cpp
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Download the model (choose your preferred quantization)
|
||||||
|
wget https://huggingface.co/GetSoloTech/Qwen3-Code-Reasoning-4B-GGUF/resolve/main/qwen3-code-reasoning-4b.Q4_K_M.gguf
|
||||||
|
|
||||||
|
# Run inference
|
||||||
|
./llama.cpp -m qwen3-code-reasoning-4b.Q4_K_M.gguf -n 4096 --repeat_penalty 1.1 -p "You are an expert competitive programmer. Read the problem and produce a correct, efficient solution. Include reasoning if helpful.\n\nProblem: Your programming problem here..."
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using with Python (llama-cpp-python)
|
||||||
|
|
||||||
|
```python
|
||||||
|
from llama_cpp import Llama
|
||||||
|
|
||||||
|
# Load the model
|
||||||
|
llm = Llama(
|
||||||
|
model_path="./qwen3-code-reasoning-4b.Q4_K_M.gguf",
|
||||||
|
n_ctx=4096,
|
||||||
|
n_threads=4
|
||||||
|
)
|
||||||
|
|
||||||
|
# Prepare input for competitive programming problem
|
||||||
|
prompt = """You are an expert competitive programmer. Read the problem and produce a correct, efficient solution. Include reasoning if helpful.
|
||||||
|
|
||||||
|
Problem: Your programming problem here..."""
|
||||||
|
|
||||||
|
# Generate solution
|
||||||
|
output = llm(
|
||||||
|
prompt,
|
||||||
|
max_tokens=4096,
|
||||||
|
temperature=0.7,
|
||||||
|
top_p=0.8,
|
||||||
|
top_k=20,
|
||||||
|
repeat_penalty=1.1
|
||||||
|
)
|
||||||
|
|
||||||
|
print(output['choices'][0]['text'])
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using with Ollama
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create a Modelfile
|
||||||
|
cat > Modelfile << EOF
|
||||||
|
FROM ./qwen3-code-reasoning-4b.Q4_K_M.gguf
|
||||||
|
TEMPLATE """{{ if .System }}<|im_start|>system
|
||||||
|
{{ .System }}<|im_end|>
|
||||||
|
{{ end }}{{ if .Prompt }}<|im_start|>user
|
||||||
|
{{ .Prompt }}<|im_end|>
|
||||||
|
{{ end }}<|im_start|>assistant
|
||||||
|
"""
|
||||||
|
PARAMETER temperature 0.7
|
||||||
|
PARAMETER top_p 0.8
|
||||||
|
PARAMETER top_k 20
|
||||||
|
PARAMETER repeat_penalty 1.1
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Create and run the model
|
||||||
|
ollama create qwen3-code-reasoning -f Modelfile
|
||||||
|
ollama run qwen3-code-reasoning "Solve this competitive programming problem: [your problem here]"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📊 Available Quantizations
|
||||||
|
|
||||||
|
| Quantization | Size | Memory Usage | Quality | Use Case |
|
||||||
|
|--------------|------|--------------|---------|----------|
|
||||||
|
| Q3_K_M | 2.08 GB | ~3 GB | Good | CPU inference, limited memory |
|
||||||
|
| Q4_K_M | 2.5 GB | ~4 GB | Better | Balanced performance/memory |
|
||||||
|
| Q5_K_M | 2.89 GB | ~5 GB | Very Good | High quality, moderate memory |
|
||||||
|
| Q6_K | 3.31 GB | ~6 GB | Excellent | High quality, more memory |
|
||||||
|
| Q8_0 | 4.28 GB | ~8 GB | Best | Maximum quality, high memory |
|
||||||
|
| F16 | 8.05 GB | ~16 GB | Original | Maximum quality, GPU recommended |
|
||||||
|
|
||||||
|
## 📈 Performance Expectations
|
||||||
|
|
||||||
|
This GGUF quantized model maintains the performance characteristics of the original finetuned model:
|
||||||
|
|
||||||
|
* **Competitive Programming Problems**: Better understanding of problem constraints and requirements
|
||||||
|
* **Code Generation**: More accurate and efficient solutions
|
||||||
|
* **Reasoning Quality**: Enhanced step-by-step reasoning for complex problems
|
||||||
|
* **Solution Completeness**: More comprehensive solutions with proper edge case handling
|
||||||
|
|
||||||
|
## 🎛️ Recommended Settings
|
||||||
|
|
||||||
|
### For Code Generation
|
||||||
|
|
||||||
|
* **Temperature**: 0.7
|
||||||
|
* **Top-p**: 0.8
|
||||||
|
* **Top-k**: 20
|
||||||
|
* **Max New Tokens**: 4096 (adjust based on problem complexity)
|
||||||
|
* **Repeat Penalty**: 1.1
|
||||||
|
|
||||||
|
### For Reasoning Tasks
|
||||||
|
|
||||||
|
* **Temperature**: 0.6
|
||||||
|
* **Top-p**: 0.95
|
||||||
|
* **Top-k**: 20
|
||||||
|
* **Max New Tokens**: 8192 (for complex reasoning)
|
||||||
|
* **Repeat Penalty**: 1.1
|
||||||
|
|
||||||
|
## 🛠️ Hardware Requirements
|
||||||
|
|
||||||
|
### Minimum Requirements
|
||||||
|
* **RAM**: 4 GB (for Q3_K_M quantization)
|
||||||
|
* **Storage**: 2.5 GB free space
|
||||||
|
* **CPU**: Multi-core processor recommended
|
||||||
|
|
||||||
|
### Recommended Requirements
|
||||||
|
* **RAM**: 8 GB or more
|
||||||
|
* **Storage**: 5 GB free space
|
||||||
|
* **GPU**: NVIDIA GPU with 4GB+ VRAM (optional, for faster inference)
|
||||||
|
|
||||||
|
## 🤝 Contributing
|
||||||
|
|
||||||
|
This GGUF model was converted from the original LoRA-finetuned model. For questions about:
|
||||||
|
|
||||||
|
* The original model: [GetSoloTech/Qwen3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Qwen3-Code-Reasoning-4B)
|
||||||
|
* The base model: [Qwen3 GitHub](https://github.com/QwenLM/Qwen3)
|
||||||
|
* The training dataset: [Code-Reasoning Repository](https://huggingface.co/datasets/GetSoloTech/Code-Reasoning)
|
||||||
|
* The training framework: [Unsloth Documentation](https://github.com/unslothai/unsloth)
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
This model follows the same license as the base model (Apache 2.0). Please refer to the base model license for details.
|
||||||
|
|
||||||
|
## 🙏 Acknowledgments
|
||||||
|
|
||||||
|
* **Qwen Team** for the excellent base model
|
||||||
|
* **Unsloth Team** for the efficient training framework
|
||||||
|
* **NVIDIA Research** for the original OpenCodeReasoning-2 dataset
|
||||||
|
* **llama.cpp community** for the GGUF format and tools
|
||||||
|
|
||||||
|
## 📞 Contact
|
||||||
|
|
||||||
|
For questions about this GGUF model, please open an issue in the repository.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Note**: This model is specifically optimized for competitive programming and code reasoning tasks. The GGUF format enables efficient inference on various hardware configurations while maintaining the model's reasoning capabilities.
|
||||||
Reference in New Issue
Block a user