初始化项目,由ModelHub XC社区提供模型
Model: GetSoloTech/Qwen3-Code-Reasoning-4B-GGUF Source: Original Platform
This commit is contained in:
41
.gitattributes
vendored
Normal file
41
.gitattributes
vendored
Normal file
@@ -0,0 +1,41 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-Code-Reasoning-4B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-Code-Reasoning-4B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-Code-Reasoning-4B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-Code-Reasoning-4B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-Code-Reasoning-4B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-Code-Reasoning-4B.f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Qwen3-Code-Reasoning-4B.Q3_K_M.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e7e26754634c8e8a0cd00c53dd60bf675f8c710b957405f47e1914f83ed207c3
|
||||
size 2075618656
|
||||
3
Qwen3-Code-Reasoning-4B.Q4_K_M.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:83a25bb48598e4b6c9f0e7dd5892e003a1c52a0ae644df8c9e0008c9dda62d8c
|
||||
size 2497281376
|
||||
3
Qwen3-Code-Reasoning-4B.Q5_K_M.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e77c9bad036097853cf7a81471e718e31a3261e6faff9de48fc3b39bb6cfb645
|
||||
size 2889514336
|
||||
3
Qwen3-Code-Reasoning-4B.Q6_K.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2b5d5661f3370fdb82b55a70aa04b437b75e679acc5cf1a0cff5d1a9903b09db
|
||||
size 3306261856
|
||||
3
Qwen3-Code-Reasoning-4B.Q8_0.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ca068d56c17ff84d7d2ebb6000378894b4647f3d4461ebeff7826674047c6ba4
|
||||
size 4280405856
|
||||
3
Qwen3-Code-Reasoning-4B.f16.gguf
Normal file
3
Qwen3-Code-Reasoning-4B.f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:224c20fd3ca6891b463618e086c1ee9201a5a3223ea6bd4df2381cc8acf47652
|
||||
size 8051285856
|
||||
180
README.md
Normal file
180
README.md
Normal file
@@ -0,0 +1,180 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- GetSoloTech/Code-Reasoning
|
||||
language:
|
||||
- en
|
||||
base_model:
|
||||
- GetSoloTech/Qwen3-Code-Reasoning-4B
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- coding
|
||||
- reasoning
|
||||
- problem-solving
|
||||
- algorithms
|
||||
- python
|
||||
- c++
|
||||
---
|
||||
|
||||
# GetSoloTech/Qwen3-Code-Reasoning-4B-GGUF
|
||||
|
||||
This is the GGUF quantized version of the [Qwen3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Qwen3-Code-Reasoning-4B) model, specifically optimized for competitive programming and code reasoning tasks. This model has been trained on the high-quality Code-Reasoning dataset to enhance its capabilities in solving complex programming problems with detailed reasoning.
|
||||
|
||||
|
||||
## 🚀 Key Features
|
||||
|
||||
* **Enhanced Code Reasoning**: Specifically trained on competitive programming problems
|
||||
* **Thinking Capabilities**: Inherits the advanced reasoning capabilities from the base model
|
||||
* **High-Quality Solutions**: Trained on solutions with ≥85% test case pass rates
|
||||
* **Structured Output**: Optimized for generating well-reasoned programming solutions
|
||||
* **Efficient Inference**: GGUF format enables fast inference on CPU and GPU
|
||||
* **Multiple Quantization Levels**: Available in various precision levels for different hardware requirements
|
||||
|
||||
### Dataset Statistics
|
||||
|
||||
* **Split**: Python
|
||||
* **Source**: High-quality competitive programming problems from TACO, APPS, CodeContests, and Codeforces
|
||||
* **Quality Filter**: Only correctly solved problems with ≥85% test case pass rates
|
||||
|
||||
## 🔧 Usage
|
||||
|
||||
### Using with llama.cpp
|
||||
|
||||
```bash
|
||||
# Download the model (choose your preferred quantization)
|
||||
wget https://huggingface.co/GetSoloTech/Qwen3-Code-Reasoning-4B-GGUF/resolve/main/qwen3-code-reasoning-4b.Q4_K_M.gguf
|
||||
|
||||
# Run inference
|
||||
./llama.cpp -m qwen3-code-reasoning-4b.Q4_K_M.gguf -n 4096 --repeat_penalty 1.1 -p "You are an expert competitive programmer. Read the problem and produce a correct, efficient solution. Include reasoning if helpful.\n\nProblem: Your programming problem here..."
|
||||
```
|
||||
|
||||
### Using with Python (llama-cpp-python)
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
# Load the model
|
||||
llm = Llama(
|
||||
model_path="./qwen3-code-reasoning-4b.Q4_K_M.gguf",
|
||||
n_ctx=4096,
|
||||
n_threads=4
|
||||
)
|
||||
|
||||
# Prepare input for competitive programming problem
|
||||
prompt = """You are an expert competitive programmer. Read the problem and produce a correct, efficient solution. Include reasoning if helpful.
|
||||
|
||||
Problem: Your programming problem here..."""
|
||||
|
||||
# Generate solution
|
||||
output = llm(
|
||||
prompt,
|
||||
max_tokens=4096,
|
||||
temperature=0.7,
|
||||
top_p=0.8,
|
||||
top_k=20,
|
||||
repeat_penalty=1.1
|
||||
)
|
||||
|
||||
print(output['choices'][0]['text'])
|
||||
```
|
||||
|
||||
### Using with Ollama
|
||||
|
||||
```bash
|
||||
# Create a Modelfile
|
||||
cat > Modelfile << EOF
|
||||
FROM ./qwen3-code-reasoning-4b.Q4_K_M.gguf
|
||||
TEMPLATE """{{ if .System }}<|im_start|>system
|
||||
{{ .System }}<|im_end|>
|
||||
{{ end }}{{ if .Prompt }}<|im_start|>user
|
||||
{{ .Prompt }}<|im_end|>
|
||||
{{ end }}<|im_start|>assistant
|
||||
"""
|
||||
PARAMETER temperature 0.7
|
||||
PARAMETER top_p 0.8
|
||||
PARAMETER top_k 20
|
||||
PARAMETER repeat_penalty 1.1
|
||||
EOF
|
||||
|
||||
# Create and run the model
|
||||
ollama create qwen3-code-reasoning -f Modelfile
|
||||
ollama run qwen3-code-reasoning "Solve this competitive programming problem: [your problem here]"
|
||||
```
|
||||
|
||||
## 📊 Available Quantizations
|
||||
|
||||
| Quantization | Size | Memory Usage | Quality | Use Case |
|
||||
|--------------|------|--------------|---------|----------|
|
||||
| Q3_K_M | 2.08 GB | ~3 GB | Good | CPU inference, limited memory |
|
||||
| Q4_K_M | 2.5 GB | ~4 GB | Better | Balanced performance/memory |
|
||||
| Q5_K_M | 2.89 GB | ~5 GB | Very Good | High quality, moderate memory |
|
||||
| Q6_K | 3.31 GB | ~6 GB | Excellent | High quality, more memory |
|
||||
| Q8_0 | 4.28 GB | ~8 GB | Best | Maximum quality, high memory |
|
||||
| F16 | 8.05 GB | ~16 GB | Original | Maximum quality, GPU recommended |
|
||||
|
||||
## 📈 Performance Expectations
|
||||
|
||||
This GGUF quantized model maintains the performance characteristics of the original finetuned model:
|
||||
|
||||
* **Competitive Programming Problems**: Better understanding of problem constraints and requirements
|
||||
* **Code Generation**: More accurate and efficient solutions
|
||||
* **Reasoning Quality**: Enhanced step-by-step reasoning for complex problems
|
||||
* **Solution Completeness**: More comprehensive solutions with proper edge case handling
|
||||
|
||||
## 🎛️ Recommended Settings
|
||||
|
||||
### For Code Generation
|
||||
|
||||
* **Temperature**: 0.7
|
||||
* **Top-p**: 0.8
|
||||
* **Top-k**: 20
|
||||
* **Max New Tokens**: 4096 (adjust based on problem complexity)
|
||||
* **Repeat Penalty**: 1.1
|
||||
|
||||
### For Reasoning Tasks
|
||||
|
||||
* **Temperature**: 0.6
|
||||
* **Top-p**: 0.95
|
||||
* **Top-k**: 20
|
||||
* **Max New Tokens**: 8192 (for complex reasoning)
|
||||
* **Repeat Penalty**: 1.1
|
||||
|
||||
## 🛠️ Hardware Requirements
|
||||
|
||||
### Minimum Requirements
|
||||
* **RAM**: 4 GB (for Q3_K_M quantization)
|
||||
* **Storage**: 2.5 GB free space
|
||||
* **CPU**: Multi-core processor recommended
|
||||
|
||||
### Recommended Requirements
|
||||
* **RAM**: 8 GB or more
|
||||
* **Storage**: 5 GB free space
|
||||
* **GPU**: NVIDIA GPU with 4GB+ VRAM (optional, for faster inference)
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
This GGUF model was converted from the original LoRA-finetuned model. For questions about:
|
||||
|
||||
* The original model: [GetSoloTech/Qwen3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Qwen3-Code-Reasoning-4B)
|
||||
* The base model: [Qwen3 GitHub](https://github.com/QwenLM/Qwen3)
|
||||
* The training dataset: [Code-Reasoning Repository](https://huggingface.co/datasets/GetSoloTech/Code-Reasoning)
|
||||
* The training framework: [Unsloth Documentation](https://github.com/unslothai/unsloth)
|
||||
|
||||
## 📄 License
|
||||
|
||||
This model follows the same license as the base model (Apache 2.0). Please refer to the base model license for details.
|
||||
|
||||
## 🙏 Acknowledgments
|
||||
|
||||
* **Qwen Team** for the excellent base model
|
||||
* **Unsloth Team** for the efficient training framework
|
||||
* **NVIDIA Research** for the original OpenCodeReasoning-2 dataset
|
||||
* **llama.cpp community** for the GGUF format and tools
|
||||
|
||||
## 📞 Contact
|
||||
|
||||
For questions about this GGUF model, please open an issue in the repository.
|
||||
|
||||
---
|
||||
|
||||
**Note**: This model is specifically optimized for competitive programming and code reasoning tasks. The GGUF format enables efficient inference on various hardware configurations while maintaining the model's reasoning capabilities.
|
||||
Reference in New Issue
Block a user