初始化项目,由ModelHub XC社区提供模型
Model: GetSoloTech/Gemma3-Code-Reasoning-4B-GGUF Source: Original Platform
This commit is contained in:
43
.gitattributes
vendored
Normal file
43
.gitattributes
vendored
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Gemma3-Code-Reasoning-4B.f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Gemma3-Code-Reasoning-4B.IQ4_XS.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:d79fe13a4f498716a2cf34b146e80372aea67adc9f7be5f9d6e4375d0f0b05c7
|
||||||
|
size 2279626656
|
||||||
3
Gemma3-Code-Reasoning-4B.Q2_K.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:577bc3b69ac890815c75c77965c4f1149ca9122e5afe1e7f922e112790fc2b2d
|
||||||
|
size 1729165216
|
||||||
3
Gemma3-Code-Reasoning-4B.Q3_K_M.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:26c7b82234134c4ef26bb181b3d9334204af4fbc298684c947c83019afd991c1
|
||||||
|
size 2098460576
|
||||||
3
Gemma3-Code-Reasoning-4B.Q4_K_M.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:664e342f1f2d5b46d35d46addf30e37813b217b76c6d94745b3d2a70258564a6
|
||||||
|
size 2489894816
|
||||||
3
Gemma3-Code-Reasoning-4B.Q5_K_M.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b1957c31d01e0d6f1619589a617f4ccf873c1c4f9a65c1945dec3b0934e20533
|
||||||
|
size 2829698976
|
||||||
3
Gemma3-Code-Reasoning-4B.Q6_K.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a4dc53effd0b1dfe84cd4f17183d62cc67bd8706158203b89ac606f71225bf51
|
||||||
|
size 3190740896
|
||||||
3
Gemma3-Code-Reasoning-4B.Q8_0.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e0fa9477d9ceefc43671dfe4656dbb01c7784708384d81cf8c6a1c2e742588f0
|
||||||
|
size 4130402976
|
||||||
3
Gemma3-Code-Reasoning-4B.f16.gguf
Normal file
3
Gemma3-Code-Reasoning-4B.f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f781c7c18eb52ca6a796a2e203af4d4895a784f10de621855aeebdc195534739
|
||||||
|
size 7767804576
|
||||||
152
README.md
Normal file
152
README.md
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
---
|
||||||
|
datasets:
|
||||||
|
- GetSoloTech/Code-Reasoning
|
||||||
|
base_model:
|
||||||
|
- GetSoloTech/Gemma3-Code-Reasoning-4B
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
tags:
|
||||||
|
- coding
|
||||||
|
- reasoning
|
||||||
|
- problem-solving
|
||||||
|
- algorithms
|
||||||
|
- python
|
||||||
|
- c++
|
||||||
|
- code-reasoning
|
||||||
|
- competitive-programming
|
||||||
|
---
|
||||||
|
|
||||||
|
# Gemma3-Code-Reasoning-4B-GGUF
|
||||||
|
|
||||||
|
This repository contains GGUF (GGML Universal Format) quantized versions of the [GetSoloTech/Gemma3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Gemma3-Code-Reasoning-4B) model, optimized for local inference with various quantization levels to balance performance and resource usage.
|
||||||
|
|
||||||
|
## 🎯 Model Overview
|
||||||
|
|
||||||
|
This is a **LoRA-finetuned** version of `gemma-3-4b-it` specifically optimized for competitive programming and code reasoning tasks. The model has been trained on the high-quality Code-Reasoning dataset to enhance its capabilities in solving complex programming problems with detailed reasoning.
|
||||||
|
|
||||||
|
## 🚀 Key Features
|
||||||
|
|
||||||
|
- **Enhanced Code Reasoning**: Specifically trained on competitive programming problems
|
||||||
|
- **Thinking Capabilities**: Inherits the advanced reasoning capabilities from the base model
|
||||||
|
- **High-Quality Solutions**: Trained on solutions with ≥85% test case pass rates
|
||||||
|
- **Structured Output**: Optimized for generating well-reasoned programming solutions
|
||||||
|
- **Efficient Training**: Uses LoRA adapters for efficient parameter updates
|
||||||
|
- **Multiple Quantization Levels**: Available in various GGUF formats for different hardware capabilities
|
||||||
|
|
||||||
|
## 📁 Available GGUF Models
|
||||||
|
| Model File | Size | Quantization | Use Case |
|
||||||
|
|------------|------|--------------|----------|
|
||||||
|
| `Gemma3-Code-Reasoning-4B.f16.gguf` | 7.77 GB | FP16 | Highest quality, requires more VRAM |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.Q8_0.gguf` | 4.13 GB | Q8_0 | High quality, good balance |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.Q6_K.gguf` | 3.19 GB | Q6_K | Good quality, moderate VRAM usage |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.Q5_K_M.gguf` | 2.83 GB | Q5_K_M | Balanced quality and size |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.Q4_K_M.gguf` | 2.49 GB | Q4_K_M | Good compression, reasonable quality |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.Q3_K_M.gguf` | 2.1 GB | Q3_K_M | Smaller size, moderate quality |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.Q2_K.gguf` | 1.73 GB | Q2_K | Smallest size, basic quality |
|
||||||
|
| `Gemma3-Code-Reasoning-4B.IQ4_XS.gguf` | 2.28 GB | IQ4_XS | Intel optimized, good quality |
|
||||||
|
|
||||||
|
## 🔧 Usage
|
||||||
|
|
||||||
|
### Using with llama.cpp
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Download a GGUF model file
|
||||||
|
wget https://huggingface.co/GetSoloTech/Gemma3-Code-Reasoning-4B-GGUF/resolve/main/Gemma3-Code-Reasoning-4B.Q4_K_M.gguf
|
||||||
|
|
||||||
|
# Run inference with llama.cpp
|
||||||
|
./llama.cpp/main -m Gemma3-Code-Reasoning-4B.Q4_K_M.gguf -n 4096 --repeat_penalty 1.1 -p "You are an expert competitive programmer. Solve this problem: [YOUR_PROBLEM_HERE]"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using with Python (llama-cpp-python)
|
||||||
|
|
||||||
|
```python
|
||||||
|
from llama_cpp import Llama
|
||||||
|
|
||||||
|
# Load the model
|
||||||
|
llm = Llama(
|
||||||
|
model_path="./Gemma3-Code-Reasoning-4B.Q4_K_M.gguf",
|
||||||
|
n_ctx=4096,
|
||||||
|
n_threads=4
|
||||||
|
)
|
||||||
|
|
||||||
|
# Prepare the prompt
|
||||||
|
prompt = """You are an expert competitive programmer. Read the problem and produce a correct, efficient solution. Include reasoning if helpful.
|
||||||
|
|
||||||
|
Problem: [YOUR_PROGRAMMING_PROBLEM_HERE]
|
||||||
|
|
||||||
|
Solution:"""
|
||||||
|
|
||||||
|
# Generate response
|
||||||
|
output = llm(
|
||||||
|
prompt,
|
||||||
|
max_tokens=4096,
|
||||||
|
temperature=1.0,
|
||||||
|
top_p=0.95,
|
||||||
|
top_k=64,
|
||||||
|
repeat_penalty=1.1
|
||||||
|
)
|
||||||
|
|
||||||
|
print(output['choices'][0]['text'])
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## 🎛️ Recommended Settings
|
||||||
|
|
||||||
|
- **Temperature**: 1.0
|
||||||
|
- **Top-p**: 0.95
|
||||||
|
- **Top-k**: 64
|
||||||
|
- **Max New Tokens**: 4096 (adjust based on problem complexity)
|
||||||
|
- **Repeat Penalty**: 1.1
|
||||||
|
|
||||||
|
|
||||||
|
## 💻 Hardware Requirements
|
||||||
|
|
||||||
|
| Quantization | Minimum VRAM | Recommended VRAM | CPU RAM |
|
||||||
|
|--------------|--------------|------------------|---------|
|
||||||
|
| FP16 | 8 GB | 12 GB | 16 GB |
|
||||||
|
| Q8_0 | 5 GB | 8 GB | 12 GB |
|
||||||
|
| Q6_K | 4 GB | 6 GB | 10 GB |
|
||||||
|
| Q5_K_M | 3 GB | 5 GB | 8 GB |
|
||||||
|
| Q4_K_M | 3 GB | 4 GB | 6 GB |
|
||||||
|
| Q3_K_M | 2 GB | 3 GB | 4 GB |
|
||||||
|
| Q2_K | 2 GB | 2 GB | 3 GB |
|
||||||
|
| IQ4_XS | 3 GB | 4 GB | 6 GB |
|
||||||
|
|
||||||
|
## 📈 Performance Expectations
|
||||||
|
|
||||||
|
This finetuned model is expected to show improved performance on:
|
||||||
|
|
||||||
|
- **Competitive Programming Problems**: Better understanding of problem constraints and requirements
|
||||||
|
- **Code Generation**: More accurate and efficient solutions
|
||||||
|
- **Reasoning Quality**: Enhanced step-by-step reasoning for complex problems
|
||||||
|
- **Solution Completeness**: More comprehensive solutions with proper edge case handling
|
||||||
|
|
||||||
|
## 🔗 Related Resources
|
||||||
|
|
||||||
|
- **Base Model**: [GetSoloTech/Gemma3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Gemma3-Code-Reasoning-4B)
|
||||||
|
- **Training Dataset**: [GetSoloTech/Code-Reasoning](https://huggingface.co/datasets/GetSoloTech/Code-Reasoning)
|
||||||
|
- **Original Gemma Model**: [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it)
|
||||||
|
- **llama.cpp**: [GitHub Repository](https://github.com/ggerganov/llama.cpp)
|
||||||
|
- **llama-cpp-python**: [PyPI Package](https://pypi.org/project/llama-cpp-python/)
|
||||||
|
|
||||||
|
## 🤝 Contributing
|
||||||
|
|
||||||
|
This model was created using the Unsloth framework and the Code-Reasoning dataset. For questions about:
|
||||||
|
|
||||||
|
- The base model: [Gemma3 Huggingface](https://huggingface.co/google/gemma-3-4b-it)
|
||||||
|
- The training dataset: [Code-Reasoning Repository](https://huggingface.co/datasets/GetSoloTech/Code-Reasoning)
|
||||||
|
- The training framework: [Unsloth Documentation](https://github.com/unslothai/unsloth)
|
||||||
|
|
||||||
|
## 🙏 Acknowledgments
|
||||||
|
|
||||||
|
- **Gemma Team** for the excellent base model
|
||||||
|
- **Unsloth Team** for the efficient training framework
|
||||||
|
- **NVIDIA Research** for the original OpenCodeReasoning-2 dataset
|
||||||
|
- **llama.cpp community** for the GGUF format and tools
|
||||||
|
|
||||||
|
## 📞 Contact
|
||||||
|
|
||||||
|
For questions about this GGUF converted model, please open an issue in the repository.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Note**: This model is specifically optimized for competitive programming and code reasoning tasks. Choose the appropriate quantization level based on your hardware capabilities and quality requirements.
|
||||||
Reference in New Issue
Block a user