commit d733883796c3ee26f3061f772ab2861e5a1b416b Author: ModelHub XC Date: Mon Jul 20 23:41:11 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: GetSoloTech/Gemma3-Code-Reasoning-4B-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..9924577 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,43 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Gemma3-Code-Reasoning-4B.f16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Gemma3-Code-Reasoning-4B.IQ4_XS.gguf b/Gemma3-Code-Reasoning-4B.IQ4_XS.gguf new file mode 100644 index 0000000..fd6d96b --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d79fe13a4f498716a2cf34b146e80372aea67adc9f7be5f9d6e4375d0f0b05c7 +size 2279626656 diff --git a/Gemma3-Code-Reasoning-4B.Q2_K.gguf b/Gemma3-Code-Reasoning-4B.Q2_K.gguf new file mode 100644 index 0000000..bd5dfce --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:577bc3b69ac890815c75c77965c4f1149ca9122e5afe1e7f922e112790fc2b2d +size 1729165216 diff --git a/Gemma3-Code-Reasoning-4B.Q3_K_M.gguf b/Gemma3-Code-Reasoning-4B.Q3_K_M.gguf new file mode 100644 index 0000000..7eb845e --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:26c7b82234134c4ef26bb181b3d9334204af4fbc298684c947c83019afd991c1 +size 2098460576 diff --git a/Gemma3-Code-Reasoning-4B.Q4_K_M.gguf b/Gemma3-Code-Reasoning-4B.Q4_K_M.gguf new file mode 100644 index 0000000..cada511 --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:664e342f1f2d5b46d35d46addf30e37813b217b76c6d94745b3d2a70258564a6 +size 2489894816 diff --git a/Gemma3-Code-Reasoning-4B.Q5_K_M.gguf b/Gemma3-Code-Reasoning-4B.Q5_K_M.gguf new file mode 100644 index 0000000..1647061 --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1957c31d01e0d6f1619589a617f4ccf873c1c4f9a65c1945dec3b0934e20533 +size 2829698976 diff --git a/Gemma3-Code-Reasoning-4B.Q6_K.gguf b/Gemma3-Code-Reasoning-4B.Q6_K.gguf new file mode 100644 index 0000000..1e04f6d --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4dc53effd0b1dfe84cd4f17183d62cc67bd8706158203b89ac606f71225bf51 +size 3190740896 diff --git a/Gemma3-Code-Reasoning-4B.Q8_0.gguf b/Gemma3-Code-Reasoning-4B.Q8_0.gguf new file mode 100644 index 0000000..6998f90 --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0fa9477d9ceefc43671dfe4656dbb01c7784708384d81cf8c6a1c2e742588f0 +size 4130402976 diff --git a/Gemma3-Code-Reasoning-4B.f16.gguf b/Gemma3-Code-Reasoning-4B.f16.gguf new file mode 100644 index 0000000..ff749df --- /dev/null +++ b/Gemma3-Code-Reasoning-4B.f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f781c7c18eb52ca6a796a2e203af4d4895a784f10de621855aeebdc195534739 +size 7767804576 diff --git a/README.md b/README.md new file mode 100644 index 0000000..57e5e1e --- /dev/null +++ b/README.md @@ -0,0 +1,152 @@ +--- +datasets: +- GetSoloTech/Code-Reasoning +base_model: +- GetSoloTech/Gemma3-Code-Reasoning-4B +pipeline_tag: text-generation +tags: +- coding +- reasoning +- problem-solving +- algorithms +- python +- c++ +- code-reasoning +- competitive-programming +--- + +# Gemma3-Code-Reasoning-4B-GGUF + +This repository contains GGUF (GGML Universal Format) quantized versions of the [GetSoloTech/Gemma3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Gemma3-Code-Reasoning-4B) model, optimized for local inference with various quantization levels to balance performance and resource usage. + +## 🎯 Model Overview + +This is a **LoRA-finetuned** version of `gemma-3-4b-it` specifically optimized for competitive programming and code reasoning tasks. The model has been trained on the high-quality Code-Reasoning dataset to enhance its capabilities in solving complex programming problems with detailed reasoning. + +## 🚀 Key Features + +- **Enhanced Code Reasoning**: Specifically trained on competitive programming problems +- **Thinking Capabilities**: Inherits the advanced reasoning capabilities from the base model +- **High-Quality Solutions**: Trained on solutions with ≥85% test case pass rates +- **Structured Output**: Optimized for generating well-reasoned programming solutions +- **Efficient Training**: Uses LoRA adapters for efficient parameter updates +- **Multiple Quantization Levels**: Available in various GGUF formats for different hardware capabilities + +## 📁 Available GGUF Models +| Model File | Size | Quantization | Use Case | +|------------|------|--------------|----------| +| `Gemma3-Code-Reasoning-4B.f16.gguf` | 7.77 GB | FP16 | Highest quality, requires more VRAM | +| `Gemma3-Code-Reasoning-4B.Q8_0.gguf` | 4.13 GB | Q8_0 | High quality, good balance | +| `Gemma3-Code-Reasoning-4B.Q6_K.gguf` | 3.19 GB | Q6_K | Good quality, moderate VRAM usage | +| `Gemma3-Code-Reasoning-4B.Q5_K_M.gguf` | 2.83 GB | Q5_K_M | Balanced quality and size | +| `Gemma3-Code-Reasoning-4B.Q4_K_M.gguf` | 2.49 GB | Q4_K_M | Good compression, reasonable quality | +| `Gemma3-Code-Reasoning-4B.Q3_K_M.gguf` | 2.1 GB | Q3_K_M | Smaller size, moderate quality | +| `Gemma3-Code-Reasoning-4B.Q2_K.gguf` | 1.73 GB | Q2_K | Smallest size, basic quality | +| `Gemma3-Code-Reasoning-4B.IQ4_XS.gguf` | 2.28 GB | IQ4_XS | Intel optimized, good quality | + +## 🔧 Usage + +### Using with llama.cpp + +```bash +# Download a GGUF model file +wget https://huggingface.co/GetSoloTech/Gemma3-Code-Reasoning-4B-GGUF/resolve/main/Gemma3-Code-Reasoning-4B.Q4_K_M.gguf + +# Run inference with llama.cpp +./llama.cpp/main -m Gemma3-Code-Reasoning-4B.Q4_K_M.gguf -n 4096 --repeat_penalty 1.1 -p "You are an expert competitive programmer. Solve this problem: [YOUR_PROBLEM_HERE]" +``` + +### Using with Python (llama-cpp-python) + +```python +from llama_cpp import Llama + +# Load the model +llm = Llama( + model_path="./Gemma3-Code-Reasoning-4B.Q4_K_M.gguf", + n_ctx=4096, + n_threads=4 +) + +# Prepare the prompt +prompt = """You are an expert competitive programmer. Read the problem and produce a correct, efficient solution. Include reasoning if helpful. + +Problem: [YOUR_PROGRAMMING_PROBLEM_HERE] + +Solution:""" + +# Generate response +output = llm( + prompt, + max_tokens=4096, + temperature=1.0, + top_p=0.95, + top_k=64, + repeat_penalty=1.1 +) + +print(output['choices'][0]['text']) +``` + + +## 🎛️ Recommended Settings + +- **Temperature**: 1.0 +- **Top-p**: 0.95 +- **Top-k**: 64 +- **Max New Tokens**: 4096 (adjust based on problem complexity) +- **Repeat Penalty**: 1.1 + + +## 💻 Hardware Requirements + +| Quantization | Minimum VRAM | Recommended VRAM | CPU RAM | +|--------------|--------------|------------------|---------| +| FP16 | 8 GB | 12 GB | 16 GB | +| Q8_0 | 5 GB | 8 GB | 12 GB | +| Q6_K | 4 GB | 6 GB | 10 GB | +| Q5_K_M | 3 GB | 5 GB | 8 GB | +| Q4_K_M | 3 GB | 4 GB | 6 GB | +| Q3_K_M | 2 GB | 3 GB | 4 GB | +| Q2_K | 2 GB | 2 GB | 3 GB | +| IQ4_XS | 3 GB | 4 GB | 6 GB | + +## 📈 Performance Expectations + +This finetuned model is expected to show improved performance on: + +- **Competitive Programming Problems**: Better understanding of problem constraints and requirements +- **Code Generation**: More accurate and efficient solutions +- **Reasoning Quality**: Enhanced step-by-step reasoning for complex problems +- **Solution Completeness**: More comprehensive solutions with proper edge case handling + +## 🔗 Related Resources + +- **Base Model**: [GetSoloTech/Gemma3-Code-Reasoning-4B](https://huggingface.co/GetSoloTech/Gemma3-Code-Reasoning-4B) +- **Training Dataset**: [GetSoloTech/Code-Reasoning](https://huggingface.co/datasets/GetSoloTech/Code-Reasoning) +- **Original Gemma Model**: [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it) +- **llama.cpp**: [GitHub Repository](https://github.com/ggerganov/llama.cpp) +- **llama-cpp-python**: [PyPI Package](https://pypi.org/project/llama-cpp-python/) + +## 🤝 Contributing + +This model was created using the Unsloth framework and the Code-Reasoning dataset. For questions about: + +- The base model: [Gemma3 Huggingface](https://huggingface.co/google/gemma-3-4b-it) +- The training dataset: [Code-Reasoning Repository](https://huggingface.co/datasets/GetSoloTech/Code-Reasoning) +- The training framework: [Unsloth Documentation](https://github.com/unslothai/unsloth) + +## 🙏 Acknowledgments + +- **Gemma Team** for the excellent base model +- **Unsloth Team** for the efficient training framework +- **NVIDIA Research** for the original OpenCodeReasoning-2 dataset +- **llama.cpp community** for the GGUF format and tools + +## 📞 Contact + +For questions about this GGUF converted model, please open an issue in the repository. + +--- + +**Note**: This model is specifically optimized for competitive programming and code reasoning tasks. Choose the appropriate quantization level based on your hardware capabilities and quality requirements. \ No newline at end of file