初始化项目，由ModelHub XC社区提供模型

Model: osmosis-ai/osmosis-mcp-4b
Source: Original Platform

README_QUANTIZATION.md

@@ -0,0 +1,63 @@
+# Model Quantization with llama.cpp
+
+This README explains how to use the `quantize_models.sh` script to create quantized versions of your GGUF models.
+
+## Prerequisites
+
+- llama.cpp must be cloned and built in this directory
+- You need a base GGUF model (default is `osmosis-mcp-4B-BF16.gguf`)
+
+## How to Use
+
+1. Make sure your base model is in the current directory
+2. Run the script:
+
+```bash
+./quantize_models.sh
+```
+
+## Supported Quantization Formats
+
+The script will create the following quantized versions:
+
+| Format   | Description                                                     | Approximate Size   |
+|----------|-----------------------------------------------------------------|-------------------|
+| Q4_K_S   | 4-bit quantization, smaller size                                | ~29% of original  |
+| Q5_K_M   | 5-bit quantization, medium size                                 | ~34% of original  |
+| Q5_K_S   | 5-bit quantization, smaller size                                | ~33% of original  |
+| Q6_K     | 6-bit quantization, balanced quality and size                   | ~38% of original  |
+| IQ4_XS   | Improved 4-bit non-linear quantization, extra small             | ~27% of original  |
+| Q8_0     | 8-bit quantization, highest quality                             | ~50% of original  |
+| Q2_K     | 2-bit quantization, extremely small                             | ~18% of original  |
+| Q3_K_L   | 3-bit quantization, larger size                                 | ~25% of original  |
+| Q3_K_M   | 3-bit quantization, medium size                                 | ~23% of original  |
+| Q3_K_S   | 3-bit quantization, smaller size                                | ~21% of original  |
+| Q4_K_M   | 4-bit quantization, medium size                                 | ~28% of original  |
+
+## Customizing the Script
+
+If you want to quantize a different base model, edit the `INPUT_MODEL` variable in the script:
+
+```bash
+# Input model file
+INPUT_MODEL="your-model-file.gguf"
+```
+
+## Time and Space Requirements
+
+- Quantization can take from several minutes to an hour depending on your hardware
+- Make sure you have enough free disk space for all the output models
+- The total disk space required will be approximately 3x the size of the original model
+
+## Using the Quantized Models
+
+Each quantized model can be used with llama.cpp tools:
+
+```bash
+llama.cpp/build/bin/llama-cli -m osmosis-mcp-4b.Q4_K_S.gguf -p "Your prompt here"
+```
+
+Choose the quantization format based on your needs:
+- Smaller quantization (Q2_K, Q3_K_S) for limited hardware resources
+- Medium quantization (Q4_K_M, Q5_K_S) for balanced performance
+- Larger quantization (Q6_K, Q8_0) for highest quality with sufficient hardware