初始化项目,由ModelHub XC社区提供模型
Model: Mungert/gemma-3-4b-it-gguf Source: Original Platform
This commit is contained in:
73
.gitattributes
vendored
Normal file
73
.gitattributes
vendored
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
|
||||||
|
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
|
||||||
|
google_gemma-3-4b-it-q3_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-bf16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-iq4_nl.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q4_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-iq3_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
llama-gemma3-cli filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q8.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gemma-3-4b-it-f16-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-bf16-q8.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-mmproj-f32.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-mmproj-bf16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gemma-3-4b-it-f16-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-mmproj-q8.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q5_k_s.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-f16-q8.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-mmproj-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gemma-3-4b-it-bf16-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q6_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
google_gemma-3-4b-it-iq4_xs.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
mmproj.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
268
README.md
Normal file
268
README.md
Normal file
@@ -0,0 +1,268 @@
|
|||||||
|
---
|
||||||
|
license: gemma
|
||||||
|
pipeline_tag: image-text-to-text
|
||||||
|
tags:
|
||||||
|
- vision
|
||||||
|
- gemma
|
||||||
|
- llama.cpp
|
||||||
|
---
|
||||||
|
|
||||||
|
# <span style="color: #7FFF7F;">Gemma-3 4B Instruct GGUF Models</span>
|
||||||
|
|
||||||
|
|
||||||
|
## How to Use Gemma 3 Vision with llama.cpp
|
||||||
|
|
||||||
|
To utilize the experimental support for Gemma 3 Vision in `llama.cpp`, follow these steps:
|
||||||
|
|
||||||
|
1. **Clone the lastest llama.cpp Repository**:
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ggml-org/llama.cpp.git
|
||||||
|
cd llama.cpp
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
2. **Build the Llama.cpp**:
|
||||||
|
|
||||||
|
Build llama.cpp as usual : https://github.com/ggml-org/llama.cpp#building-the-project
|
||||||
|
|
||||||
|
Once llama.cpp is built Copy the ./llama.cpp/build/bin/llama-gemma3-cli to a chosen folder.
|
||||||
|
|
||||||
|
3. **Download the Gemma 3 gguf file**:
|
||||||
|
|
||||||
|
https://huggingface.co/Mungert/gemma-3-4b-it-gguf/tree/main
|
||||||
|
|
||||||
|
Choose a gguf file without the mmproj in the name
|
||||||
|
|
||||||
|
Example gguf file : https://huggingface.co/Mungert/gemma-3-4b-it-gguf/resolve/main/google_gemma-3-4b-it-q4_k_l.gguf
|
||||||
|
|
||||||
|
Copy this file to your chosen folder.
|
||||||
|
|
||||||
|
4. **Download the Gemma 3 mmproj file**
|
||||||
|
|
||||||
|
https://huggingface.co/Mungert/gemma-3-4b-it-gguf/tree/main
|
||||||
|
|
||||||
|
Choose a file with mmproj in the name
|
||||||
|
|
||||||
|
Example mmproj file : https://huggingface.co/Mungert/gemma-3-4b-it-gguf/resolve/main/google_gemma-3-4b-it-mmproj-bf16.gguf
|
||||||
|
|
||||||
|
Copy this file to your chosen folder.
|
||||||
|
|
||||||
|
5. Copy images to the same folder as the gguf files or alter paths appropriately.
|
||||||
|
|
||||||
|
In the example below the gguf files, images and llama-gemma-cli are in the same folder.
|
||||||
|
|
||||||
|
Example image: image https://huggingface.co/Mungert/gemma-3-4b-it-gguf/resolve/main/car-1.jpg
|
||||||
|
|
||||||
|
Copy this file to your chosen folder.
|
||||||
|
|
||||||
|
6. **Run the CLI Tool**:
|
||||||
|
|
||||||
|
From your chosen folder :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
llama-gemma3-cli -m google_gemma-3-4b-it-q4_k_l.gguf --mmproj google_gemma-3-4b-it-mmproj-bf16.gguf
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
Running in chat mode, available commands:
|
||||||
|
/image <path> load an image
|
||||||
|
/clear clear the chat history
|
||||||
|
/quit or /exit exit the program
|
||||||
|
|
||||||
|
> /image car-1.jpg
|
||||||
|
Encoding image car-1.jpg
|
||||||
|
Image encoded in 46305 ms
|
||||||
|
Image decoded in 19302 ms
|
||||||
|
|
||||||
|
> what is the image of
|
||||||
|
Here's a breakdown of what's in the image:
|
||||||
|
|
||||||
|
**Subject:** The primary subject is a black Porsche Panamera Turbo driving on a highway.
|
||||||
|
|
||||||
|
**Details:**
|
||||||
|
|
||||||
|
* **Car:** It's a sleek, modern Porsche Panamera Turbo, identifiable by its distinctive rear design, the "PORSCHE" lettering, and the "Panamera Turbo" badge. The license plate reads "CVC-911".
|
||||||
|
* **Setting:** The car is on a multi-lane highway, with a blurred background of trees, a distant building, and a cloudy sky. The lighting suggests it's either dusk or dawn.
|
||||||
|
* **Motion:** The image captures the car in motion, with a slight motion blur to convey speed.
|
||||||
|
|
||||||
|
**Overall Impression:** The image conveys a sense of speed, luxury, and power. It's a well-composed shot that highlights the car's design and performance.
|
||||||
|
|
||||||
|
Do you want me to describe any specific aspect of the image in more detail, or perhaps analyze its composition?
|
||||||
|
```
|
||||||
|
|
||||||
|
# <span id="testllm" style="color: #7F7FFF;">🚀 If you find these models useful</span>
|
||||||
|
|
||||||
|
Please click like ❤️ . Also I’d really appreciate it if you could test my Network Monitor Assistant at 👉 [Network Monitor Assitant](https://readyforquantum.com).
|
||||||
|
💬 Click the **chat icon** (bottom right of the main and dashboard pages) . Choose a LLM; toggle between the LLM Types TurboLLM -> FreeLLM -> TestLLM.
|
||||||
|
|
||||||
|
### What I'm Testing
|
||||||
|
I'm experimenting with **function calling** against my network monitoring service. Using small open source models. I am into the question "How small can it go and still function".
|
||||||
|
🟡 **TestLLM** – Runs **Phi-4-mini-instruct** using phi-4-mini-q4_0.gguf , llama.cpp on 6 threads of a Cpu VM (Should take about 15s to load. Inference speed is quite slow and it only processes one user prompt at a time—still working on scaling!). If you're curious, I'd be happy to share how it works! .
|
||||||
|
|
||||||
|
### The other Available AI Assistants
|
||||||
|
🟢 **TurboLLM** – Uses **gpt-4o-mini** Fast! . Note: tokens are limited since OpenAI models are pricey, but you can [Login](https://readyforquantum.com) or [Download](https://readyforquantum.com/download/?utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface_repo_readme) the Quantum Network Monitor agent to get more tokens, Alternatively use the TestLLM .
|
||||||
|
🔵 **HugLLM** – Runs **open-source Hugging Face models** Fast, Runs small models (≈8B) hence lower quality, Get 2x more tokens (subject to Hugging Face API availability)
|
||||||
|
|
||||||
|
|
||||||
|
## **Choosing the Right Model Format**
|
||||||
|
|
||||||
|
Selecting the correct model format depends on your **hardware capabilities** and **memory constraints**.
|
||||||
|
|
||||||
|
### **BF16 (Brain Float 16) – Use if BF16 acceleration is available**
|
||||||
|
- A 16-bit floating-point format designed for **faster computation** while retaining good precision.
|
||||||
|
- Provides **similar dynamic range** as FP32 but with **lower memory usage**.
|
||||||
|
- Recommended if your hardware supports **BF16 acceleration** (check your device’s specs).
|
||||||
|
- Ideal for **high-performance inference** with **reduced memory footprint** compared to FP32.
|
||||||
|
|
||||||
|
📌 **Use BF16 if:**
|
||||||
|
✔ Your hardware has native **BF16 support** (e.g., newer GPUs, TPUs).
|
||||||
|
✔ You want **higher precision** while saving memory.
|
||||||
|
✔ You plan to **requantize** the model into another format.
|
||||||
|
|
||||||
|
📌 **Avoid BF16 if:**
|
||||||
|
❌ Your hardware does **not** support BF16 (it may fall back to FP32 and run slower).
|
||||||
|
❌ You need compatibility with older devices that lack BF16 optimization.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### **F16 (Float 16) – More widely supported than BF16**
|
||||||
|
- A 16-bit floating-point **high precision** but with less of range of values than BF16.
|
||||||
|
- Works on most devices with **FP16 acceleration support** (including many GPUs and some CPUs).
|
||||||
|
- Slightly lower numerical precision than BF16 but generally sufficient for inference.
|
||||||
|
|
||||||
|
📌 **Use F16 if:**
|
||||||
|
✔ Your hardware supports **FP16** but **not BF16**.
|
||||||
|
✔ You need a **balance between speed, memory usage, and accuracy**.
|
||||||
|
✔ You are running on a **GPU** or another device optimized for FP16 computations.
|
||||||
|
|
||||||
|
📌 **Avoid F16 if:**
|
||||||
|
❌ Your device lacks **native FP16 support** (it may run slower than expected).
|
||||||
|
❌ You have memory limtations.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### **Quantized Models (Q4_K, Q6_K, Q8, etc.) – For CPU & Low-VRAM Inference**
|
||||||
|
Quantization reduces model size and memory usage while maintaining as much accuracy as possible.
|
||||||
|
- **Lower-bit models (Q4_K)** → **Best for minimal memory usage**, may have lower precision.
|
||||||
|
- **Higher-bit models (Q6_K, Q8_0)** → **Better accuracy**, requires more memory.
|
||||||
|
|
||||||
|
📌 **Use Quantized Models if:**
|
||||||
|
✔ You are running inference on a **CPU** and need an optimized model.
|
||||||
|
✔ Your device has **low VRAM** and cannot load full-precision models.
|
||||||
|
✔ You want to reduce **memory footprint** while keeping reasonable accuracy.
|
||||||
|
|
||||||
|
📌 **Avoid Quantized Models if:**
|
||||||
|
❌ You need **maximum accuracy** (full-precision models are better for this).
|
||||||
|
❌ Your hardware has enough VRAM for higher-precision formats (BF16/F16).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### **Summary Table: Model Format Selection**
|
||||||
|
|
||||||
|
| Model Format | Precision | Memory Usage | Device Requirements | Best Use Case |
|
||||||
|
|--------------|------------|---------------|----------------------|---------------|
|
||||||
|
| **BF16** | Highest | High | BF16-supported GPU/CPUs | High-speed inference with reduced memory |
|
||||||
|
| **F16** | High | High | FP16-supported devices | GPU inference when BF16 isn’t available |
|
||||||
|
| **Q4_K** | Low | Very Low | CPU or Low-VRAM devices | Best for memory-constrained environments |
|
||||||
|
| **Q6_K** | Medium Low | Low | CPU with more memory | Better accuracy while still being quantized |
|
||||||
|
| **Q8** | Medium | Moderate | CPU or GPU with enough VRAM | Best accuracy among quantized models |
|
||||||
|
|
||||||
|
|
||||||
|
## **Included Files & Details**
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-bf16.gguf`
|
||||||
|
- Model weights preserved in **BF16**.
|
||||||
|
- Use this if you want to **requantize** the model into a different format.
|
||||||
|
- Best if your device supports **BF16 acceleration**.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-f16.gguf`
|
||||||
|
- Model weights stored in **F16**.
|
||||||
|
- Use if your device supports **FP16**, especially if BF16 is not available.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-bf16-q8.gguf`
|
||||||
|
- **Output & embeddings** remain in **BF16**.
|
||||||
|
- All other layers quantized to **Q8_0**.
|
||||||
|
- Use if your device supports **BF16** and you want a quantized version.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-f16-q8.gguf`
|
||||||
|
- **Output & embeddings** remain in **F16**.
|
||||||
|
- All other layers quantized to **Q8_0**.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-q4_k_l.gguf`
|
||||||
|
- **Output & embeddings** quantized to **Q8_0**.
|
||||||
|
- All other layers quantized to **Q4_K**.
|
||||||
|
- Good for **CPU inference** with limited memory.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-q4_k_m.gguf`
|
||||||
|
- Similar to Q4_K.
|
||||||
|
- Another option for **low-memory CPU inference**.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-q4_k_s.gguf`
|
||||||
|
- Smallest **Q4_K** variant, using less memory at the cost of accuracy.
|
||||||
|
- Best for **very low-memory setups**.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-q6_k_l.gguf`
|
||||||
|
- **Output & embeddings** quantized to **Q8_0**.
|
||||||
|
- All other layers quantized to **Q6_K** .
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-q6_k_m.gguf`
|
||||||
|
- A mid-range **Q6_K** quantized model for balanced performance .
|
||||||
|
- Suitable for **CPU-based inference** with **moderate memory**.
|
||||||
|
|
||||||
|
### `google_gemma-3-4b-it-q8.gguf`
|
||||||
|
- Fully **Q8** quantized model for better accuracy.
|
||||||
|
- Requires **more memory** but offers higher precision.
|
||||||
|
|
||||||
|
|
||||||
|
# Gemma 3 model card
|
||||||
|
|
||||||
|
**Model Page**: [Gemma](https://ai.google.dev/gemma/docs/core)
|
||||||
|
|
||||||
|
**Resources and Technical Documentation**:
|
||||||
|
|
||||||
|
* [Gemma 3 Technical Report][g3-tech-report]
|
||||||
|
* [Responsible Generative AI Toolkit][rai-toolkit]
|
||||||
|
* [Gemma on Kaggle][kaggle-gemma]
|
||||||
|
* [Gemma on Vertex Model Garden][vertex-mg-gemma3]
|
||||||
|
|
||||||
|
**Terms of Use**: [Terms][terms]
|
||||||
|
|
||||||
|
**Authors**: Google DeepMind
|
||||||
|
|
||||||
|
## Model Information
|
||||||
|
|
||||||
|
Summary description and brief definition of inputs and outputs.
|
||||||
|
|
||||||
|
### Description
|
||||||
|
|
||||||
|
Gemma is a family of lightweight, state-of-the-art open models from Google,
|
||||||
|
built from the same research and technology used to create the Gemini models.
|
||||||
|
Gemma 3 models are multimodal, handling text and image input and generating text
|
||||||
|
output, with open weights for both pre-trained variants and instruction-tuned
|
||||||
|
variants. Gemma 3 has a large, 128K context window, multilingual support in over
|
||||||
|
140 languages, and is available in more sizes than previous versions. Gemma 3
|
||||||
|
models are well-suited for a variety of text generation and image understanding
|
||||||
|
tasks, including question answering, summarization, and reasoning. Their
|
||||||
|
relatively small size makes it possible to deploy them in environments with
|
||||||
|
limited resources such as laptops, desktops or your own cloud infrastructure,
|
||||||
|
democratizing access to state of the art AI models and helping foster innovation
|
||||||
|
for everyone.
|
||||||
|
|
||||||
|
### Inputs and outputs
|
||||||
|
|
||||||
|
- **Input:**
|
||||||
|
- Text string, such as a question, a prompt, or a document to be summarized
|
||||||
|
- Images, normalized to 896 x 896 resolution and encoded to 256 tokens
|
||||||
|
each
|
||||||
|
- Total input context of 128K tokens for the 4B, 12B, and 27B sizes, and
|
||||||
|
32K tokens for the 1B size
|
||||||
|
|
||||||
|
- **Output:**
|
||||||
|
- Generated text in response to the input, such as an answer to a
|
||||||
|
question, analysis of image content, or a summary of a document
|
||||||
|
- Total output context of 8192 tokens
|
||||||
|
|
||||||
|
|
||||||
|
## Credits
|
||||||
|
|
||||||
|
Thanks [Bartowski](https://huggingface.co/bartowski) for imartix upload. And your guidance on quantization that has enabled me to produce these gguf file.
|
||||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
|||||||
|
{"framework": "pytorch", "task": "image-text-to-text", "allow_remote": true}
|
||||||
3
gemma-3-4b-it-bf16-q8_0.gguf
Normal file
3
gemma-3-4b-it-bf16-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e6a32ecc44453a6538d81b0cb4115ca58e0863a003ee88ce1ee6ac049098f29d
|
||||||
|
size 4759372096
|
||||||
3
gemma-3-4b-it-f16-q6_k.gguf
Normal file
3
gemma-3-4b-it-f16-q6_k.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:9dbfaa2e93a3aac10d6ba1db9efa9cf0e46efc02ebe5575ec35063ccfa3387e3
|
||||||
|
size 3982279264
|
||||||
3
gemma-3-4b-it-f16-q8_0.gguf
Normal file
3
gemma-3-4b-it-f16-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0835b91284b341dbf1250e0828780319dd788b9c6822ad06bd8ba7e9acb1b049
|
||||||
|
size 4759372096
|
||||||
3
google_gemma-3-4b-it-bf16-q8.gguf
Normal file
3
google_gemma-3-4b-it-bf16-q8.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:43c9324ba0174824e80ac9348284c060344561206d7200bda85bdb0df308b53d
|
||||||
|
size 4759372096
|
||||||
3
google_gemma-3-4b-it-bf16.gguf
Normal file
3
google_gemma-3-4b-it-bf16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:80f21bfd9fb13c6e67c77f445dd55c9a03fdc912def4894bf741f1511ba1342b
|
||||||
|
size 7767474272
|
||||||
3
google_gemma-3-4b-it-f16-q8.gguf
Normal file
3
google_gemma-3-4b-it-f16-q8.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2d37b7be6c52c077ab88d942bed8e8c971a939891cf1ca14eadb137d22aa74cc
|
||||||
|
size 4759372096
|
||||||
3
google_gemma-3-4b-it-iq3_xs.gguf
Normal file
3
google_gemma-3-4b-it-iq3_xs.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:86dde70bd1739395a684b016ef96fc6156210f451de659a75713e2b9430b4822
|
||||||
|
size 1863254336
|
||||||
3
google_gemma-3-4b-it-iq4_nl.gguf
Normal file
3
google_gemma-3-4b-it-iq4_nl.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:73c52c7078b58fcd15cd0d481b42a48ad3cd74e7cf8034352bb3101c95ad58b5
|
||||||
|
size 2363375936
|
||||||
3
google_gemma-3-4b-it-iq4_xs.gguf
Normal file
3
google_gemma-3-4b-it-iq4_xs.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3c44379412daf2e010150b90e6373c0d98a147dd634eb72fced2a6b092125a2c
|
||||||
|
size 2263105856
|
||||||
3
google_gemma-3-4b-it-mmproj-bf16.gguf
Normal file
3
google_gemma-3-4b-it-mmproj-bf16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:dfd94f00498e29cf2bf391ebe1a1a7ed6d151e75d05063f61ebbb2aaf8859cb5
|
||||||
|
size 851251104
|
||||||
3
google_gemma-3-4b-it-mmproj-f16.gguf
Normal file
3
google_gemma-3-4b-it-mmproj-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8c0fb064b019a6972856aaae2c7e4792858af3ca4561be2dbf649123ba6c40cb
|
||||||
|
size 851251104
|
||||||
3
google_gemma-3-4b-it-mmproj-f32.gguf
Normal file
3
google_gemma-3-4b-it-mmproj-f32.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:804f41f3860612815bdd915ae382cd1966ec94b801dd051ecb4a96018cd97acf
|
||||||
|
size 1679290272
|
||||||
3
google_gemma-3-4b-it-mmproj-q8.gguf
Normal file
3
google_gemma-3-4b-it-mmproj-q8.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:33b32907e4e7ef4ab8d29da6444f3033b27a242a772db7136fdc3e4d9195edc3
|
||||||
|
size 588612384
|
||||||
3
google_gemma-3-4b-it-q3_k_m.gguf
Normal file
3
google_gemma-3-4b-it-q3_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0ca6b1f6a78b27734cbac7e7c3d5cde3f7420d56478fc9186c82fe5c7553725f
|
||||||
|
size 2098323776
|
||||||
3
google_gemma-3-4b-it-q3_k_s.gguf
Normal file
3
google_gemma-3-4b-it-q3_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6a13953c679d88db95ce8c5050da64aedd83281dae90e301990e34d3d1f2c733
|
||||||
|
size 1937228096
|
||||||
3
google_gemma-3-4b-it-q4_0.gguf
Normal file
3
google_gemma-3-4b-it-q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2f2105104de860cdbd6042347d67e1f0b132ea64d0b53e5b6619706e4d6f6193
|
||||||
|
size 2525905216
|
||||||
3
google_gemma-3-4b-it-q4_1.gguf
Normal file
3
google_gemma-3-4b-it-q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:cce8329e50eb457e869e2733ef7b0323cd85a9ba7e28186fe226e77e03d14fe0
|
||||||
|
size 2726445376
|
||||||
3
google_gemma-3-4b-it-q4_k_m.gguf
Normal file
3
google_gemma-3-4b-it-q4_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2067cd277a277b16f2cd465eb10765d6d768682c7896e78269ed4e509b041c59
|
||||||
|
size 2489758016
|
||||||
3
google_gemma-3-4b-it-q4_k_s.gguf
Normal file
3
google_gemma-3-4b-it-q4_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5e86d5c003c337e486b4faa8608ac94b56b8bcfa1d23ac0519b31a33ab519138
|
||||||
|
size 2377793856
|
||||||
3
google_gemma-3-4b-it-q5_k_m.gguf
Normal file
3
google_gemma-3-4b-it-q5_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8d2180aa3eeb65dfae0bb777ecf956320a47d1ce5575711bbdebc37c87082d82
|
||||||
|
size 2829562176
|
||||||
3
google_gemma-3-4b-it-q5_k_s.gguf
Normal file
3
google_gemma-3-4b-it-q5_k_s.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2e4ece964fe19d232ed52b2b8fc74863e221f4230f7e3065f7c549352970fe4c
|
||||||
|
size 2764456256
|
||||||
3
google_gemma-3-4b-it-q6_k_m.gguf
Normal file
3
google_gemma-3-4b-it-q6_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4dda4283f62cbc5cbf16e11cf55c48490a9a57ad5d0125f136943e228b52ea8c
|
||||||
|
size 3190604096
|
||||||
3
google_gemma-3-4b-it-q8.gguf
Normal file
3
google_gemma-3-4b-it-q8.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ec257cb2b77f753e0ec77091c3046e5d6d9fe4ad622e65f7b894de1b16f369b2
|
||||||
|
size 4130226272
|
||||||
3
llama-gemma3-cli
Normal file
3
llama-gemma3-cli
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b91510639e5a0eb0528d5d5c868d3f0e5392376bd0502f12676699c828174ac2
|
||||||
|
size 2186584
|
||||||
3
mmproj.gguf
Normal file
3
mmproj.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8c0fb064b019a6972856aaae2c7e4792858af3ca4561be2dbf649123ba6c40cb
|
||||||
|
size 851251104
|
||||||
Reference in New Issue
Block a user