matrixportalx/gemma-2-9b-it-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
gemma-2-9b-it-GGUF/README.md
ModelHub XC a7d06f48f1 初始化项目，由ModelHub XC社区提供模型 
Model: matrixportalx/gemma-2-9b-it-GGUF
Source: Original Platform
2026-05-30 12:33:18 +08:00

8.4 KiB
Raw Permalink Blame History

base_model, library_name, license, pipeline_tag, tags, extra_gated_heading, extra_gated_prompt, extra_gated_button_content
base_model library_name license pipeline_tag tags extra_gated_heading extra_gated_prompt extra_gated_button_content
google/gemma-2-9b-it transformers gemma text-generation
conversational
llama-cpp
matrixportal
Access Gemma on Hugging Face To access Gemma on Hugging Face, you’re required to review and agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging Face and click below. Requests are processed immediately. Acknowledge license

matrixportal/gemma-2-9b-it-GGUF

This model was converted to GGUF format from google/gemma-2-9b-it using llama.cpp via the ggml.ai's all-gguf-same-where space. Refer to the original model card for more details on the model.

You can access the list of available quantized model files and download links here.

✅ Quantized Models Download List

🔍 Recommended Quantizations

  • ✨ General CPU Use: Q4_K_M (Best balance of speed/quality)
  • 📱 ARM Devices: Q4_0 (Optimized for ARM CPUs)
  • 🏆 Maximum Quality: Q8_0 (Near-original quality)

📦 Full Quantization Options

🚀 Download 🔢 Type 📝 Notes
Download Q2_K Basic quantization
Download Q3_K_S Small size
Download Q3_K_M Balanced quality
Download Q3_K_L Better quality
Download Q4_0 Fast on ARM
Download Q4_K_S Fast, recommended
Download Q4_K_M ⭐ Best balance
Download Q5_0 Good quality
Download Q5_K_S Balanced
Download Q5_K_M High quality
Download Q6_K 🏆 Very good quality
Download Q8_0 ⚡ Fast, best quality
Download F16 Maximum accuracy

💡 Tip: Use F16 for maximum precision when quality is critical

🚀 Tools for Running GGUF Quantized LLMs Locally

🖥️ Desktop Applications

Application Description Download Link
Llama.cpp A fast and efficient inference engine for GGUF models. GitHub Repository
Ollama A streamlined solution for running LLMs locally. Website
AnythingLLM An AI-powered knowledge management tool. GitHub Repository
Open WebUI A user-friendly web interface for running local LLMs. GitHub Repository
GPT4All A user-friendly desktop application supporting various LLMs, compatible with GGUF models. GitHub Repository
LM Studio A desktop application designed to run and manage local LLMs, supporting GGUF format. Website
GPT4All Chat A chat application compatible with GGUF models for local, offline interactions. GitHub Repository

📱 Mobile Applications

Application Description Download Link
ChatterUI A simple and lightweight LLM app for mobile devices. GitHub Repository
Maid Mobile Artificial Intelligence Distribution for running AI models on mobile devices. GitHub Repository
PocketPal AI A mobile AI assistant powered by local models. GitHub Repository
Layla A flexible platform for running various AI models on mobile devices. Website

🎨 Image Generation Applications

Application Description Download Link
Stable Diffusion An open-source AI model for generating images from text. GitHub Repository
Stable Diffusion WebUI A web application providing access to Stable Diffusion models via a browser interface. GitHub Repository
Local Dream Android Stable Diffusion with Snapdragon NPU acceleration. Also supports CPU inference. GitHub Repository
Stable-Diffusion-Android (SDAI) An open-source AI art application for Android devices, enabling digital art creation. GitHub Repository