45 lines
1.6 KiB
Markdown
45 lines
1.6 KiB
Markdown
|
---
|
|||
|
|
license: gemma
|
|||
|
|
library_name: mlx
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
extra_gated_heading: Access Gemma on Hugging Face
|
|||
|
|
extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and
|
|||
|
|
agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging
|
|||
|
|
Face and click below. Requests are processed immediately.
|
|||
|
|
extra_gated_button_content: Acknowledge license
|
|||
|
|
tags:
|
|||
|
|
- conversational
|
|||
|
|
- mlx
|
|||
|
|
- ollama
|
|||
|
|
base_model: google/gemma-2-2b-it
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Rocky Gemma 2B (It)
|
|||
|
|
|
|||
|
|
Rocky Gemma 2B is a fine-tuned model based on `google/gemma-2-2b-it`, customized to speak and act like Rocky.
|
|||
|
|
|
|||
|
|
## 🦙 Run with Ollama
|
|||
|
|
|
|||
|
|
You can download and run this model instantly on your local Ollama instance with a single command:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
curl -sSL https://huggingface.co/Phani1479432/rocky-gemma-2b/raw/main/Modelfile | ollama create rocky -f - && ollama run rocky
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### What this command does:
|
|||
|
|
1. **Downloads the Modelfile**: Fetches the custom configuration specifying stop parameters and formatting templates directly from Hugging Face.
|
|||
|
|
2. **Creates the Model locally**: Registers a new local model named `rocky` in Ollama.
|
|||
|
|
3. **Downloads the GGUF Weights**: Automatically pulls the quantized GGUF weights (`rocky-q4_k_m.gguf`) directly from Hugging Face.
|
|||
|
|
4. **Starts the Chat**: Launches an interactive shell session for you to chat with Rocky.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ⚡ Run with MLX
|
|||
|
|
|
|||
|
|
If you are on Apple Silicon, you can use the MLX library to run the fused model weights:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
pip install mlx-lm
|
|||
|
|
python -m mlx_lm.generate --model Phani1479432/rocky-gemma-2b --prompt "Hey Rocky, what's your training routine like?"
|
|||
|
|
```
|