---
license: apache-2.0
language:
  - en
pipeline_tag: text-generation

base_model: Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning

tags:

- gguf
- llama.cpp
- ollama
- reasoning
- smollm2
---

# SmolLM2-135M-Instruct-Reasoning-GGUF

GGUF conversions of:

Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning

## Available Quantizations

| Quantization | Recommended Use                   |
| ------------ | --------------------------------- |
| Q4_0         | Same as Q4_K_M                    |
| Q4_K_M       | Best balance of quality and size  |
| Q5_K_M       | Better quality                    |
| Q6_K         | Better quaity smaller than Q8_0   |
| Q8_0         | Highest quality quantized version |

## Chat Template

This model uses the standard SmolLM2 chat template.

<|im_start|>system
You are a helpful AI assistant named SmolLM, trained by Hugging Face
<|im_end|>

<|im_start|>user
Hello
<|im_end|>

<|im_start|>assistant

## Ollama Example

FROM ./SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf

TEMPLATE """
{{ .Prompt }}
"""

## llama.cpp Example

./llama-cli 
-m SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf

## Notes

When using GGUF versions, ensure the same chat template used during training is applied during inference. Incorrect templates can significantly reduce model quality.

## Source Model

Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning