Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
SmolLM2-135M-Instruct-bnb-4…/README.md
ModelHub XC 2561157d6d 初始化项目，由ModelHub XC社区提供模型 
Model: Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning-GGUF
Source: Original Platform
2026-09-01 15:30:23 +08:00

1.4 KiB
Raw Permalink Blame History

license, language, pipeline_tag, base_model, tags
license language pipeline_tag base_model tags
apache-2.0
en
text-generation Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning
gguf
llama.cpp
ollama
reasoning
smollm2

SmolLM2-135M-Instruct-Reasoning-GGUF

GGUF conversions of:

Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning

Available Quantizations

Quantization Recommended Use
Q4_0 Same as Q4_K_M
Q4_K_M Best balance of quality and size
Q5_K_M Better quality
Q6_K Better quaity smaller than Q8_0
Q8_0 Highest quality quantized version

Chat Template

This model uses the standard SmolLM2 chat template.

<|im_start|>system You are a helpful AI assistant named SmolLM, trained by Hugging Face <|im_end|>

<|im_start|>user Hello <|im_end|>

<|im_start|>assistant

Ollama Example

FROM ./SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf

TEMPLATE """ {{ .Prompt }} """

llama.cpp Example

./llama-cli -m SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf

Notes

When using GGUF versions, ensure the same chat template used during training is applied during inference. Incorrect templates can significantly reduce model quality.

Source Model

Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning