license, language, library_name, pipeline_tag, tags, base_model, datasets

license	language	library_name	pipeline_tag	tags	base_model	datasets
apache-2.0	en	gguf	text-generation	gguf llama.cpp quantized granite reasoning unsloth text-generation	ermiaazarkhalili/Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth	ermiaazarkhalili/claude-reasoning-distillation

Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth-GGUF

GGUF quantizations of Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth for CPU and edge inference with llama.cpp, Ollama, LM Studio, and other GGUF runtimes.

This model is a fine-tune of Granite 4.1-8B for reasoning distillation (chain-of-thought), trained with Unsloth on claude-reasoning-distillation.

• Full-precision model: Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth
• Base model: Granite 4.1-8B
• Parameters: 8B

Available Quantizations

Quant	Size	Recommended Use	File
Q2_K	3.41 GB	Smallest, lowest quality — quick tests / very constrained devices	granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q2_k.gguf
Q3_K_M	4.35 GB	Small, acceptable quality	granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q3_k_m.gguf
Q4_K_M	5.35 GB	Recommended — best size/quality balance	granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q4_k_m.gguf
Q5_K_M	6.25 GB	High quality, larger	granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q5_k_m.gguf
Q6_K	7.22 GB	Very high quality, near-fp16	granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q6_k.gguf
Q8_0	9.35 GB	Near-lossless, largest	granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q8_0.gguf

Q4_K_M is the recommended default for most users.

Usage

Download a single quant

pip install -U "huggingface_hub[cli]"
hf download ermiaazarkhalili/Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth-GGUF \
  --include "*q4_k_m*.gguf" --local-dir ./Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth-GGUF

llama.cpp

# build: https://github.com/ggerganov/llama.cpp
./llama-cli -m ./Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth-GGUF/granite-4.1-8b-sft-claude-opus-reasoning-unsloth.q2_k.gguf \
  -p "Solve step by step: What is the sum of the first 10 prime numbers?" -n 512

Ollama

ollama run hf.co/ermiaazarkhalili/Granite-4.1-8B-SFT-Claude-Opus-Reasoning-Unsloth-GGUF:Q4_K_M "Solve step by step: What is the sum of the first 10 prime numbers?"

Training Outcome

Metric	Value
SLURM Job ID	38330897
Runtime	32m 49s
Final Training Loss	0.7895
Peak VRAM	9.26 GB
GPU	H100 80GB HBM3 (MIG 3g.40gb)

License

APACHE-2.0 — see the base model for full terms.

Acknowledgments

• Unsloth for 2x faster fine-tuning
• llama.cpp for GGUF quantization
• Base model developers (ibm-granite)
• Compute Canada / DRAC for HPC resources