初始化项目,由ModelHub XC社区提供模型
Model: Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning-GGUF Source: Original Platform
This commit is contained in:
67
README.md
Normal file
67
README.md
Normal file
@@ -0,0 +1,67 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
|
||||
base_model: Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning
|
||||
|
||||
tags:
|
||||
|
||||
- gguf
|
||||
- llama.cpp
|
||||
- ollama
|
||||
- reasoning
|
||||
- smollm2
|
||||
---
|
||||
|
||||
# SmolLM2-135M-Instruct-Reasoning-GGUF
|
||||
|
||||
GGUF conversions of:
|
||||
|
||||
Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning
|
||||
|
||||
## Available Quantizations
|
||||
|
||||
| Quantization | Recommended Use |
|
||||
| ------------ | --------------------------------- |
|
||||
| Q4_0 | Same as Q4_K_M |
|
||||
| Q4_K_M | Best balance of quality and size |
|
||||
| Q5_K_M | Better quality |
|
||||
| Q6_K | Better quaity smaller than Q8_0 |
|
||||
| Q8_0 | Highest quality quantized version |
|
||||
|
||||
## Chat Template
|
||||
|
||||
This model uses the standard SmolLM2 chat template.
|
||||
|
||||
<|im_start|>system
|
||||
You are a helpful AI assistant named SmolLM, trained by Hugging Face
|
||||
<|im_end|>
|
||||
|
||||
<|im_start|>user
|
||||
Hello
|
||||
<|im_end|>
|
||||
|
||||
<|im_start|>assistant
|
||||
|
||||
## Ollama Example
|
||||
|
||||
FROM ./SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf
|
||||
|
||||
TEMPLATE """
|
||||
{{ .Prompt }}
|
||||
"""
|
||||
|
||||
## llama.cpp Example
|
||||
|
||||
./llama-cli
|
||||
-m SmolLM2-135M-Instruct-bnb-4bit-reasoning-Q4_K_M.gguf
|
||||
|
||||
## Notes
|
||||
|
||||
When using GGUF versions, ensure the same chat template used during training is applied during inference. Incorrect templates can significantly reduce model quality.
|
||||
|
||||
## Source Model
|
||||
|
||||
Avtrkrb/SmolLM2-135M-Instruct-bnb-4bit-reasoning
|
||||
Reference in New Issue
Block a user