prithivMLmods/SmolLM2-CoT-360M-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: prithivMLmods/SmolLM2-CoT-360M-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-17 12:42:14 +08:00
commit 8b5789d7dc
7 changed files with 340 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
.gitattributes vendored Normal file
View File

@@ -0,0 +1,39 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
SmolLM2-CoT-360M.F16.gguf filter=lfs diff=lfs merge=lfs -text
SmolLM2-CoT-360M.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
SmolLM2-CoT-360M.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
SmolLM2-CoT-360M.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

288
README.md Normal file
View File

@@ -0,0 +1,288 @@
---
license: apache-2.0
language:
- en
base_model:
- prithivMLmods/SmolLM2-CoT-360M
datasets:
- prithivMLmods/Deepthink-Reasoning
library_name: transformers
tags:
- smolLM
- llama
- CoT
- Thinker
- text-generation-inference
pipeline_tag: text-generation
---
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/bMONeEIzYGnh7b7oppgBN.png)
# **SMOLLM CoT 360M GGUF ON CUSTOM SYNTHETIC DATA**
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. Fine-tuning a language model like SmolLM involves several steps, from setting up the environment to training the model and saving the results. Below is a detailed step-by-step guide based on the provided notebook file
# How to use with `Transformers`
```bash
pip install transformers
```
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "prithivMLmods/SmolLM2-CoT-360M"
device = "cuda" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
# for multiple GPUs install accelerate and do `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
messages = [{"role": "user", "content": "What is the capital of France."}]
input_text=tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0]))
```
### **Step 1: Setting Up the Environment**
Before diving into fine-tuning, you need to set up your environment with the necessary libraries and tools.
1. **Install Required Libraries**:
- Install the necessary Python libraries using `pip`. These include `transformers`, `datasets`, `trl`, `torch`, `accelerate`, `bitsandbytes`, and `wandb`.
- These libraries are essential for working with Hugging Face models, datasets, and training loops.
```python
!pip install transformers datasets trl torch accelerate bitsandbytes wandb
```
2. **Import Necessary Modules**:
- Import the required modules from the installed libraries. These include `AutoModelForCausalLM`, `AutoTokenizer`, `TrainingArguments`, `pipeline`, `load_dataset`, and `SFTTrainer`.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, pipeline
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer, setup_chat_format
import torch
import os
```
3. **Detect Device (GPU, MPS, or CPU)**:
- Detect the available hardware (GPU, MPS, or CPU) to ensure the model runs on the most efficient device.
```python
device = (
"cuda"
if torch.cuda.is_available()
else "mps" if torch.backends.mps.is_available() else "cpu"
)
```
---
### **Step 2: Load the Pre-trained Model and Tokenizer**
Next, load the pre-trained SmolLM model and its corresponding tokenizer.
1. **Load the Model and Tokenizer**:
- Use `AutoModelForCausalLM` and `AutoTokenizer` to load the SmolLM model and tokenizer from Hugging Face.
```python
model_name = "HuggingFaceTB/SmolLM2-360M"
model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path=model_name)
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path=model_name)
```
2. **Set Up Chat Format**:
- Use the `setup_chat_format` function to prepare the model and tokenizer for chat-based tasks.
```python
model, tokenizer = setup_chat_format(model=model, tokenizer=tokenizer)
```
3. **Test the Base Model**:
- Test the base model with a simple prompt to ensure it’s working correctly.
```python
prompt = "Explain AGI ?"
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0 if device == "cuda" else -1)
print(pipe(prompt, max_new_tokens=200))
```
4. **If: Encountering**:
- Chat template is already added to the tokenizer, indicates that the tokenizer already has a predefined chat template, which prevents the setup_chat_format() from modifying it again.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model_name = "HuggingFaceTB/SmolLM2-1.7B-Instruct"
model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path=model_name)
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path=model_name)
tokenizer.chat_template = None
from trl.models.utils import setup_chat_format
model, tokenizer = setup_chat_format(model=model, tokenizer=tokenizer)
prompt = "Explain AGI?"
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device=0)
print(pipe(prompt, max_new_tokens=200))
```
*📍 Else Skip the Part [ Step 4 ] !*
---
### **Step 3: Load and Prepare the Dataset**
Fine-tuning requires a dataset. In this case, we’re using a custom dataset called `Deepthink-Reasoning`.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/JIwUAT-NpqpN18zUdo6uW.png)
1. **Load the Dataset**:
- Use the `load_dataset` function to load the dataset from Hugging Face.
```python
ds = load_dataset("prithivMLmods/Deepthink-Reasoning")
```
2. **Tokenize the Dataset**:
- Define a tokenization function that processes the dataset in batches. This function applies the chat template to each prompt-response pair and tokenizes the text.
```python
def tokenize_function(examples):
prompts = [p.strip() for p in examples["prompt"]]
responses = [r.strip() for r in examples["response"]]
texts = [
tokenizer.apply_chat_template(
[{"role": "user", "content": p}, {"role": "assistant", "content": r}],
tokenize=False
)
for p, r in zip(prompts, responses)
]
return tokenizer(texts, truncation=True, padding="max_length", max_length=512)
```
3. **Apply Tokenization**:
- Apply the tokenization function to the dataset.
```python
ds = ds.map(tokenize_function, batched=True)
```
---
### **Step 4: Configure Training Arguments**
Set up the training arguments to control the fine-tuning process.
1. **Define Training Arguments**:
- Use `TrainingArguments` to specify parameters like batch size, learning rate, number of steps, and optimization settings.
```python
use_bf16 = torch.cuda.is_bf16_supported()
training_args = TrainingArguments(
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
warmup_steps=5,
max_steps=60,
learning_rate=2e-4,
fp16=not use_bf16,
bf16=use_bf16,
logging_steps=1,
optim="adamw_8bit",
weight_decay=0.01,
lr_scheduler_type="linear",
seed=3407,
output_dir="outputs",
report_to="wandb",
)
```
---
### **Step 5: Initialize the Trainer**
Initialize the `SFTTrainer` with the model, tokenizer, dataset, and training arguments.
```python
trainer = SFTTrainer(
model=model,
processing_class=tokenizer,
train_dataset=ds["train"],
args=training_args,
)
```
---
### **Step 6: Start Training**
Begin the fine-tuning process by calling the `train` method on the trainer.
```python
trainer.train()
```
---
### **Step 7: Save the Fine-Tuned Model**
After training, save the fine-tuned model and tokenizer to a local directory.
1. **Save Model and Tokenizer**:
- Use the `save_pretrained` method to save the model and tokenizer.
```python
save_directory = "/content/my_model"
model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)
```
2. **Zip and Download the Model**:
- Zip the saved directory and download it for future use.
```python
import shutil
shutil.make_archive(save_directory, 'zip', save_directory)
from google.colab import files
files.download(f"{save_directory}.zip")
```
---
### **Run with Ollama [Ollama Run]**
Ollama makes running machine learning models simple and efficient. Follow these steps to set up and run your GGUF models quickly.
## Quick Start: Step-by-Step Guide
| Step | Description | Command / Instructions |
|------|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1 | **Install Ollama 🦙** | Download Ollama from [https://ollama.com/download](https://ollama.com/download) and install it on your system. |
| 2 | **Create Your Model File** | - Create a file named after your model, e.g., `metallama`. |
| | | - Add the following line to specify the base model: |
| | | ```bash |
| | | FROM Llama-3.2-1B.F16.gguf |
| | | ``` |
| | | - Ensure the base model file is in the same directory. |
| 3 | **Create and Patch the Model** | Run the following commands to create and verify your model: |
| | | ```bash |
| | | ollama create metallama -f ./metallama |
| | | ollama list |
| | | ``` |
| 4 | **Run the Model** | Use the following command to start your model: |
| | | ```bash |
| | | ollama run metallama |
| | | ``` |
| 5 | **Interact with the Model** | Once the model is running, interact with it: |
| | | ```plaintext |
| | | >>> Tell me about Space X. |
| | | Space X, the private aerospace company founded by Elon Musk, is revolutionizing space exploration... |
| | | ``` |
### **Model & Quant**
| **Item** | **Link** |
|----------|----------|
| **Model** | [SmolLM2-CoT-360M](https://huggingface.co/prithivMLmods/SmolLM2-CoT-360M) |
| **Quantized Version** | [SmolLM2-CoT-360M-GGUF](https://huggingface.co/prithivMLmods/SmolLM2-CoT-360M-GGUF) |
### **Conclusion**
Fine-tuning SmolLM involves setting up the environment, loading the model and dataset, configuring training parameters, and running the training loop. By following these steps, you can adapt SmolLM to your specific use case, whether it’s for reasoning tasks, chat-based applications, or other NLP tasks.
This process is highly customizable, so feel free to experiment with different datasets, hyperparameters, and training strategies to achieve the best results for your project.
---

3
SmolLM2-CoT-360M.F16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c9c60898a97eb9665b090210ea9fa455cecbe768d164cde93297415412ecbec2
size 725553536

3
SmolLM2-CoT-360M.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ec963be5ad2749a0d84a4786791b67175b07dbdc597bd58fa4f8ab7487ffae28
size 270590336

3
SmolLM2-CoT-360M.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ed2a17980abeaf54826e42a11f2950865e703b1ead5b5380b46ffac5053c44f3
size 289943936

3
SmolLM2-CoT-360M.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4aa1f0a370c5110b95bbad7e4833de36f1f9710c8833a67f882743e1efdc2619
size 386404736

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}