初始化项目,由ModelHub XC社区提供模型
Model: srmty/smolLM2-360M-instruct-math-v1 Source: Original Platform
This commit is contained in:
70
README.md
Normal file
70
README.md
Normal file
@@ -0,0 +1,70 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: HuggingFaceTB/SmolLM2-360M
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- smollm2
|
||||
- instruction-tuning
|
||||
- supervised-fine-tuning
|
||||
- alpaca
|
||||
- math
|
||||
- reasoning
|
||||
- gpteacher
|
||||
- metamath
|
||||
- small-language-models
|
||||
---
|
||||
|
||||
# SmolLM2-360M Math Instruct
|
||||
|
||||
This is an instruction-tuned version of `HuggingFaceTB/SmolLM2-360M`.
|
||||
|
||||
The model was fine-tuned on a small mixed instruction dataset containing general instruction-following examples and math reasoning examples.
|
||||
## Quick start
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
model_id = "srmty/smolLM2-360M-math-instruct"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
|
||||
device_map="auto" if torch.cuda.is_available() else None,
|
||||
)
|
||||
|
||||
if tokenizer.pad_token is None:
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
|
||||
model.eval()
|
||||
|
||||
```
|
||||
|
||||
|
||||
## Training Data
|
||||
|
||||
The training mix used:
|
||||
|
||||
- `teknium/GPTeacher-General-Instruct`
|
||||
- `meta-math/MetaMathQA` subset
|
||||
|
||||
The data was formatted using Alpaca-style prompts.
|
||||
|
||||
## Prompt Format
|
||||
|
||||
Use this format during inference:
|
||||
|
||||
```text
|
||||
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
|
||||
|
||||
### Instruction:
|
||||
{instruction}
|
||||
|
||||
### Input:
|
||||
{input}
|
||||
|
||||
### Response:
|
||||
Reference in New Issue
Block a user