初始化项目，由ModelHub XC社区提供模型
Model: PursuitOfDataScience/llama3.2-1b-thinking Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
212
README.md
Normal file
212
README.md
Normal file
@@ -0,0 +1,212 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
license: other
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- llama
|
||||
- chat
|
||||
- sft
|
||||
- reasoning
|
||||
- cot
|
||||
- ultrachat
|
||||
- mixture-of-thoughts
|
||||
- dpo
|
||||
base_model: meta-llama/Llama-3.2-1B
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# PursuitOfDataScience/llama3.2-1b-thinking
|
||||
|
||||
This repository contains a three-stage fine-tuned version of **meta-llama/Llama-3.2-1B**:
|
||||
|
||||
1. **Supervised fine-tuning (SFT)** on a local copy of **HuggingFaceH4/ultrachat_200k**
|
||||
using an instruction-style, multi-turn chat objective.
|
||||
2. **Reasoning training** to enhance step-by-step reasoning capabilities,
|
||||
building on the SFT model using the `open-r1/Mixture-of-Thoughts` dataset.
|
||||
3. **Direct Preference Optimization (DPO)** alignment using the `mlabonne/orpo-dpo-mix-40k` dataset
|
||||
to improve response quality and alignment with human preferences.
|
||||
|
||||
## Model details
|
||||
|
||||
- **Base model**: `meta-llama/Llama-3.2-1B`
|
||||
- **Stage 1 objective**: Supervised fine-tuning for helpful, concise chat responses
|
||||
on Ultrachat-style conversations.
|
||||
- **Stage 2 objective**: Specialized reasoning training to improve logical reasoning and
|
||||
Chain of Thought (CoT) capabilities using step-by-step reasoning traces from `open-r1/Mixture-of-Thoughts`.
|
||||
- **Stage 3 objective**: DPO alignment to refine responses based on preference data from `mlabonne/orpo-dpo-mix-40k`,
|
||||
enhancing safety, helpfulness, and adherence to user constraints.
|
||||
- **Context length**: Up to 131072 tokens (subject to the base model config).
|
||||
- **Training data**:
|
||||
- SFT: multi-turn dialogues from `HuggingFaceH4/ultrachat_200k`.
|
||||
- Reasoning: `open-r1/Mixture-of-Thoughts` dataset with step-by-step reasoning traces.
|
||||
- DPO: preference pairs from `mlabonne/orpo-dpo-mix-40k`.
|
||||
|
||||
## Inference usage
|
||||
|
||||
The model is trained in a **chat-style** setup. At inference time, prompts are built
|
||||
as a list of `messages` and passed through the model's native `chat_template`
|
||||
via `tokenizer.apply_chat_template`:
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
repo_id = "PursuitOfDataScience/llama3.2-1b-thinking"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(repo_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
repo_id,
|
||||
device_map="auto",
|
||||
)
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": (
|
||||
"You are a helpful, concise assistant. "
|
||||
"Write clear, well-structured answers that follow the user's constraints."
|
||||
),
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Explain how someone can build a consistent daily learning habit.",
|
||||
},
|
||||
]
|
||||
|
||||
prompt_text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
)
|
||||
|
||||
inputs = tokenizer(prompt_text, return_tensors="pt").to(model.device)
|
||||
|
||||
outputs = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=512,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
temperature=0.7,
|
||||
top_p=0.9,
|
||||
do_sample=True,
|
||||
)
|
||||
|
||||
# Decode only the generated continuation (excluding the prompt tokens)
|
||||
generated_tokens = outputs[0][inputs["input_ids"].shape[1]:]
|
||||
response = tokenizer.decode(generated_tokens, skip_special_tokens=True)
|
||||
print(response)
|
||||
```
|
||||
|
||||
### Multi-turn example
|
||||
|
||||
```python
|
||||
messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": (
|
||||
"You are a helpful, concise assistant. "
|
||||
"Write clear, well-structured answers that follow the user's constraints."
|
||||
),
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Describe the main trade-offs between using small and large language models.",
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "Small models are cheaper and faster, while large models are usually more capable...",
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Give me a bullet-point summary from the perspective of a startup.",
|
||||
},
|
||||
]
|
||||
|
||||
prompt_text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
)
|
||||
|
||||
inputs = tokenizer(prompt_text, return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=256,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
temperature=0.7,
|
||||
top_p=0.9,
|
||||
do_sample=True,
|
||||
)
|
||||
response = tokenizer.decode(
|
||||
outputs[0][inputs["input_ids"].shape[1]:],
|
||||
skip_special_tokens=True,
|
||||
)
|
||||
print(response)
|
||||
```
|
||||
|
||||
### Chain of Thought (CoT) reasoning example
|
||||
|
||||
For reasoning tasks, the model can generate step-by-step thoughts using `<think>` tags:
|
||||
|
||||
```python
|
||||
messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": (
|
||||
"You are a helpful, concise assistant. "
|
||||
"Use Chain of Thought reasoning with <think> tags for complex problems."
|
||||
),
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "If a train travels 60 km in 1 hour, how long will it take to travel 180 km?",
|
||||
},
|
||||
]
|
||||
|
||||
prompt_text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
)
|
||||
|
||||
inputs = tokenizer(prompt_text, return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=512,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
temperature=0.7,
|
||||
top_p=0.9,
|
||||
do_sample=True,
|
||||
)
|
||||
response = tokenizer.decode(
|
||||
outputs[0][inputs["input_ids"].shape[1]:],
|
||||
skip_special_tokens=True,
|
||||
)
|
||||
print(response)
|
||||
# Example output: <think> The train travels 60 km in 1 hour, so speed is 60 km/h. For 180 km, time = distance / speed = 180 / 60 = 3 hours. </think> It will take 3 hours.
|
||||
```
|
||||
|
||||
## Training pipeline (summary)
|
||||
|
||||
1. **Instruction SFT (Ultrachat)**:
|
||||
- Conversations are converted into lists of `messages`.
|
||||
- For each assistant turn, a single training example is built using
|
||||
`tokenizer.apply_chat_template`.
|
||||
- Loss is applied only on assistant tokens; system and user tokens are masked.
|
||||
|
||||
2. **Reasoning Training**:
|
||||
- Fine-tuning on the `open-r1/Mixture-of-Thoughts` dataset with step-by-step reasoning traces to enhance CoT capabilities.
|
||||
- Uses reinforcement learning or supervised methods to align with logical reasoning patterns.
|
||||
|
||||
3. **DPO Alignment**:
|
||||
- Fine-tuning with Direct Preference Optimization on the `mlabonne/orpo-dpo-mix-40k` dataset.
|
||||
- Optimizes the model to prefer chosen responses over rejected ones, improving overall alignment.
|
||||
|
||||
## Limitations
|
||||
|
||||
- This is a relatively small (1B parameter) model and may hallucinate or
|
||||
struggle on complex, multi-step reasoning tasks.
|
||||
- Outputs may be inaccurate, unsafe, or biased. Always verify critical
|
||||
information before using it in production.
|
||||
36
config.json
Normal file
36
config.json
Normal file
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"dtype": "bfloat16",
|
||||
"eos_token_id": 128001,
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2048,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 8192,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 16,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 128001,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": {
|
||||
"factor": 32.0,
|
||||
"high_freq_factor": 4.0,
|
||||
"low_freq_factor": 1.0,
|
||||
"original_max_position_embeddings": 8192,
|
||||
"rope_type": "llama3"
|
||||
},
|
||||
"rope_theta": 500000.0,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "4.57.1",
|
||||
"use_cache": false,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
10
generation_config.json
Normal file
10
generation_config.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 128000,
|
||||
"eos_token_id": [
|
||||
128001
|
||||
],
|
||||
"pad_token_id": 128001,
|
||||
"transformers_version": "4.57.1",
|
||||
"use_cache": false
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c5996c8e2ffa1a18f668531c022df29eff74d17a97cccee0219fc88cfaecc3ae
|
||||
size 2471645608
|
||||
23
special_tokens_map.json
Normal file
23
special_tokens_map.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|begin_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|end_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
|
||||
size 17209920
|
||||
2063
tokenizer_config.json
Normal file
2063
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
3
training_args.bin
Normal file
3
training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0ba6afbea00f01bf2c85893273c3ff67b7256f1487b806068ebf056259a3c1f9
|
||||
size 6456
|
||||
Reference in New Issue
Block a user