126 lines
3.2 KiB
Markdown
126 lines
3.2 KiB
Markdown
---
|
|
library_name: transformers
|
|
base_model:
|
|
- FlameF0X/Qwen2-0.2B-pt
|
|
license: apache-2.0
|
|
datasets:
|
|
- Salesforce/wikitext
|
|
- roneneldan/TinyStories
|
|
- FlameF0X/arXiv-AI-ML
|
|
- Skylion007/openwebtext
|
|
- flytech/python-codes-25k
|
|
- bookcorpus/bookcorpus
|
|
- HuggingFaceH4/ultrachat_200k
|
|
- openai/gsm8k
|
|
- microsoft/orca-math-word-problems-200k
|
|
- laion/OIG
|
|
- microsoft/wiki_qa
|
|
metrics:
|
|
- accuracy
|
|
model-index:
|
|
- name: FlameF0X/Qwen2-0.2B-it
|
|
results:
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
id: openai/gsm8k
|
|
name: GSM8K
|
|
type: gsm8k
|
|
config: main
|
|
split: test
|
|
metrics:
|
|
- name: Accuracy
|
|
type: accuracy
|
|
value: 2.00
|
|
verified: false
|
|
source:
|
|
name: Local Benchmark
|
|
url: https://huggingface.co/FlameF0X/Qwen2-0.2B-it
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
id: TIGER-Lab/MMLU-Pro
|
|
name: MMLU-Pro
|
|
type: TIGER-Lab/MMLU-Pro
|
|
config: default
|
|
split: test
|
|
metrics:
|
|
- name: Accuracy
|
|
type: accuracy
|
|
value: 4.00
|
|
verified: false
|
|
source:
|
|
name: Local Benchmark
|
|
url: https://huggingface.co/FlameF0X/Qwen2-0.2B-it
|
|
---
|
|
|
|
## Evaluation Results
|
|
|
|
| Benchmark | Score |
|
|
|-----------|-------|
|
|
| GSM8K (test) | 2.00% |
|
|
| MMLU-Pro (test) | 4.00% |
|
|
|
|
> Results obtained via local evaluation. Given the model size (0.2B parameters), low benchmark scores are expected.
|
|
|
|
## Model Usage
|
|
```python
|
|
import torch
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
model_path = "FlameF0X/Qwen2-0.2B-it"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_path,
|
|
torch_dtype="auto",
|
|
device_map="auto",
|
|
trust_remote_code=True
|
|
)
|
|
|
|
messages = [
|
|
{"role": "system", "content": "You are a helpful assistant."},
|
|
{"role": "user", "content": "Explain how a transformer model works in one sentence."}
|
|
]
|
|
|
|
text = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=False,
|
|
add_generation_prompt=True
|
|
)
|
|
|
|
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
|
|
|
generated_ids = model.generate(
|
|
**model_inputs,
|
|
max_new_tokens=128,
|
|
do_sample=True,
|
|
temperature=0.7
|
|
)
|
|
|
|
generated_ids = [
|
|
output_ids[len(input_ids):]
|
|
for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
|
]
|
|
|
|
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
|
print(f"--- Assistant Response ---\n{response}")
|
|
```
|
|
|
|
## Training Data
|
|
|
|
This model was instruction-tuned on a mixture of:
|
|
|
|
- `Salesforce/wikitext` — General text
|
|
- `roneneldan/TinyStories` — Short story generation
|
|
- `FlameF0X/arXiv-AI-ML` — AI/ML research papers
|
|
- `Skylion007/openwebtext` — Web text
|
|
- `flytech/python-codes-25k` — Python code
|
|
- `bookcorpus/bookcorpus` — Books
|
|
- `HuggingFaceH4/ultrachat_200k` — Instruction following
|
|
- `openai/gsm8k` — Math reasoning
|
|
- `microsoft/orca-math-word-problems-200k` — Math word problems
|
|
- `laion/OIG` — Open instruction generalist
|
|
- `microsoft/wiki_qa` — Question answering |