初始化项目,由ModelHub XC社区提供模型
Model: OEvortex/vortex-3b Source: Original Platform
This commit is contained in:
147
README.md
Normal file
147
README.md
Normal file
@@ -0,0 +1,147 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
license: other
|
||||
tags:
|
||||
- HelpingAI
|
||||
- vortex
|
||||
datasets:
|
||||
- OEvortex/Vortex-50k
|
||||
license_name: helpingai
|
||||
license_link: LICENSE.md
|
||||
pipeline_tag: text-generation
|
||||
model-index:
|
||||
- name: vortex-3b
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: AI2 Reasoning Challenge (25-Shot)
|
||||
type: ai2_arc
|
||||
config: ARC-Challenge
|
||||
split: test
|
||||
args:
|
||||
num_few_shot: 25
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 31.91
|
||||
name: normalized accuracy
|
||||
source:
|
||||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OEvortex/vortex-3b
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: HellaSwag (10-Shot)
|
||||
type: hellaswag
|
||||
split: validation
|
||||
args:
|
||||
num_few_shot: 10
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: 56.89
|
||||
name: normalized accuracy
|
||||
source:
|
||||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OEvortex/vortex-3b
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: MMLU (5-Shot)
|
||||
type: cais/mmlu
|
||||
config: all
|
||||
split: test
|
||||
args:
|
||||
num_few_shot: 5
|
||||
metrics:
|
||||
- type: acc
|
||||
value: 27.32
|
||||
name: accuracy
|
||||
source:
|
||||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OEvortex/vortex-3b
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: TruthfulQA (0-shot)
|
||||
type: truthful_qa
|
||||
config: multiple_choice
|
||||
split: validation
|
||||
args:
|
||||
num_few_shot: 0
|
||||
metrics:
|
||||
- type: mc2
|
||||
value: 37.39
|
||||
source:
|
||||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OEvortex/vortex-3b
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: Winogrande (5-shot)
|
||||
type: winogrande
|
||||
config: winogrande_xl
|
||||
split: validation
|
||||
args:
|
||||
num_few_shot: 5
|
||||
metrics:
|
||||
- type: acc
|
||||
value: 60.14
|
||||
name: accuracy
|
||||
source:
|
||||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OEvortex/vortex-3b
|
||||
name: Open LLM Leaderboard
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Text Generation
|
||||
dataset:
|
||||
name: GSM8k (5-shot)
|
||||
type: gsm8k
|
||||
config: main
|
||||
split: test
|
||||
args:
|
||||
num_few_shot: 5
|
||||
metrics:
|
||||
- type: acc
|
||||
value: 0.91
|
||||
name: accuracy
|
||||
source:
|
||||
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=OEvortex/vortex-3b
|
||||
name: Open LLM Leaderboard
|
||||
---
|
||||

|
||||
**Model Overview**
|
||||
|
||||
vortex-3b is a 2.78 billion parameter causal language model created by OEvortex that is derived from EleutherAI's Pythia-2.8b and fine-tuned on Vortex-50k dataset'
|
||||
|
||||
```python
|
||||
from transformers import pipeline
|
||||
|
||||
# Initialize the pipeline
|
||||
pipe = pipeline("text-generation", model="OEvortex/vortex-3b")
|
||||
|
||||
# Use the pipeline
|
||||
text = "Once upon a time"
|
||||
generated_text = pipe(text, max_length=100, do_sample=True)[0]['generated_text']
|
||||
|
||||
print(generated_text)
|
||||
```
|
||||
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
||||
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_OEvortex__vortex-3b)
|
||||
|
||||
|
||||
| Metric | vortex 3b | vortex 3b-v2 | dolly-v2-3b | pythia-2.8b-deduped |
|
||||
|---------|----------:|-------------:|------------------:|----------------------------------:|
|
||||
| Avg. | 35.76 | 37.46 | 25.26 | 36.72 |
|
||||
| AI2 Reasoning Challenge (25-Shot) | 31.91 | 39.68 | 22.83 | 36.26 |
|
||||
| HellaSwag (10-Shot) | 56.89 | 65.04 | 26.55 | 60.66 |
|
||||
| MMLU (5-Shot) | 27.32 | 25.09 | 24.7 | 26.78 |
|
||||
| TruthfulQA (0-shot) | 37.39 | 33.80 | 0 | 35.56 |
|
||||
| Winogrande (5-shot) | 60.14 | 59.12 | 59.43 | 60.22 |
|
||||
| GSM8k (5-shot) | 0.91 | 2.05 | 1.86 | 0.83 |
|
||||
|
||||
Reference in New Issue
Block a user