初始化项目，由ModelHub XC社区提供模型
Model: refuelai/Llama-3-Refueled Source: Original Platform
This commit is contained in:
ModelHub XC
84
README.md
Normal file
84
README.md
Normal file
@@ -0,0 +1,84 @@
|
||||
---
|
||||
license: cc-by-nc-4.0
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
tags:
|
||||
- data labeling
|
||||
---
|
||||
<div style="width: auto; margin-left: auto; margin-right: auto; background-color:black">
|
||||
<img src="https://assets-global.website-files.com/6423879a8f63c1bb18d74bfa/648818d56d04c3bdf36d71ab_Refuel_rev8-01_ts-p-1600.png" alt="Refuel.ai" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
||||
</div>
|
||||
|
||||
## Model Details
|
||||
|
||||
RefuelLLM-2-small, aka Llama-3-Refueled, is a Llama3-8B base model instruction tuned on a corpus of 2750+ datasets, spanning tasks such as classification, reading comprehension, structured attribute extraction and entity resolution. We're excited to open-source the model for the community to build on top of.
|
||||
|
||||
* More details about [RefuelLLM-2 family of models](https://www.refuel.ai/blog-posts/announcing-refuel-llm-2)
|
||||
* You can also try out the models in our [LLM playground](https://labs.refuel.ai/playground)
|
||||
|
||||
**Model developers** - Refuel AI
|
||||
|
||||
**Input** - Text only.
|
||||
|
||||
**Output** - Text only.
|
||||
|
||||
**Architecture** - Llama-3-Refueled is built on top of Llama-3-8B-instruct which is an auto-regressive language model that uses an optimized transformer architecture.
|
||||
|
||||
**Release Date** - May 8, 2024.
|
||||
|
||||
**License** - [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/deed.en)
|
||||
|
||||
## How to use
|
||||
|
||||
This repository contains weights for Llama-3-Refueled that are compatible for use with HuggingFace. See the snippet below for usage with Transformers:
|
||||
|
||||
```python
|
||||
>>> import torch
|
||||
>>> from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
>>> model_id = "refuelai/Llama-3-Refueled"
|
||||
>>> tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
>>> model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
|
||||
|
||||
>>> messages = [{"role": "user", "content": "Is this comment toxic or non-toxic: RefuelLLM is the new way to label text data!"}]
|
||||
|
||||
>>> inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")
|
||||
|
||||
>>> outputs = model.generate(inputs, max_new_tokens=20)
|
||||
>>> print(tokenizer.decode(outputs[0]))
|
||||
```
|
||||
|
||||
## Training Data
|
||||
|
||||
The model was both trained on over 4 Billion tokens, spanning 2750+ NLP tasks. Our training collection consists majorly of:
|
||||
1. Human annotated datasets like Flan, Task Source, and the Aya collection
|
||||
2. Synthetic datasets like OpenOrca, OpenHermes and WizardLM
|
||||
3. Proprietary datasets developed or licensed by Refuel AI
|
||||
|
||||
## Benchmarks
|
||||
|
||||
In this section, we report the results for Refuel models on our benchmark of labeling tasks. For details on the methodology see [here](https://refuel.ai/blog-posts/announcing-refuel-llm-2).
|
||||
|
||||
<table>
|
||||
<tr></tr>
|
||||
<tr><th>Provider</th><th>Model</th><th colspan="4" style="text-align: center">LLM Output Quality (by task type)</tr>
|
||||
<tr><td></td><td></td><td>Overall</td><td>Classification</td><td>Reading Comprehension</td><td>Structure Extraction</td><td>Entity Matching</td><td></td></tr>
|
||||
<tr><td>Refuel</td><td>RefuelLLM-2</td><td>83.82%</td><td>84.94%</td><td>76.03%</td><td>88.16%</td><td>92.00%</td><td></td></tr>
|
||||
<tr><td>OpenAI</td><td>GPT-4-Turbo</td><td>80.88%</td><td>81.77%</td><td>72.08%</td><td>84.79%</td><td>97.20%</td><td></td></tr>
|
||||
<tr><td>Refuel</td><td>RefuelLLM-2-small (Llama-3-Refueled)</td><td>79.67%</td><td>81.72%</td><td>70.04%</td><td>84.28%</td><td>92.00%</td><td></td></tr>
|
||||
<tr><td>Anthropic</td><td>Claude-3-Opus</td><td>79.19%</td><td>82.49%</td><td>67.30%</td><td>88.25%</td><td>94.96%</td><td></td></tr>
|
||||
<tr><td>Meta</td><td>Llama3-70B-Instruct</td><td>78.20%</td><td>79.38%</td><td>66.03%</td><td>85.96%</td><td>94.13%</td><td></td></tr>
|
||||
<tr><td>Google</td><td>Gemini-1.5-Pro</td><td>74.59%</td><td>73.52%</td><td>60.67%</td><td>84.27%</td><td>98.48%</td><td></td></tr>
|
||||
<tr><td>Mistral</td><td>Mixtral-8x7B-Instruct</td><td>62.87%</td><td>79.11%</td><td>45.56%</td><td>47.08%</td><td>86.52%</td><td></td></tr>
|
||||
<tr><td>Anthropic</td><td>Claude-3-Sonnet</td><td>70.99%</td><td>79.91%</td><td>45.44%</td><td>78.10%</td><td>96.34%</td><td></td></tr>
|
||||
<tr><td>Anthropic</td><td>Claude-3-Haiku</td><td>69.23%</td><td>77.27%</td><td>50.19%</td><td>84.97%</td><td>54.08%</td><td></td></tr>
|
||||
<tr><td>OpenAI</td><td>GPT-3.5-Turbo</td><td>68.13%</td><td>74.39%</td><td>53.21%</td><td>69.40%</td><td>80.41%</td><td></td></tr>
|
||||
<tr><td>Meta</td><td>Llama3-8B-Instruct</td><td>62.30%</td><td>68.52%</td><td>49.16%</td><td>65.09%</td><td>63.61%</td><td></td></tr>
|
||||
</table>
|
||||
|
||||
|
||||
## Limitations
|
||||
|
||||
The Llama-3-Refueled does not have any moderation mechanisms. We're looking forward to engaging with the community
|
||||
on ways to make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
|
||||
Reference in New Issue
Block a user