109 lines
4.5 KiB
Markdown
109 lines
4.5 KiB
Markdown
|
---
|
|||
|
|
tasks:
|
|||
|
|
- text-generation
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
|
|||
|
|
license: Apache License 2.0
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# ✨ Falcon-7B-Instruct
|
|||
|
|
|
|||
|
|
**Falcon-7B-Instruct 是 [TII](https://www.tii.ae) 在 [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b)的基础上建立的7B参数因果解码器专用模型，并在chat/instruct数据集的混合中进行了微调。它是在Apache 2.0许可下提供的。**
|
|||
|
|
|
|||
|
|
*Paper coming soon 😊.*
|
|||
|
|
|
|||
|
|
## 为什么使用Falcon-7B-Instruct?
|
|||
|
|
|
|||
|
|
* **您正在寻找一个基于[Falcon-7B](https://huggingface.co/tiiuae/falcon-7b)的即用型chat/instruct模型.**
|
|||
|
|
* **猎鹰-7B是一个强大的基础模型，性能优于可比的开源模型** (e.g., [MPT-7B](https://huggingface.co/mosaicml/mpt-7b), [StableLM](https://github.com/Stability-AI/StableLM), [RedPajama](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-7B-v0.1) etc.), 得益于在1,500B tokens 的 [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) 通过精心策划的语料库来加强。See the [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
|
|||
|
|
* **它有一个为推理而优化的架构**, with FlashAttention ([Dao et al., 2022](https://arxiv.org/abs/2205.14135)) and multiquery ([Shazeer et al., 2019](https://arxiv.org/abs/1911.02150)).
|
|||
|
|
|
|||
|
|
💬 **这是一个指导性的模型，对于进一步的微调可能并不理想。** 如果你有兴趣建立你自己的指示/聊天模型，我们建议从[猎鹰-7B](https://huggingface.co/tiiuae/falcon-7b)开始.
|
|||
|
|
|
|||
|
|
🔥 **想找一个更强大的模型吗？** [Falcon-40B-Instruct](https://huggingface.co/tiiuae/falcon-40b-instruct) 是Falcon-7B-Instruct的大哥!
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from modelscope.utils.constant import Tasks
|
|||
|
|
from modelscope.pipelines import pipeline
|
|||
|
|
pipe = pipeline(task=Tasks.text_generation, model='AI-ModelScope/falcon-7b-instruct', model_revision='v1.0.1', device='cuda')
|
|||
|
|
query="Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:"
|
|||
|
|
result = pipe(query)
|
|||
|
|
print(result)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
💥 **Falcon LLMs require PyTorch 2.0 for use with `transformers`!**
|
|||
|
|
|
|||
|
|
|
|||
|
|
# Model Card for Falcon-7B-Instruct
|
|||
|
|
|
|||
|
|
## 模型细节
|
|||
|
|
|
|||
|
|
### 模型描述
|
|||
|
|
|
|||
|
|
- **开发者/单位:** [https://www.tii.ae](https://www.tii.ae);
|
|||
|
|
- **模型类型:** Causal decoder-only;
|
|||
|
|
- **语言(NLP):** English and French;
|
|||
|
|
- **许可证:** Apache 2.0;
|
|||
|
|
- **根据模型进行微调:** [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b).
|
|||
|
|
|
|||
|
|
### 模型来源
|
|||
|
|
|
|||
|
|
- **Paper:** *coming soon*.
|
|||
|
|
|
|||
|
|
## 用途
|
|||
|
|
|
|||
|
|
### 直接使用
|
|||
|
|
|
|||
|
|
猎鹰-7B-Instruct已经在指示和聊天数据集的混合中进行了微调。
|
|||
|
|
|
|||
|
|
### 范围外的使用
|
|||
|
|
|
|||
|
|
在没有充分评估风险和缓解措施的情况下进行生产使用；任何可能被认为是不负责任或有害的使用情况。
|
|||
|
|
|
|||
|
|
## 偏见、风险和局限性
|
|||
|
|
|
|||
|
|
Falcon-7B-Instruct主要是在英语数据上训练的，不会适当地推广到其他语言。此外，由于它是在代表网络的大规模语料库上训练的，它将带有网上常见的定型观念和偏见。
|
|||
|
|
|
|||
|
|
### 建议
|
|||
|
|
|
|||
|
|
我们建议Falcon-7B-Instruct的用户制定护栏，并对任何生产使用采取适当的预防措施。
|
|||
|
|
|
|||
|
|
|
|||
|
|
## 训练细节
|
|||
|
|
|
|||
|
|
### 训练数据
|
|||
|
|
|
|||
|
|
Falcon-7B-Instruct在250M tokens混合的指示/聊天数据集上进行了微调。
|
|||
|
|
|
|||
|
|
| **Data source** | **Fraction** | **Tokens** | **Description** |
|
|||
|
|
|--------------------|--------------|------------|-----------------------------------|
|
|||
|
|
| [Bai ze](https://github.com/project-baize/baize-chatbot) | 65% | 164M | chat |
|
|||
|
|
| [GPT4All](https://github.com/nomic-ai/gpt4all) | 25% | 62M | instruct |
|
|||
|
|
| [GPTeacher](https://github.com/teknium1/GPTeacher) | 5% | 11M | instruct |
|
|||
|
|
| [RefinedWeb-English](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) | 5% | 13M | massive web crawl |
|
|||
|
|
|
|||
|
|
|
|||
|
|
The data was tokenized with the Falcon-[7B](https://huggingface.co/tiiuae/falcon-7b)/[40B](https://huggingface.co/tiiuae/falcon-40b) tokenizer.
|
|||
|
|
|
|||
|
|
|
|||
|
|
## 评价
|
|||
|
|
|
|||
|
|
*Paper coming soon.*
|
|||
|
|
|
|||
|
|
See the [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) for early results.
|
|||
|
|
|
|||
|
|
请注意，这个模型变体没有针对NLP基准进行优化。
|
|||
|
|
|
|||
|
|
|
|||
|
|
## 技术参数
|
|||
|
|
|
|||
|
|
有关预训练的更多信息, 请见[Falcon-7B](https://huggingface.co/tiiuae/falcon-7b).
|
|||
|
|
|
|||
|
|
## 许可证
|
|||
|
|
|
|||
|
|
Falcon-7B-Instruct is made available under the Apache 2.0 license.
|
|||
|
|
|
|||
|
|
## 联系
|
|||
|
|
falconllm@tii.ae
|