初始化项目，由ModelHub XC社区提供模型
Model: AI-ModelScope/SeaLLM-7B-chat Source: Original Platform
This commit is contained in:
ModelHub XC
160
README.md
Normal file
160
README.md
Normal file
@@ -0,0 +1,160 @@
|
||||
---
|
||||
license: other
|
||||
license_name: seallms
|
||||
license_link: https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/blob/main/LICENSE
|
||||
extra_gated_prompt: >-
|
||||
You agree to not use the models for any harmful, inappropriate, unethical or
|
||||
illegal purpose or intention. You agree to perform your own red teaming and
|
||||
provide related safety and security measures before deployment for any product
|
||||
relevant to our models and demos, and you must abide by and comply with local
|
||||
governance and regulations. In no event shall the models' authors be held
|
||||
liable for any claim, damages, or other liability arising from the use of the
|
||||
released weights, codes, or demos. The models and demos may be subject to
|
||||
export controls or restrictions in the United States or other countries or
|
||||
regions. You shall comply with applicable laws and regulations in your use of
|
||||
the demos.
|
||||
extra_gated_fields:
|
||||
Company: text
|
||||
Country: text
|
||||
I agree to use this model for non-commercial use ONLY: checkbox
|
||||
language:
|
||||
- en
|
||||
- vi
|
||||
- id
|
||||
- ms
|
||||
- th
|
||||
- km
|
||||
- lo
|
||||
- my
|
||||
- tl
|
||||
- zh
|
||||
---
|
||||
<p align="center">
|
||||
<img src="seal_logo.png" width="200" />
|
||||
</p>
|
||||
|
||||
# SeaLLMs - Large Language Models for Southeast Asia
|
||||
|
||||
|
||||
<p align="center">
|
||||
<a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a>
|
||||
|
||||
<a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 DEMO</a>
|
||||
|
||||
<a href="https://github.com/DAMO-NLP-SG/SeaLLMs" target="_blank" rel="noopener">Github</a>
|
||||
|
||||
<a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a>
|
||||
</p>
|
||||
|
||||
## SeaLLM-chat-7B
|
||||
|
||||
This a **7B Chat** version of SeaLLMs. It Vietnamese 🇻🇳, Indonesian 🇮🇩, Thai 🇹🇭, Malay 🇲🇾, Khmer 🇰🇭, Lao 🇱🇦, Tagalog 🇵🇭 and Burmese 🇲🇲.
|
||||
It may have lower capability than the 13B models but it is much more memory-efficient and faster.
|
||||
|
||||
Visit our <a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a> and <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a> for more details.
|
||||
|
||||
<blockquote style="color:red">
|
||||
<p><strong style="color: red">Terms of Use and License</strong>:
|
||||
By using our released weights, codes, and demos, you agree to and comply with the terms and conditions specified in our <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/edit/main/LICENSE" target="_blank" rel="noopener">SeaLLMs Terms Of Use</a>.
|
||||
</blockquote>
|
||||
|
||||
> **Disclaimer**:
|
||||
> We must note that even though the weights, codes, and demos are released in an open manner, similar to other pre-trained language models, and despite our best efforts in red teaming and safety fine-tuning and enforcement, our models come with potential risks, including but not limited to inaccurate, misleading or potentially harmful generation.
|
||||
> Developers and stakeholders should perform their own red teaming and provide related security measures before deployment, and they must abide by and comply with local governance and regulations.
|
||||
> In no event shall the authors be held liable for any claim, damages, or other liability arising from the use of the released weights, codes, or demos.
|
||||
|
||||
> The logo was generated by DALL-E 3.
|
||||
|
||||
## 示例代码:
|
||||
|
||||
SeaLLM models work the same way as Llama-2, so the Llama-2 generation codebase should be sufficient to run.
|
||||
However, as this is a chat model, you should wrap the prompt/instruction using the following format function.
|
||||
|
||||
You should also turn off add_special_tokens with `tokenizer.add_special_tokens = False`.
|
||||
|
||||
```python
|
||||
from modelscope import AutoTokenizer,Model
|
||||
from modelscope import snapshot_download
|
||||
import torch
|
||||
from typing import List, Tuple
|
||||
|
||||
BOS_TOKEN = '<s>'
|
||||
EOS_TOKEN = '</s>'
|
||||
|
||||
B_INST, E_INST = "[INST]", "[/INST]"
|
||||
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
|
||||
|
||||
SYSTEM_PROMPT = """You are a multilingual, helpful, respectful and honest assistant. \
|
||||
Please always answer as helpfully as possible, while being safe. Your \
|
||||
answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure \
|
||||
that your responses are socially unbiased and positive in nature.
|
||||
|
||||
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not \
|
||||
correct. If you don't know the answer to a question, please don't share false information.
|
||||
|
||||
As a multilingual assistant, you must respond and follow instructions in the native language of the user by default, unless told otherwise. \
|
||||
Your response should adapt to the norms and customs of the respective language and culture.
|
||||
"""
|
||||
|
||||
def chat_multiturn_seq_format(
|
||||
message: str,
|
||||
history: List[Tuple[str, str]] = [],
|
||||
):
|
||||
"""
|
||||
```
|
||||
<bos>[INST] B_SYS SytemPrompt E_SYS Prompt [/INST] Answer <eos>
|
||||
<bos>[INST] Prompt [/INST] Answer <eos>
|
||||
<bos>[INST] Prompt [/INST]
|
||||
```
|
||||
As the format auto-add <bos>, please turn off add_special_tokens with `tokenizer.add_special_tokens = False`
|
||||
Inputs:
|
||||
message: the current prompt
|
||||
history: list of list indicating previous conversation. [[message1, response1], [message2, response2]]
|
||||
Outputs:
|
||||
full_prompt: the prompt that should go into the chat model
|
||||
|
||||
e.g:
|
||||
full_prompt = chat_multiturn_seq_format("Hello world")
|
||||
output = model.generate(tokenizer.encode(full_prompt, add_special_tokens=False), ...)
|
||||
"""
|
||||
text = ''
|
||||
for i, (prompt, res) in enumerate(history):
|
||||
if i == 0:
|
||||
text += f"{BOS_TOKEN}{B_INST} {B_SYS} {SYSTEM_PROMPT} {E_SYS} {prompt} {E_INST}"
|
||||
else:
|
||||
text += f"{BOS_TOKEN}{B_INST} {prompt}{end_instr}"
|
||||
if res is not None:
|
||||
text += f" {res} {EOS_TOKEN} "
|
||||
if len(history) == 0 or text.strip() == '':
|
||||
text = f"{BOS_TOKEN}{B_INST} {B_SYS} {SYSTEM_PROMPT} {E_SYS} {message} {E_INST}"
|
||||
else:
|
||||
text += f"{BOS_TOKEN}{B_INST} {message} {E_INST}"
|
||||
return text
|
||||
|
||||
local_dir = snapshot_download("AI-ModelScope/SeaLLM-7B-chat",revision='master')
|
||||
|
||||
model = Model.from_pretrained(local_dir, revision='master', device_map='auto', torch_dtype=torch.float16)
|
||||
tokenizer = AutoTokenizer.from_pretrained(local_dir, revision='master')
|
||||
|
||||
full_prompt = chat_multiturn_seq_format("Hello world")
|
||||
inputs = tokenizer(full_prompt, add_special_tokens=False, return_tensors="pt")
|
||||
# Generate
|
||||
generate_ids = model.generate(inputs.input_ids.to(model.device), max_length=512,do_sample=True,top_k=10,num_return_sequences=1)
|
||||
print(tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0])
|
||||
```
|
||||
|
||||
|
||||
## Citation
|
||||
|
||||
If you find our project useful, we hope you would kindly star our repo and cite our work as follows: Corresponding Author: [l.bing@alibaba-inc.com](mailto:l.bing@alibaba-inc.com)
|
||||
|
||||
```
|
||||
@article{damonlpsg2023seallm,
|
||||
author = {Xuan-Phi Nguyen*, Wenxuan Zhang*, Xin Li*, Mahani Aljunied*,
|
||||
Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang,
|
||||
Chaoqun Liu, Hang Zhang, Lidong Bing},
|
||||
title = {SeaLLMs - Large Language Models for Southeast Asia},
|
||||
year = 2023,
|
||||
Eprint = {arXiv:2312.00738},
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user