160 lines
7.0 KiB
Markdown
160 lines
7.0 KiB
Markdown
---
|
|
license: other
|
|
license_name: seallms
|
|
license_link: https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/blob/main/LICENSE
|
|
extra_gated_prompt: >-
|
|
You agree to not use the models for any harmful, inappropriate, unethical or
|
|
illegal purpose or intention. You agree to perform your own red teaming and
|
|
provide related safety and security measures before deployment for any product
|
|
relevant to our models and demos, and you must abide by and comply with local
|
|
governance and regulations. In no event shall the models' authors be held
|
|
liable for any claim, damages, or other liability arising from the use of the
|
|
released weights, codes, or demos. The models and demos may be subject to
|
|
export controls or restrictions in the United States or other countries or
|
|
regions. You shall comply with applicable laws and regulations in your use of
|
|
the demos.
|
|
extra_gated_fields:
|
|
Company: text
|
|
Country: text
|
|
I agree to use this model for non-commercial use ONLY: checkbox
|
|
language:
|
|
- en
|
|
- vi
|
|
- id
|
|
- ms
|
|
- th
|
|
- km
|
|
- lo
|
|
- my
|
|
- tl
|
|
- zh
|
|
---
|
|
<p align="center">
|
|
<img src="seal_logo.png" width="200" />
|
|
</p>
|
|
|
|
# SeaLLMs - Large Language Models for Southeast Asia
|
|
|
|
|
|
<p align="center">
|
|
<a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a>
|
|
|
|
<a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 DEMO</a>
|
|
|
|
<a href="https://github.com/DAMO-NLP-SG/SeaLLMs" target="_blank" rel="noopener">Github</a>
|
|
|
|
<a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a>
|
|
</p>
|
|
|
|
## SeaLLM-chat-7B
|
|
|
|
This a **7B Chat** version of SeaLLMs. It Vietnamese 🇻🇳, Indonesian 🇮🇩, Thai 🇹🇭, Malay 🇲🇾, Khmer 🇰🇭, Lao 🇱🇦, Tagalog 🇵🇭 and Burmese 🇲🇲.
|
|
It may have lower capability than the 13B models but it is much more memory-efficient and faster.
|
|
|
|
Visit our <a href="https://arxiv.org/pdf/2312.00738.pdf" target="_blank" rel="noopener">Technical Report</a> and <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b" target="_blank" rel="noopener"> 🤗 Tech Memo</a> for more details.
|
|
|
|
<blockquote style="color:red">
|
|
<p><strong style="color: red">Terms of Use and License</strong>:
|
|
By using our released weights, codes, and demos, you agree to and comply with the terms and conditions specified in our <a href="https://huggingface.co/SeaLLMs/SeaLLM-Chat-13b/edit/main/LICENSE" target="_blank" rel="noopener">SeaLLMs Terms Of Use</a>.
|
|
</blockquote>
|
|
|
|
> **Disclaimer**:
|
|
> We must note that even though the weights, codes, and demos are released in an open manner, similar to other pre-trained language models, and despite our best efforts in red teaming and safety fine-tuning and enforcement, our models come with potential risks, including but not limited to inaccurate, misleading or potentially harmful generation.
|
|
> Developers and stakeholders should perform their own red teaming and provide related security measures before deployment, and they must abide by and comply with local governance and regulations.
|
|
> In no event shall the authors be held liable for any claim, damages, or other liability arising from the use of the released weights, codes, or demos.
|
|
|
|
> The logo was generated by DALL-E 3.
|
|
|
|
## 示例代码:
|
|
|
|
SeaLLM models work the same way as Llama-2, so the Llama-2 generation codebase should be sufficient to run.
|
|
However, as this is a chat model, you should wrap the prompt/instruction using the following format function.
|
|
|
|
You should also turn off add_special_tokens with `tokenizer.add_special_tokens = False`.
|
|
|
|
```python
|
|
from modelscope import AutoTokenizer,Model
|
|
from modelscope import snapshot_download
|
|
import torch
|
|
from typing import List, Tuple
|
|
|
|
BOS_TOKEN = '<s>'
|
|
EOS_TOKEN = '</s>'
|
|
|
|
B_INST, E_INST = "[INST]", "[/INST]"
|
|
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
|
|
|
|
SYSTEM_PROMPT = """You are a multilingual, helpful, respectful and honest assistant. \
|
|
Please always answer as helpfully as possible, while being safe. Your \
|
|
answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure \
|
|
that your responses are socially unbiased and positive in nature.
|
|
|
|
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not \
|
|
correct. If you don't know the answer to a question, please don't share false information.
|
|
|
|
As a multilingual assistant, you must respond and follow instructions in the native language of the user by default, unless told otherwise. \
|
|
Your response should adapt to the norms and customs of the respective language and culture.
|
|
"""
|
|
|
|
def chat_multiturn_seq_format(
|
|
message: str,
|
|
history: List[Tuple[str, str]] = [],
|
|
):
|
|
"""
|
|
```
|
|
<bos>[INST] B_SYS SytemPrompt E_SYS Prompt [/INST] Answer <eos>
|
|
<bos>[INST] Prompt [/INST] Answer <eos>
|
|
<bos>[INST] Prompt [/INST]
|
|
```
|
|
As the format auto-add <bos>, please turn off add_special_tokens with `tokenizer.add_special_tokens = False`
|
|
Inputs:
|
|
message: the current prompt
|
|
history: list of list indicating previous conversation. [[message1, response1], [message2, response2]]
|
|
Outputs:
|
|
full_prompt: the prompt that should go into the chat model
|
|
|
|
e.g:
|
|
full_prompt = chat_multiturn_seq_format("Hello world")
|
|
output = model.generate(tokenizer.encode(full_prompt, add_special_tokens=False), ...)
|
|
"""
|
|
text = ''
|
|
for i, (prompt, res) in enumerate(history):
|
|
if i == 0:
|
|
text += f"{BOS_TOKEN}{B_INST} {B_SYS} {SYSTEM_PROMPT} {E_SYS} {prompt} {E_INST}"
|
|
else:
|
|
text += f"{BOS_TOKEN}{B_INST} {prompt}{end_instr}"
|
|
if res is not None:
|
|
text += f" {res} {EOS_TOKEN} "
|
|
if len(history) == 0 or text.strip() == '':
|
|
text = f"{BOS_TOKEN}{B_INST} {B_SYS} {SYSTEM_PROMPT} {E_SYS} {message} {E_INST}"
|
|
else:
|
|
text += f"{BOS_TOKEN}{B_INST} {message} {E_INST}"
|
|
return text
|
|
|
|
local_dir = snapshot_download("AI-ModelScope/SeaLLM-7B-chat",revision='master')
|
|
|
|
model = Model.from_pretrained(local_dir, revision='master', device_map='auto', torch_dtype=torch.float16)
|
|
tokenizer = AutoTokenizer.from_pretrained(local_dir, revision='master')
|
|
|
|
full_prompt = chat_multiturn_seq_format("Hello world")
|
|
inputs = tokenizer(full_prompt, add_special_tokens=False, return_tensors="pt")
|
|
# Generate
|
|
generate_ids = model.generate(inputs.input_ids.to(model.device), max_length=512,do_sample=True,top_k=10,num_return_sequences=1)
|
|
print(tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0])
|
|
```
|
|
|
|
|
|
## Citation
|
|
|
|
If you find our project useful, we hope you would kindly star our repo and cite our work as follows: Corresponding Author: [l.bing@alibaba-inc.com](mailto:l.bing@alibaba-inc.com)
|
|
|
|
```
|
|
@article{damonlpsg2023seallm,
|
|
author = {Xuan-Phi Nguyen*, Wenxuan Zhang*, Xin Li*, Mahani Aljunied*,
|
|
Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang,
|
|
Chaoqun Liu, Hang Zhang, Lidong Bing},
|
|
title = {SeaLLMs - Large Language Models for Southeast Asia},
|
|
year = 2023,
|
|
Eprint = {arXiv:2312.00738},
|
|
}
|
|
``` |