298 lines
8.4 KiB
Markdown
298 lines
8.4 KiB
Markdown
---
|
||
license: apache-2.0
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
model-index:
|
||
- name: Rubra-Qwen2-7B-Instruct
|
||
results:
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
type: MMLU
|
||
name: MMLU
|
||
metrics:
|
||
- type: 5-shot
|
||
value: 68.88
|
||
verified: false
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
type: GPQA
|
||
name: GPQA
|
||
metrics:
|
||
- type: 0-shot
|
||
value: 30.36
|
||
verified: false
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
type: GSM-8K
|
||
name: GSM-8K
|
||
metrics:
|
||
- type: 8-shot, CoT
|
||
value: 75.82
|
||
verified: false
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
type: MATH
|
||
name: MATH
|
||
metrics:
|
||
- type: 4-shot, CoT
|
||
value: 28.72
|
||
verified: false
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
type: MT-bench
|
||
name: MT-bench
|
||
metrics:
|
||
- type: GPT-4 as Judge
|
||
value: 8.08
|
||
verified: false
|
||
tags:
|
||
- function-calling
|
||
- tool-calling
|
||
- agentic
|
||
- rubra
|
||
- conversational
|
||
- 对话
|
||
- 工具调用
|
||
- 量化
|
||
language:
|
||
- en
|
||
- zh
|
||
---
|
||
|
||
# Qwen2 7B Instruct
|
||
|
||
## Model description
|
||
本模型是在进一步微调 [Qwen/Qwen2-7B-Instruct](https://www.modelscope.cn/models/qwen/Qwen2-7B-Instruct) 的结果. 它能够进行多轮复杂的工具调用。
|
||
|
||
## Training
|
||
|
||
本模型在包含多样化工具的调用、聊天数据集和Instruct数据的专有数据集上进行了进一步post-training（freeze tuned和DPO）。
|
||
|
||
## 使用方法
|
||
|
||
您可以使用Hugging Face的`transformers`和rubra库[rubra-tools](https://github.com/rubra-ai/rubra-tools)来使用该模型，如下所示：
|
||
|
||
```
|
||
pip install rubra_tools torch==2.3.0 transformers accelerate modelscope
|
||
```
|
||
|
||
您还需要安装Node.js和npm。安装后，请安装`jsonrepair`包 - 用于修复模型的一些罕见错误。
|
||
|
||
```
|
||
npm install jsonrepair
|
||
```
|
||
|
||
### 1. 加载模型
|
||
```python
|
||
from modelscope import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
from rubra_tools import preprocess_input, postprocess_output
|
||
|
||
model_id = "rubraAI/Meta-Llama-3-8B-Instruct"
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id,
|
||
torch_dtype="auto",
|
||
device_map="auto",
|
||
)
|
||
```
|
||
|
||
### 2. 定义函数
|
||
这里我们使用4个函数来解决一个简单的数学问题：
|
||
```python
|
||
functions = [
|
||
{
|
||
'type': 'function',
|
||
'function': {
|
||
'name': 'addition',
|
||
'description': "Adds two numbers together",
|
||
'parameters': {
|
||
'type': 'object',
|
||
'properties': {
|
||
'a': {
|
||
'description': 'First number to add',
|
||
'type': 'string'
|
||
},
|
||
'b': {
|
||
'description': 'Second number to add',
|
||
'type': 'string'
|
||
}
|
||
},
|
||
'required': []
|
||
}
|
||
}
|
||
},
|
||
{
|
||
'type': 'function',
|
||
'function': {
|
||
'name': 'subtraction',
|
||
'description': "Subtracts two numbers",
|
||
'parameters': {
|
||
'type': 'object',
|
||
'properties': {
|
||
'a': {
|
||
'description': 'First number to be subtracted from',
|
||
'type': 'string'
|
||
},
|
||
'b': {
|
||
'description': 'Number to subtract',
|
||
'type': 'string'
|
||
}
|
||
},
|
||
'required': []
|
||
}
|
||
}
|
||
},
|
||
{
|
||
'type': 'function',
|
||
'function': {
|
||
'name': 'multiplication',
|
||
'description': "Multiply two numbers together",
|
||
'parameters': {
|
||
'type': 'object',
|
||
'properties': {
|
||
'a': {
|
||
'description': 'First number to multiply',
|
||
'type': 'string'
|
||
},
|
||
'b': {
|
||
'description': 'Second number to multiply',
|
||
'type': 'string'
|
||
}
|
||
},
|
||
'required': []
|
||
}
|
||
}
|
||
},
|
||
{
|
||
'type': 'function',
|
||
'function': {
|
||
'name': 'division',
|
||
'description': "Divide two numbers",
|
||
'parameters': {
|
||
'type': 'object',
|
||
'properties': {
|
||
'a': {
|
||
'description': 'First number to use as the dividend',
|
||
'type': 'string'
|
||
},
|
||
'b': {
|
||
'description': 'Second number to use as the divisor',
|
||
'type': 'string'
|
||
}
|
||
},
|
||
'required': []
|
||
}
|
||
}
|
||
},
|
||
]
|
||
```
|
||
|
||
### 3. 开始对话
|
||
```python
|
||
messages = [
|
||
{"role": "system", "content": "You are a helpful assistant."},
|
||
{"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"},
|
||
]
|
||
|
||
def run_model(messages, functions):
|
||
## Format messages in Rubra's format
|
||
formatted_msgs = preprocess_input(msgs=messages, tools=functions)
|
||
|
||
text = tokenizer.apply_chat_template(
|
||
formatted_msgs,
|
||
tokenize=False,
|
||
add_generation_prompt=True
|
||
)
|
||
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||
|
||
generated_ids = model.generate(
|
||
model_inputs.input_ids,
|
||
max_new_tokens=512
|
||
)
|
||
generated_ids = [
|
||
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
||
]
|
||
|
||
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
||
return response
|
||
|
||
raw_output = run_model(messages, functions)
|
||
# Check if there's a function call
|
||
function_call = postprocess_output(raw_output)
|
||
if function_call:
|
||
print(function_call)
|
||
else:
|
||
print(raw_output)
|
||
```
|
||
|
||
您应该会看到以下输出，这是AI助手进行的工具调用：
|
||
```
|
||
[{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}]
|
||
```
|
||
|
||
### 4. 将执行工具调用的结果添加到消息历史记录并继续对话
|
||
|
||
```python
|
||
if function_call:
|
||
# append the assistant tool call msg
|
||
messages.append({"role": "assistant", "tool_calls": function_call})
|
||
# append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10.
|
||
messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'})
|
||
raw_output1 = run_model(messages, functions)
|
||
# Check if there's a function call
|
||
|
||
function_call = postprocess_output(raw_output1)
|
||
if function_call:
|
||
print(function_call)
|
||
else:
|
||
print(raw_output)
|
||
```
|
||
|
||
LLM将再次进行调用
|
||
```
|
||
[{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}]
|
||
```
|
||
|
||
## 框架版本
|
||
- Transformers 4.41.2
|
||
- Pytorch 2.3.1+cu121
|
||
- Datasets 2.19.2
|
||
- Tokenizers 0.19.1
|
||
|
||
## 限制和偏见
|
||
|
||
虽然该模型在广泛的任务上表现良好，但它可能仍然会产生偏见或错误的输出。用户在使用该模型时应谨慎，并在敏感或高风险应用中保持批判性的判断。模型的输出受其训练数据的影响，可能包含固有的偏见。
|
||
|
||
## 道德考虑
|
||
|
||
用户应确保该模型的部署符合道德指南，并考虑所生成文本的潜在社会影响。我们强烈反对将该模型滥用于生成有害或误导性内容。
|
||
|
||
## 致谢
|
||
|
||
我们感谢Alibaba Cloud提供的基础模型。
|
||
|
||
## 联系方式
|
||
|
||
如果对此模型有任何问题或意见或建议，请联系 [rubra团队](mailto:rubra@acorn.io)。
|
||
|
||
## 引用信息
|
||
|
||
如果您使用了这项工作，请引用如下：
|
||
|
||
```
|
||
@misc {rubra_ai_2024,
|
||
author = { Sanjay Nadhavajhala and Yingbei Tong },
|
||
title = { Rubra-Qwen2-7B-Instruct },
|
||
year = 2024,
|
||
url = { https://huggingface.co/rubra-ai/Qwen2-7B-Instruct },
|
||
doi = { 10.57967/hf/2658 },
|
||
publisher = { Hugging Face }
|
||
}
|
||
``` |