初始化项目，由ModelHub XC社区提供模型
Model: rubraAI/Qwen2-7B-Instruct Source: Original Platform
This commit is contained in:
ModelHub XC
298
README.md
Normal file
298
README.md
Normal file
@@ -0,0 +1,298 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
model-index:
|
||||
- name: Rubra-Qwen2-7B-Instruct
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
dataset:
|
||||
type: MMLU
|
||||
name: MMLU
|
||||
metrics:
|
||||
- type: 5-shot
|
||||
value: 68.88
|
||||
verified: false
|
||||
- task:
|
||||
type: text-generation
|
||||
dataset:
|
||||
type: GPQA
|
||||
name: GPQA
|
||||
metrics:
|
||||
- type: 0-shot
|
||||
value: 30.36
|
||||
verified: false
|
||||
- task:
|
||||
type: text-generation
|
||||
dataset:
|
||||
type: GSM-8K
|
||||
name: GSM-8K
|
||||
metrics:
|
||||
- type: 8-shot, CoT
|
||||
value: 75.82
|
||||
verified: false
|
||||
- task:
|
||||
type: text-generation
|
||||
dataset:
|
||||
type: MATH
|
||||
name: MATH
|
||||
metrics:
|
||||
- type: 4-shot, CoT
|
||||
value: 28.72
|
||||
verified: false
|
||||
- task:
|
||||
type: text-generation
|
||||
dataset:
|
||||
type: MT-bench
|
||||
name: MT-bench
|
||||
metrics:
|
||||
- type: GPT-4 as Judge
|
||||
value: 8.08
|
||||
verified: false
|
||||
tags:
|
||||
- function-calling
|
||||
- tool-calling
|
||||
- agentic
|
||||
- rubra
|
||||
- conversational
|
||||
- 对话
|
||||
- 工具调用
|
||||
- 量化
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
---
|
||||
|
||||
# Qwen2 7B Instruct
|
||||
|
||||
## Model description
|
||||
本模型是在进一步微调 [Qwen/Qwen2-7B-Instruct](https://www.modelscope.cn/models/qwen/Qwen2-7B-Instruct) 的结果. 它能够进行多轮复杂的工具调用。
|
||||
|
||||
## Training
|
||||
|
||||
本模型在包含多样化工具的调用、聊天数据集和Instruct数据的专有数据集上进行了进一步post-training（freeze tuned和DPO）。
|
||||
|
||||
## 使用方法
|
||||
|
||||
您可以使用Hugging Face的`transformers`和rubra库[rubra-tools](https://github.com/rubra-ai/rubra-tools)来使用该模型，如下所示：
|
||||
|
||||
```
|
||||
pip install rubra_tools torch==2.3.0 transformers accelerate modelscope
|
||||
```
|
||||
|
||||
您还需要安装Node.js和npm。安装后，请安装`jsonrepair`包 - 用于修复模型的一些罕见错误。
|
||||
|
||||
```
|
||||
npm install jsonrepair
|
||||
```
|
||||
|
||||
### 1. 加载模型
|
||||
```python
|
||||
from modelscope import AutoModelForCausalLM, AutoTokenizer
|
||||
import torch
|
||||
from rubra_tools import preprocess_input, postprocess_output
|
||||
|
||||
model_id = "rubraAI/Meta-Llama-3-8B-Instruct"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
torch_dtype="auto",
|
||||
device_map="auto",
|
||||
)
|
||||
```
|
||||
|
||||
### 2. 定义函数
|
||||
这里我们使用4个函数来解决一个简单的数学问题：
|
||||
```python
|
||||
functions = [
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': 'addition',
|
||||
'description': "Adds two numbers together",
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'a': {
|
||||
'description': 'First number to add',
|
||||
'type': 'string'
|
||||
},
|
||||
'b': {
|
||||
'description': 'Second number to add',
|
||||
'type': 'string'
|
||||
}
|
||||
},
|
||||
'required': []
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': 'subtraction',
|
||||
'description': "Subtracts two numbers",
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'a': {
|
||||
'description': 'First number to be subtracted from',
|
||||
'type': 'string'
|
||||
},
|
||||
'b': {
|
||||
'description': 'Number to subtract',
|
||||
'type': 'string'
|
||||
}
|
||||
},
|
||||
'required': []
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': 'multiplication',
|
||||
'description': "Multiply two numbers together",
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'a': {
|
||||
'description': 'First number to multiply',
|
||||
'type': 'string'
|
||||
},
|
||||
'b': {
|
||||
'description': 'Second number to multiply',
|
||||
'type': 'string'
|
||||
}
|
||||
},
|
||||
'required': []
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': 'division',
|
||||
'description': "Divide two numbers",
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'a': {
|
||||
'description': 'First number to use as the dividend',
|
||||
'type': 'string'
|
||||
},
|
||||
'b': {
|
||||
'description': 'Second number to use as the divisor',
|
||||
'type': 'string'
|
||||
}
|
||||
},
|
||||
'required': []
|
||||
}
|
||||
}
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
### 3. 开始对话
|
||||
```python
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a helpful assistant."},
|
||||
{"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"},
|
||||
]
|
||||
|
||||
def run_model(messages, functions):
|
||||
## Format messages in Rubra's format
|
||||
formatted_msgs = preprocess_input(msgs=messages, tools=functions)
|
||||
|
||||
text = tokenizer.apply_chat_template(
|
||||
formatted_msgs,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True
|
||||
)
|
||||
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||||
|
||||
generated_ids = model.generate(
|
||||
model_inputs.input_ids,
|
||||
max_new_tokens=512
|
||||
)
|
||||
generated_ids = [
|
||||
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
||||
]
|
||||
|
||||
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
||||
return response
|
||||
|
||||
raw_output = run_model(messages, functions)
|
||||
# Check if there's a function call
|
||||
function_call = postprocess_output(raw_output)
|
||||
if function_call:
|
||||
print(function_call)
|
||||
else:
|
||||
print(raw_output)
|
||||
```
|
||||
|
||||
您应该会看到以下输出，这是AI助手进行的工具调用：
|
||||
```
|
||||
[{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}]
|
||||
```
|
||||
|
||||
### 4. 将执行工具调用的结果添加到消息历史记录并继续对话
|
||||
|
||||
```python
|
||||
if function_call:
|
||||
# append the assistant tool call msg
|
||||
messages.append({"role": "assistant", "tool_calls": function_call})
|
||||
# append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10.
|
||||
messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'})
|
||||
raw_output1 = run_model(messages, functions)
|
||||
# Check if there's a function call
|
||||
|
||||
function_call = postprocess_output(raw_output1)
|
||||
if function_call:
|
||||
print(function_call)
|
||||
else:
|
||||
print(raw_output)
|
||||
```
|
||||
|
||||
LLM将再次进行调用
|
||||
```
|
||||
[{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}]
|
||||
```
|
||||
|
||||
## 框架版本
|
||||
- Transformers 4.41.2
|
||||
- Pytorch 2.3.1+cu121
|
||||
- Datasets 2.19.2
|
||||
- Tokenizers 0.19.1
|
||||
|
||||
## 限制和偏见
|
||||
|
||||
虽然该模型在广泛的任务上表现良好，但它可能仍然会产生偏见或错误的输出。用户在使用该模型时应谨慎，并在敏感或高风险应用中保持批判性的判断。模型的输出受其训练数据的影响，可能包含固有的偏见。
|
||||
|
||||
## 道德考虑
|
||||
|
||||
用户应确保该模型的部署符合道德指南，并考虑所生成文本的潜在社会影响。我们强烈反对将该模型滥用于生成有害或误导性内容。
|
||||
|
||||
## 致谢
|
||||
|
||||
我们感谢Alibaba Cloud提供的基础模型。
|
||||
|
||||
## 联系方式
|
||||
|
||||
如果对此模型有任何问题或意见或建议，请联系 [rubra团队](mailto:rubra@acorn.io)。
|
||||
|
||||
## 引用信息
|
||||
|
||||
如果您使用了这项工作，请引用如下：
|
||||
|
||||
```
|
||||
@misc {rubra_ai_2024,
|
||||
author = { Sanjay Nadhavajhala and Yingbei Tong },
|
||||
title = { Rubra-Qwen2-7B-Instruct },
|
||||
year = 2024,
|
||||
url = { https://huggingface.co/rubra-ai/Qwen2-7B-Instruct },
|
||||
doi = { 10.57967/hf/2658 },
|
||||
publisher = { Hugging Face }
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user