--- license: apache-2.0 library_name: transformers pipeline_tag: text-generation model-index: - name: Rubra-Qwen2-7B-Instruct results: - task: type: text-generation dataset: type: MMLU name: MMLU metrics: - type: 5-shot value: 68.88 verified: false - task: type: text-generation dataset: type: GPQA name: GPQA metrics: - type: 0-shot value: 30.36 verified: false - task: type: text-generation dataset: type: GSM-8K name: GSM-8K metrics: - type: 8-shot, CoT value: 75.82 verified: false - task: type: text-generation dataset: type: MATH name: MATH metrics: - type: 4-shot, CoT value: 28.72 verified: false - task: type: text-generation dataset: type: MT-bench name: MT-bench metrics: - type: GPT-4 as Judge value: 8.08 verified: false tags: - function-calling - tool-calling - agentic - rubra - conversational - 对话 - 工具调用 - 量化 language: - en - zh --- # Qwen2 7B Instruct ## Model description 本模型是在进一步微调 [Qwen/Qwen2-7B-Instruct](https://www.modelscope.cn/models/qwen/Qwen2-7B-Instruct) 的结果. 它能够进行多轮复杂的工具调用。 ## Training 本模型在包含多样化工具的调用、聊天数据集和Instruct数据的专有数据集上进行了进一步post-training（freeze tuned和DPO）。 ## 使用方法 您可以使用Hugging Face的`transformers`和rubra库[rubra-tools](https://github.com/rubra-ai/rubra-tools)来使用该模型，如下所示： ``` pip install rubra_tools torch==2.3.0 transformers accelerate modelscope ``` 您还需要安装Node.js和npm。安装后，请安装`jsonrepair`包 - 用于修复模型的一些罕见错误。 ``` npm install jsonrepair ``` ### 1. 加载模型 ```python from modelscope import AutoModelForCausalLM, AutoTokenizer import torch from rubra_tools import preprocess_input, postprocess_output model_id = "rubraAI/Meta-Llama-3-8B-Instruct" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype="auto", device_map="auto", ) ``` ### 2. 定义函数 这里我们使用4个函数来解决一个简单的数学问题： ```python functions = [ { 'type': 'function', 'function': { 'name': 'addition', 'description': "Adds two numbers together", 'parameters': { 'type': 'object', 'properties': { 'a': { 'description': 'First number to add', 'type': 'string' }, 'b': { 'description': 'Second number to add', 'type': 'string' } }, 'required': [] } } }, { 'type': 'function', 'function': { 'name': 'subtraction', 'description': "Subtracts two numbers", 'parameters': { 'type': 'object', 'properties': { 'a': { 'description': 'First number to be subtracted from', 'type': 'string' }, 'b': { 'description': 'Number to subtract', 'type': 'string' } }, 'required': [] } } }, { 'type': 'function', 'function': { 'name': 'multiplication', 'description': "Multiply two numbers together", 'parameters': { 'type': 'object', 'properties': { 'a': { 'description': 'First number to multiply', 'type': 'string' }, 'b': { 'description': 'Second number to multiply', 'type': 'string' } }, 'required': [] } } }, { 'type': 'function', 'function': { 'name': 'division', 'description': "Divide two numbers", 'parameters': { 'type': 'object', 'properties': { 'a': { 'description': 'First number to use as the dividend', 'type': 'string' }, 'b': { 'description': 'Second number to use as the divisor', 'type': 'string' } }, 'required': [] } } }, ] ``` ### 3. 开始对话 ```python messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the result of four plus six? Take the result and add 2? Then multiply by 5 and then divide by two"}, ] def run_model(messages, functions): ## Format messages in Rubra's format formatted_msgs = preprocess_input(msgs=messages, tools=functions) text = tokenizer.apply_chat_template( formatted_msgs, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( model_inputs.input_ids, max_new_tokens=512 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ] response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] return response raw_output = run_model(messages, functions) # Check if there's a function call function_call = postprocess_output(raw_output) if function_call: print(function_call) else: print(raw_output) ``` 您应该会看到以下输出，这是AI助手进行的工具调用： ``` [{'id': 'fc65a533', 'function': {'name': 'addition', 'arguments': '{"a": "4", "b": "6"}'}, 'type': 'function'}] ``` ### 4. 将执行工具调用的结果添加到消息历史记录并继续对话 ```python if function_call: # append the assistant tool call msg messages.append({"role": "assistant", "tool_calls": function_call}) # append the result of the tool call in openai format, in this case, the value of add 6 to 4 is 10. messages.append({'role': 'tool', 'tool_call_id': function_call[0]["id"], 'name': function_call[0]["function"]["name"], 'content': '10'}) raw_output1 = run_model(messages, functions) # Check if there's a function call function_call = postprocess_output(raw_output1) if function_call: print(function_call) else: print(raw_output) ``` LLM将再次进行调用 ``` [{'id': '2ffc3de4', 'function': {'name': 'addition', 'arguments': '{"a": "10", "b": "2"}'}, 'type': 'function'}] ``` ## 框架版本 - Transformers 4.41.2 - Pytorch 2.3.1+cu121 - Datasets 2.19.2 - Tokenizers 0.19.1 ## 限制和偏见 虽然该模型在广泛的任务上表现良好，但它可能仍然会产生偏见或错误的输出。用户在使用该模型时应谨慎，并在敏感或高风险应用中保持批判性的判断。模型的输出受其训练数据的影响，可能包含固有的偏见。 ## 道德考虑 用户应确保该模型的部署符合道德指南，并考虑所生成文本的潜在社会影响。我们强烈反对将该模型滥用于生成有害或误导性内容。 ## 致谢 我们感谢Alibaba Cloud提供的基础模型。 ## 联系方式 如果对此模型有任何问题或意见或建议，请联系 [rubra团队](mailto:rubra@acorn.io)。 ## 引用信息 如果您使用了这项工作，请引用如下： ``` @misc {rubra_ai_2024, author = { Sanjay Nadhavajhala and Yingbei Tong }, title = { Rubra-Qwen2-7B-Instruct }, year = 2024, url = { https://huggingface.co/rubra-ai/Qwen2-7B-Instruct }, doi = { 10.57967/hf/2658 }, publisher = { Hugging Face } } ```