初始化项目,由ModelHub XC社区提供模型
Model: KwaiKEG/kagentlms_qwen_7b_mat Source: Original Platform
This commit is contained in:
86
README.md
Normal file
86
README.md
Normal file
@@ -0,0 +1,86 @@
|
||||
---
|
||||
license: cc-by-nc-nd-4.0
|
||||
datasets:
|
||||
- kwaikeg/KAgentInstruct
|
||||
- kwaikeg/KAgentBench
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
|
||||
KwaiAgents ([Github](https://github.com/KwaiKEG/KwaiAgents)) is a series of Agent-related works open-sourced by the [KwaiKEG](https://github.com/KwaiKEG) from [Kuaishou Technology](https://www.kuaishou.com/en). The open-sourced content includes:
|
||||
|
||||
1. **KAgentSys-Lite**: An experimental Agent Loop implemented based on open-source search engines, browsers, time, calendar, weather, and other tools, which is only missing the memory mechanism and some search capabilities compared to the system in the paper.
|
||||
2. **KAgentLMs**: A series of large language models with Agent capabilities such as planning, reflection, and tool-use, acquired through the Meta-agent tuning proposed in the paper.
|
||||
3. **KAgentInstruct**: Fine-tuned data of instructions generated by the Meta-agent in the paper.
|
||||
4. **KAgentBench**: Over 3,000 human-edited, automated evaluation data for testing Agent capabilities, with evaluation dimensions including planning, tool-use, reflection, concluding, and profiling.
|
||||
|
||||
|
||||
## User Guide
|
||||
|
||||
### Direct usage
|
||||
|
||||
Tutorial can refer to [QwenLM/Qwen](https://github.com/QwenLM/Qwen)
|
||||
```python
|
||||
from modelscope import AutoModelForCausalLM, AutoTokenizer
|
||||
from modelscope import GenerationConfig
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained("KwaiKEG/kagentlms_qwen_7b_mat", trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"KwaiKEG/kagentlms_qwen_7b_mat",
|
||||
device_map="auto",
|
||||
trust_remote_code=True
|
||||
).eval()
|
||||
|
||||
response, history = model.chat(tokenizer, "你好", history=None)
|
||||
print(response)
|
||||
```
|
||||
|
||||
### AgentLMs as service
|
||||
We recommend using [vLLM](https://github.com/vllm-project/vllm) and [FastChat](https://github.com/lm-sys/FastChat) to deploy the model inference service. First, you need to install the corresponding packages (for detailed usage, please refer to the documentation of the two projects):
|
||||
```bash
|
||||
pip install vllm
|
||||
pip install "fschat[model_worker,webui]"
|
||||
```
|
||||
To deploy KAgentLMs, you first need to start the controller in one terminal.
|
||||
```bash
|
||||
python -m fastchat.serve.controller
|
||||
```
|
||||
Secondly, you should use the following command in another terminal for single-gpu inference service deployment:
|
||||
```bash
|
||||
python -m fastchat.serve.vllm_worker --model-path $model_path --trust-remote-code
|
||||
```
|
||||
Where `$model_path` is the local path of the model downloaded. If the GPU does not support Bfloat16, you can add `--dtype half` to the command line.
|
||||
|
||||
Thirdly, start the REST API server in the third terminal.
|
||||
```bash
|
||||
python -m fastchat.serve.openai_api_server --host localhost --port 8888
|
||||
```
|
||||
|
||||
Finally, you can use the curl command to invoke the model same as the OpenAI calling format. Here's an example:
|
||||
```bash
|
||||
curl http://localhost:8888/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "kagentlms_qwen_7b_mat", "messages": [{"role": "user", "content": "Who is Andy Lau"}]}'
|
||||
```
|
||||
|
||||
### Citation
|
||||
```
|
||||
@article{pan2023kwaiagents,
|
||||
author = {Haojie Pan and
|
||||
Zepeng Zhai and
|
||||
Hao Yuan and
|
||||
Yaojia Lv and
|
||||
Ruiji Fu and
|
||||
Ming Liu and
|
||||
Zhongyuan Wang and
|
||||
Bing Qin
|
||||
},
|
||||
title = {KwaiAgents: Generalized Information-seeking Agent System with Large Language Models},
|
||||
journal = {CoRR},
|
||||
volume = {abs/2312.04889},
|
||||
year = {2023}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user