48 lines
1.7 KiB
Markdown
48 lines
1.7 KiB
Markdown
|
---
|
|||
|
|
<!-- 该部分为参数配置部分 -->
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
<!-- 公共内容部分 -->
|
|||
|
|
|
|||
|
|
## 模型加载和推理
|
|||
|
|
更多关于模型加载和推理的问题参考[模型的推理Pipeline](https://modelscope.cn/docs/%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%8E%A8%E7%90%86Pipeline)。
|
|||
|
|
|
|||
|
|
未指定task，请从“模型介绍”中获取使用说明和代码范例。
|
|||
|
|
|
|||
|
|
更多使用说明请参阅[ModelScope文档中心](http://www.modelscope.cn/#/docs)。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
<!-- 在线使用独有内容部分 -->
|
|||
|
|
升级ModelScope版本
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
pip install "modelscope==1.7.2rc0" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
|
|||
|
|
```
|
|||
|
|
推理代码
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
import torch
|
|||
|
|
from modelscope import snapshot_download, Model
|
|||
|
|
from modelscope.models.nlp.llama2 import Llama2Tokenizer
|
|||
|
|
|
|||
|
|
model_dir = snapshot_download("modelscope/Llama-2-7b-ms", revision='v1.0.1',
|
|||
|
|
ignore_file_pattern = [r'\w+\.safetensors'])
|
|||
|
|
model = Model.from_pretrained(model_dir, device_map='auto', torch_dtype=torch.float16)
|
|||
|
|
tokenizer = Llama2Tokenizer.from_pretrained(model_dir)
|
|||
|
|
|
|||
|
|
prompt = "Hey, are you conscious? Can you talk to me?"
|
|||
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
|||
|
|
|
|||
|
|
# Generate
|
|||
|
|
generate_ids = model.generate(inputs.input_ids, max_length=30)
|
|||
|
|
print(tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0])
|
|||
|
|
```
|
|||
|
|
---
|
|||
|
|
<!-- 本地使用独有内容部分 -->
|
|||
|
|
## 下载并安装ModelScope library
|
|||
|
|
更多关于下载安装ModelScope library的问题参考[环境安装](https://modelscope.cn/docs/%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85)。
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
|
|||
|
|
```
|