48 lines
1.7 KiB
Markdown
48 lines
1.7 KiB
Markdown
---
|
||
<!-- 该部分为参数配置部分 -->
|
||
|
||
---
|
||
<!-- 公共内容部分 -->
|
||
|
||
## 模型加载和推理
|
||
更多关于模型加载和推理的问题参考[模型的推理Pipeline](https://modelscope.cn/docs/%E6%A8%A1%E5%9E%8B%E7%9A%84%E6%8E%A8%E7%90%86Pipeline)。
|
||
|
||
未指定task，请从“模型介绍”中获取使用说明和代码范例。
|
||
|
||
更多使用说明请参阅[ModelScope文档中心](http://www.modelscope.cn/#/docs)。
|
||
|
||
---
|
||
<!-- 在线使用独有内容部分 -->
|
||
升级ModelScope版本
|
||
|
||
```
|
||
pip install "modelscope==1.7.2rc0" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
|
||
```
|
||
推理代码
|
||
|
||
```python
|
||
import torch
|
||
from modelscope import snapshot_download, Model
|
||
from modelscope.models.nlp.llama2 import Llama2Tokenizer
|
||
|
||
model_dir = snapshot_download("modelscope/Llama-2-7b-ms", revision='v1.0.1',
|
||
ignore_file_pattern = [r'\w+\.safetensors'])
|
||
model = Model.from_pretrained(model_dir, device_map='auto', torch_dtype=torch.float16)
|
||
tokenizer = Llama2Tokenizer.from_pretrained(model_dir)
|
||
|
||
prompt = "Hey, are you conscious? Can you talk to me?"
|
||
inputs = tokenizer(prompt, return_tensors="pt")
|
||
|
||
# Generate
|
||
generate_ids = model.generate(inputs.input_ids, max_length=30)
|
||
print(tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0])
|
||
```
|
||
---
|
||
<!-- 本地使用独有内容部分 -->
|
||
## 下载并安装ModelScope library
|
||
更多关于下载安装ModelScope library的问题参考[环境安装](https://modelscope.cn/docs/%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85)。
|
||
|
||
```python
|
||
pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
|
||
```
|