Llama-2-7b-ms/quickstart.md

模型加载和推理

更多关于模型加载和推理的问题参考模型的推理Pipeline。

未指定task，请从“模型介绍”中获取使用说明和代码范例。

更多使用说明请参阅ModelScope文档中心。

---

升级ModelScope版本

pip install "modelscope==1.7.2rc0" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html

推理代码

import torch
from modelscope import snapshot_download, Model
from modelscope.models.nlp.llama2 import Llama2Tokenizer

model_dir = snapshot_download("modelscope/Llama-2-7b-ms", revision='v1.0.1', 
                              ignore_file_pattern = [r'\w+\.safetensors'])
model = Model.from_pretrained(model_dir, device_map='auto', torch_dtype=torch.float16)
tokenizer = Llama2Tokenizer.from_pretrained(model_dir)

prompt = "Hey, are you conscious? Can you talk to me?"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
generate_ids = model.generate(inputs.input_ids, max_length=30)
print(tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0])

---

下载并安装ModelScope library

更多关于下载安装ModelScope library的问题参考环境安装。

pip install "modelscope[audio,cv,nlp,multi-modal,science]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html