初始化项目，由ModelHub XC社区提供模型

Model: AI-ModelScope/Mistral-7B-Instruct-v0.1
Source: Original Platform

README.md

@@ -0,0 +1,144 @@
+---
+license: apache-2.0
+pipeline_tag: text-generation
+tags:
+- finetuned
+inference:
+  parameters:
+    temperature: 0.7
+---
+
+# Model Card for Mistral-7B-Instruct-v0.1
+
+The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) generative text model using a variety of publicly available conversation datasets.
+
+For full details of this model please read our [release blog post](https://mistral.ai/news/announcing-mistral-7b/)
+
+## Instruction format
+
+In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.
+
+E.g.
+```
+text = "<s>[INST] What is your favourite condiment? [/INST]"
+"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
+"[INST] Do you have mayonnaise recipes? [/INST]"
+```
+
+This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
+
+```python
+# pip install "transformers>=4.34" "modelscope>=1.9.2"
+from modelscope import AutoModelForCausalLM, AutoTokenizer, snapshot_download
+import torch
+
+device = "cuda" # the device to load the model onto
+model_dir = snapshot_download('AI-ModelScope/Mistral-7B-Instruct-v0.1', revision='v1.0.0')
+model = AutoModelForCausalLM.from_pretrained(model_dir, torch_dtype=torch.bfloat16, device_map=device)
+tokenizer = AutoTokenizer.from_pretrained(model_dir)
+
+messages = [
+    {"role": "user", "content": "What is your favourite condiment?"},
+    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
+    {"role": "user", "content": "Do you have mayonnaise recipes?"}
+]
+
+encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
+
+model_inputs = encodeds.to(device)
+model.to(device)
+
+generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
+decoded = tokenizer.batch_decode(generated_ids)
+print(decoded[0])
+```
+
+## 微调(SFT)
+
+**代码链接**: https://github.com/modelscope/swift/tree/main/examples/pytorch/llm
+
+1. 支持的sft方法: lora, qlora, 全参数微调, ...
+2. 支持的模型: qwen系列, qwen-vl系列, baichuan系列, chatglm2系列, llama系列, openbuddy-llama系列, internlm系列, xverse系列, ...
+3. 支持的特性: 模型量化, DDP, 模型并行, gradient checkpointing, 梯度累加, 支持推送ModelScope Hub, 自定义数据集, 多模态和Agent SFT, 多轮对话, ...
+
+使用lora+ddp+deepspeed SFT mistral-7b-chat的脚本 (需要2\*21GB显存)
+```bash
+# https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/scripts/mistral_7b_chat/lora_ddp_ds/sft.sh
+# Experimental environment: 2 * A10
+# 2 * 21GB GPU memory
+nproc_per_node=2
+
+PYTHONPATH=../../.. \
+CUDA_VISIBLE_DEVICES=0,1 \
+torchrun \
+    --nproc_per_node=$nproc_per_node \
+    --master_port 29500 \
+    src/llm_sft.py \
+    --model_type mistral-7b-chat \
+    --sft_type lora \
+    --template_type llama \
+    --dtype bf16 \
+    --output_dir output \
+    --ddp_backend nccl \
+    --dataset leetcode-python-en \
+    --train_dataset_sample -1 \
+    --num_train_epochs 1 \
+    --max_length 4096 \
+    --lora_rank 8 \
+    --lora_alpha 32 \
+    --lora_dropout_p 0. \
+    --lora_target_modules ALL \
+    --gradient_checkpointing true \
+    --batch_size 1 \
+    --weight_decay 0. \
+    --learning_rate 1e-4 \
+    --gradient_accumulation_steps $(expr 16 / $nproc_per_node) \
+    --max_grad_norm 0.5 \
+    --warmup_ratio 0.03 \
+    --eval_steps 100 \
+    --save_steps 100 \
+    --save_total_limit 2 \
+    --logging_steps 10 \
+    --push_to_hub false \
+    --hub_model_id mistral-7b-chat-qlora \
+    --hub_private_repo true \
+    --hub_token 'your-sdk-token' \
+    --deepspeed_config_path 'ds_config/zero2.json' \
+    --only_save_model true \
+
+```
+
+## Model Architecture
+This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:
+- Grouped-Query Attention
+- Sliding-Window Attention
+- Byte-fallback BPE tokenizer
+
+## Troubleshooting
+- If you see the following error:
+```
+Traceback (most recent call last):
+File "", line 1, in
+File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
+config, kwargs = AutoConfig.from_pretrained(
+File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
+config_class = CONFIG_MAPPING[config_dict["model_type"]]
+File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
+raise KeyError(key)
+KeyError: 'mistral'
+```
+
+Installing transformers from source should solve the issue
+pip install git+https://github.com/huggingface/transformers
+
+This should not be required after transformers-v4.33.4.
+
+## Limitations
+
+The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance. 
+It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
+make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
+
+## The Mistral AI Team
+
+Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.