初始化项目,由ModelHub XC社区提供模型
Model: KoalaAI/OPT-1.3b-Chat Source: Original Platform
This commit is contained in:
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
61
README.md
Normal file
61
README.md
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
---
|
||||||
|
license: other
|
||||||
|
widget:
|
||||||
|
- text: 'What is the capital of France?'
|
||||||
|
- text: 'What is wikipedia?'
|
||||||
|
- text: 'What is a meme?'
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
tags:
|
||||||
|
- conversational
|
||||||
|
- chat
|
||||||
|
- assistant
|
||||||
|
---
|
||||||
|
# OPT-1.3b-Chat
|
||||||
|
|
||||||
|
This is a text generation model based on the [OPT-1.3B](https://huggingface.co/facebook/opt-1.3b) model from Meta, trained using the Deepspeed library. The model can generate natural and engaging conversational responses given a user input.
|
||||||
|
|
||||||
|
A Demo is [available here](https://huggingface.co/spaces/KoalaAI/OPT-Chat)
|
||||||
|
The model is best at simple Q&A style questions, not open-ended ones like ChatGPT.
|
||||||
|
|
||||||
|
## Training Details
|
||||||
|
|
||||||
|
- The base model is [OPT-1.3B](https://huggingface.co/facebook/opt-1.3b), a decoder-only transformer with 1.3 billion parameters, pre-trained on a large text corpus using the causal language modeling objective.
|
||||||
|
- The model was trained on a single NVIDIA A100 GPU using the Deepspeed pipeline parallelism and ZeRO optimizer.
|
||||||
|
|
||||||
|
## Model Details
|
||||||
|
- Number of parameters: 1.3 billion
|
||||||
|
- Number of layers: 24
|
||||||
|
- Number of attention heads: 16
|
||||||
|
- Context size: 2048
|
||||||
|
- Vocabulary size: 50,265
|
||||||
|
- Embedding size: 1280
|
||||||
|
- Feed-forward size: 5120
|
||||||
|
- Dropout rate: 0.1
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
You can use this model directly with the Hugging Face pipeline for text generation:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import pipeline
|
||||||
|
generator = pipeline('text-generation', model='DarwinAnim8or/OPT-1.3b-Chat')
|
||||||
|
generator("Hello, how are you?")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Suggested formatting
|
||||||
|
The training data uses the following format:
|
||||||
|
```
|
||||||
|
Human: <question>
|
||||||
|
Assistant: <answer>
|
||||||
|
```
|
||||||
|
|
||||||
|
It is recommended to follow the same format as closely as possible for the best results.
|
||||||
|
We do intend on creating another model that is trained on the openassistant dataset in the future.
|
||||||
|
|
||||||
|
## License
|
||||||
|
This model is licensed under the [OPT-175B license](https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/MODEL_LICENSE.md), which is a non-commercial research license. Please read the full license terms before using this model.
|
||||||
|
|
||||||
|
## Ethical Considerations
|
||||||
|
This model is intended for research purposes only and should not be used for any malicious or harmful applications. The model may generate offensive or inappropriate content that does not reflect the views or opinions of the authors or Microsoft. Users are responsible for ensuring that the generated content complies with ethical and legal standards.
|
||||||
32
config.json
Normal file
32
config.json
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
{
|
||||||
|
"_name_or_path": "facebook/opt-1.3b",
|
||||||
|
"_remove_final_layer_norm": false,
|
||||||
|
"activation_dropout": 0.0,
|
||||||
|
"activation_function": "relu",
|
||||||
|
"architectures": [
|
||||||
|
"OPTForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 2,
|
||||||
|
"do_layer_norm_before": true,
|
||||||
|
"dropout": 0.1,
|
||||||
|
"enable_bias": true,
|
||||||
|
"end_token_id": 2,
|
||||||
|
"eos_token_id": 2,
|
||||||
|
"ffn_dim": 8192,
|
||||||
|
"hidden_size": 2048,
|
||||||
|
"init_std": 0.02,
|
||||||
|
"layer_norm_elementwise_affine": true,
|
||||||
|
"layerdrop": 0.0,
|
||||||
|
"max_position_embeddings": 2048,
|
||||||
|
"model_type": "opt",
|
||||||
|
"num_attention_heads": 32,
|
||||||
|
"num_hidden_layers": 24,
|
||||||
|
"pad_token_id": 2,
|
||||||
|
"prefix": "</s>",
|
||||||
|
"torch_dtype": "float16",
|
||||||
|
"transformers_version": "4.32.1",
|
||||||
|
"use_cache": true,
|
||||||
|
"vocab_size": 50272,
|
||||||
|
"word_embed_proj_dim": 2048
|
||||||
|
}
|
||||||
50001
merges.txt
Normal file
50001
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:48c559568bb4aea4c57895b611b9d9ab4f6cf40648e565dd1e52d02a86e673b9
|
||||||
|
size 2631561680
|
||||||
3
pytorch_model.bin
Normal file
3
pytorch_model.bin
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5f658dc4b8d77e95a8432459f6b6a03ea5b98111a2e83a7252e7f80742b2b07e
|
||||||
|
size 2631625301
|
||||||
1228
training.log
Normal file
1228
training.log
Normal file
File diff suppressed because one or more lines are too long
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user