初始化项目，由ModelHub XC社区提供模型
Model: jojo-ai-mst/MyanmarGPT-Big Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
BIN
MyanmarGPT Big.jpeg
Normal file
BIN
MyanmarGPT Big.jpeg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
101
README.md
Normal file
101
README.md
Normal file
@@ -0,0 +1,101 @@
|
||||
---
|
||||
license: creativeml-openrail-m
|
||||
language:
|
||||
- my
|
||||
---
|
||||
|
||||
# MyanmarGPT-Big
|
||||
|
||||
- Multi-language model with Burmese Text and 1.42 Billion Parameters.
|
||||
- Supports 61 Languages.
|
||||
|
||||
<img src="./MyanmarGPT%20Big.jpeg" alt="MyanmarGPT Big Cover Photo" width=400 height=400/>
|
||||
|
||||
Everyone can fine-tune this model.
|
||||
Designed primarily for Burmese text completion, this model serves as a foundational framework for fine-tuning various natural language processing tasks specific to the Burmese language context.
|
||||
|
||||
## About the project
|
||||
|
||||
> Everyone has the right to create AI in Myanmar.
|
||||
|
||||
As Myanmar embarks on its journey towards AI democratization, a strategic and collaborative approach is crucial. Addressing challenges and seizing opportunities in sectors such as agriculture, healthcare, and education can position Myanmar as a regional leader in harnessing the benefits of AI for the betterment of its people and the advancement of its economy. With careful planning and investment, Myanmar has the potential to create a vibrant AI ecosystem that empowers individuals, businesses, and the nation as a whole.
|
||||
|
||||
## MyanmarGPT
|
||||
|
||||
There is already MyanmarGPT which 125 M parameters. But people in Myanmar has been asking me for More precise model with more weights,
|
||||
Here is the MyanmarGPT-Big Model now.
|
||||
|
||||
You can use the [MyanmarGPT model 125 M](https://huggingface.co/jojo-ai-mst/MyanmarGPT) which is lightweight and free to use.
|
||||
|
||||
### Model Description
|
||||
|
||||
- **Developed by:** [Min Si Thu](https://www.linkedin.com/in/min-si-thu/)
|
||||
- **Model type:** [GPT2]
|
||||
- **Language(s) (NLP):** MultiLanguage, But especially Burmese Language
|
||||
- **License:** CreativeML-OpenRail-M
|
||||
- **Finetuned from model [optional]:** [mGPT]
|
||||
|
||||
## How to use
|
||||
|
||||
### Using pipeline
|
||||
|
||||
```shell
|
||||
pip install transformers
|
||||
```
|
||||
|
||||
```python
|
||||
from transformers import pipeline
|
||||
|
||||
pipe = pipeline("text-generation", model="jojo-ai-mst/MyanmarGPT-Big")
|
||||
outputs = pipe("အီတလီ",do_sample=False)
|
||||
print(outputs)
|
||||
```
|
||||
|
||||
### Using Model Generator
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained("jojo-ai-mst/MyanmarGPT-Big")
|
||||
model = AutoModelForCausalLM.from_pretrained("jojo-ai-mst/MyanmarGPT-Big")
|
||||
|
||||
|
||||
input_ids = tokenizer.encode("ချစ်သား", return_tensors='pt')
|
||||
output = model.generate(input_ids, max_length=50)
|
||||
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Applied Uses
|
||||
|
||||
MyanmarGPT-Big can be used for the following use cases.
|
||||
|
||||
Text generation, Chatbots and Virtual Assistants, Content Summarization, Translations, Question-Answering System, and Sentiment Analysis.
|
||||
|
||||
### Direct Use
|
||||
|
||||
Originally crafted for text completion in Burmese, this model functions as a fundamental asset for various Natural Language Processing (NLP) tasks. Although its primary role is presently centered on aiding in text generation and completion, it harbors considerable potential for broader applications. Researchers and developers have the option to refine this model using specialized datasets, thereby expanding its utility to other NLP domains, including summarization and instruction-based tasks. Nevertheless, it is crucial to acknowledge that when dealing with high-stakes decisions or comprehending domain-specific terminology, additional specialized training for the model is advised to ensure optimal accuracy and reliability.
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
Users need to recognize the inherent limitations and biases present in language models.
|
||||
Responsible usage is crucial, particularly in sensitive contexts, as this model is not designed to generate misleading or harmful content.
|
||||
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
While the MyanmarGPT-Big excels in handling general Burmese text, its effectiveness might be limited when dealing with daily-life spoken burmese words.
|
||||
Users are encouraged to perform comprehensive testing tailored to their specific use cases.
|
||||
|
||||
|
||||
## mGPT
|
||||
|
||||
Special thanks to [mGPT](https://huggingface.co/ai-forever/mGPT) Project by [ai-forever](https://huggingface.co/ai-forever).
|
||||
Without mGPT, MyanmarGPT-Big would have taken a long time to move on from building from scratch.
|
||||
|
||||
## Contact
|
||||
|
||||
Reach me via
|
||||
- LinkedIn - [Min Si Thu](https://www.linkedin.com/in/min-si-thu/)
|
||||
- GitHub - [Min Si Thu](http://github.com/MinSiThu)
|
||||
- Medium - [Min Si Thu](https://medium.com/@minsithu_53495)
|
||||
- Hashnode - [Min Si Thu](https://hashnode.com/@MinSiThu)
|
||||
25
adapter_config.json
Normal file
25
adapter_config.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"alpha_pattern": {},
|
||||
"auto_mapping": null,
|
||||
"base_model_name_or_path": "ai-forever/mGPT",
|
||||
"bias": "none",
|
||||
"fan_in_fan_out": true,
|
||||
"inference_mode": true,
|
||||
"init_lora_weights": true,
|
||||
"layers_pattern": null,
|
||||
"layers_to_transform": null,
|
||||
"loftq_config": {},
|
||||
"lora_alpha": 32,
|
||||
"lora_dropout": 0.05,
|
||||
"megatron_config": null,
|
||||
"megatron_core": "megatron.core",
|
||||
"modules_to_save": null,
|
||||
"peft_type": "LORA",
|
||||
"r": 16,
|
||||
"rank_pattern": {},
|
||||
"revision": null,
|
||||
"target_modules": [
|
||||
"c_attn"
|
||||
],
|
||||
"task_type": "CAUSAL_LM"
|
||||
}
|
||||
3
adapter_model.safetensors
Normal file
3
adapter_model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a962d8c0feb6df6bf357a2c992e3547ab8f2b78ef2c3d57e2e9503391133d354
|
||||
size 12589136
|
||||
35
config.json
Normal file
35
config.json
Normal file
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"_name_or_path": "ai-forever/mGPT",
|
||||
"activation_function": "gelu_new",
|
||||
"architectures": [
|
||||
"GPT2LMHeadModel"
|
||||
],
|
||||
"attn_pdrop": 0.1,
|
||||
"bos_token_id": 0,
|
||||
"embd_pdrop": 0.1,
|
||||
"eos_token_id": 5,
|
||||
"gradient_checkpointing": false,
|
||||
"initializer_range": 0.02,
|
||||
"layer_norm_epsilon": 1e-05,
|
||||
"model_type": "gpt2",
|
||||
"n_ctx": 2048,
|
||||
"n_embd": 2048,
|
||||
"n_head": 16,
|
||||
"n_inner": null,
|
||||
"n_layer": 24,
|
||||
"n_positions": 2048,
|
||||
"pad_token_id": 1,
|
||||
"reorder_and_upcast_attn": false,
|
||||
"resid_pdrop": 0.1,
|
||||
"scale_attn_by_inverse_layer_idx": false,
|
||||
"scale_attn_weights": true,
|
||||
"summary_activation": null,
|
||||
"summary_first_dropout": 0.1,
|
||||
"summary_proj_to_labels": true,
|
||||
"summary_type": "cls_index",
|
||||
"summary_use_proj": true,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.36.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 100000
|
||||
}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 5,
|
||||
"pad_token_id": 1,
|
||||
"transformers_version": "4.36.0"
|
||||
}
|
||||
99738
merges.txt
Normal file
99738
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8edaebbbbe05ab877016502816c7065e034d033a46b5ba781a4a989ea51c944f
|
||||
size 4998981392
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:967da22d986ba044f3c6934d385adf4c281443a0d378c4375df2ff9273366c4b
|
||||
size 671436856
|
||||
299
model.safetensors.index.json
Normal file
299
model.safetensors.index.json
Normal file
@@ -0,0 +1,299 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 5670387712
|
||||
},
|
||||
"weight_map": {
|
||||
"transformer.h.0.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.0.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.1.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.10.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.11.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.12.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.13.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.14.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.15.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.16.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.17.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.18.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.19.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.2.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.20.mlp.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.20.mlp.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.attn.c_attn.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.attn.c_attn.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.attn.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.attn.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.ln_1.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.ln_1.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.ln_2.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.ln_2.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.mlp.c_fc.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.mlp.c_fc.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.mlp.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.21.mlp.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.attn.c_attn.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.attn.c_attn.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.attn.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.attn.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.ln_1.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.ln_1.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.ln_2.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.ln_2.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.mlp.c_fc.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.mlp.c_fc.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.mlp.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.22.mlp.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.attn.c_attn.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.attn.c_attn.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.attn.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.attn.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.ln_1.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.ln_1.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.ln_2.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.ln_2.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.mlp.c_fc.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.mlp.c_fc.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.mlp.c_proj.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.23.mlp.c_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.h.3.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.3.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.4.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.5.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.6.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.7.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.8.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.attn.c_attn.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.attn.c_attn.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.attn.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.attn.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.ln_1.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.ln_1.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.ln_2.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.ln_2.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.mlp.c_fc.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.mlp.c_fc.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.mlp.c_proj.bias": "model-00001-of-00002.safetensors",
|
||||
"transformer.h.9.mlp.c_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.ln_f.bias": "model-00002-of-00002.safetensors",
|
||||
"transformer.ln_f.weight": "model-00002-of-00002.safetensors",
|
||||
"transformer.wpe.weight": "model-00001-of-00002.safetensors",
|
||||
"transformer.wte.weight": "model-00001-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
44
special_tokens_map.json
Normal file
44
special_tokens_map.json
Normal file
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"mask_token": {
|
||||
"content": "<mask>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<pad>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"sep_token": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
199830
tokenizer.json
Normal file
199830
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
67
tokenizer_config.json
Normal file
67
tokenizer_config.json
Normal file
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"1": {
|
||||
"content": "<pad>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"2": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"3": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"4": {
|
||||
"content": "<mask>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"5": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"bos_token": "<s>",
|
||||
"clean_up_tokenization_spaces": true,
|
||||
"eos_token": "<|endoftext|>",
|
||||
"errors": "replace",
|
||||
"mask_token": "<mask>",
|
||||
"model_max_length": 2048,
|
||||
"pad_token": "<pad>",
|
||||
"padding_side": "left",
|
||||
"sep_token": "</s>",
|
||||
"tokenizer_class": "GPT2Tokenizer",
|
||||
"truncation_side": "left",
|
||||
"trust_remote_code": false,
|
||||
"unk_token": "<unk>"
|
||||
}
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user