初始化项目,由ModelHub XC社区提供模型
Model: Heralax/MythoMakiseMerged-13b Source: Original Platform
This commit is contained in:
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
MythoMakiseMerged-13b-q5km.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
ggml-model-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
MythoMakiseMerged-13b-q5km.gguf
Normal file
3
MythoMakiseMerged-13b-q5km.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:34e6b10bf0c458834f874a7b86968c5fd143fb766a2906c1d0f6f7b654062c9e
|
||||
size 9229924224
|
||||
42
README.md
Normal file
42
README.md
Normal file
@@ -0,0 +1,42 @@
|
||||
---
|
||||
license: llama2
|
||||
---
|
||||
## KEY DETAILS
|
||||
Prompt format: SillyTavern
|
||||
Base model: MythoMax-L2-13b
|
||||
What's new: finetuned on the script of a visual novel that was processed and revamped by GPT-4 to make ~1300 high-quality training examples. The end goal was a model that could speak like a specific character from that game, but the end result was a model that seems to excel in banter, conversation, and roleplay overall.
|
||||
Note: compared to the original MythoMakise-13b, this model has 33% of MythoMax-L2-13b merged back into it, so that it better retains MythoMax's intelligence with MythoMakise's personality and style. The result of this seems to be pretty good so far. Ironcially, the model seems better at roleplaying characters other than the one it was originally created to mimic.
|
||||
|
||||
### LONG FORM
|
||||
|
||||
A finetune of MythoMax-13b on lines extracted from the script of Steins;Gate. Rather than simply giving the model "previous line\nline to predict" a custom script was used to group conversations into training examples.
|
||||
Despite being finetuned on one character's lines from one visual novel, I've found (at least in my initial testing) that the model does an excellent job of roleplaying other characters too, probably because the creative writing GPT-4 did on top of the already-well-written Steins;Gate script was very high-quality. The model might be best at roleplaying characters if the personality of that character is similar to the character it was originally made to act like.
|
||||
|
||||
Besides being built for RP, I bet that this model could be used in any generic conversational role. Just don't expect it to be accurate, or good at anything other than talking.
|
||||
|
||||
The model is not censored.
|
||||
|
||||
This variation has MythoMax merged back into it with 33% weighting to make it more stable and intelligent while retaining its Kurisu-ness and better personality. In my experience, this seems to be the decisive change that led to higher-quality outputs.
|
||||
|
||||
### Prompt format
|
||||
I know it's wasteful as hell, don't judge me, this is the SillyTavern prompt format (discovered using the simple proxy for ST). I finetuned the model on this so that it would perform better on that frontend.
|
||||
```
|
||||
## {{charname}}:
|
||||
- You're "{{charname}}" in this never-ending roleplay with "{{user}}".
|
||||
### Input:\n
|
||||
[user description (note, square brackets are a part of it)]
|
||||
Description of the character's personality would go here (a 'character card')
|
||||
|
||||
### Response:
|
||||
(OOC) Understood. I will take this info into account for the roleplay. (end OOC)
|
||||
|
||||
### New Roleplay:
|
||||
### Instruction:
|
||||
#### {{char}}:
|
||||
whatever the char says, this is the chat history
|
||||
#### {{user}}:
|
||||
whatever the user says, this is the chat history
|
||||
... repeated some number of times ...
|
||||
### Response 2 paragraphs, engaging, natural, authentic, descriptive, creative):
|
||||
#### {char}:
|
||||
```
|
||||
3
added_tokens.json
Normal file
3
added_tokens.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"<pad>": 32000
|
||||
}
|
||||
26
config.json
Normal file
26
config.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"_name_or_path": "op_amadeus_models/MythoMax-Kurisu-13b-e1/",
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 5120,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 13824,
|
||||
"max_position_embeddings": 4096,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 40,
|
||||
"num_hidden_layers": 40,
|
||||
"num_key_value_heads": 40,
|
||||
"pad_token_id": 0,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.31.0",
|
||||
"use_cache": false,
|
||||
"vocab_size": 32000
|
||||
}
|
||||
3
ggml-model-f16.gguf
Normal file
3
ggml-model-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5365e77995d9a340d7e502cd4a5d5b9c682d41401c5e54064208a918f0d3786a
|
||||
size 26033303392
|
||||
3
model-00001-of-00006.safetensors
Normal file
3
model-00001-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d10bf79d311e67dc0113808fecf538e402a5d11e4c35f99a8d68e92bb23674b3
|
||||
size 5305947768
|
||||
3
model-00002-of-00006.safetensors
Normal file
3
model-00002-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2bb1e0472d33b1424fc8dbc723f92a1e81ae886518bd22c8074c9239edbb51e0
|
||||
size 4970423944
|
||||
3
model-00003-of-00006.safetensors
Normal file
3
model-00003-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:abe3e88038b6566e6f803e552a32d3cb01c42af81567f0d29081b70e55526de1
|
||||
size 4970424216
|
||||
3
model-00004-of-00006.safetensors
Normal file
3
model-00004-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ecc5231773a496a89531d98b831372a1f7d51ce19b09ed9178d88a2427dc010d
|
||||
size 4933703456
|
||||
3
model-00005-of-00006.safetensors
Normal file
3
model-00005-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5ed28a9941620d9e64503be4bb986cbd4dd14aeca099a01c03dcc37edc581db6
|
||||
size 4933724168
|
||||
3
model-00006-of-00006.safetensors
Normal file
3
model-00006-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1c2f754ecf9f0daca797bc52000ec53724663d0f450dac02f77ced62daa0469d
|
||||
size 1572917168
|
||||
1
model.safetensors.index.json
Normal file
1
model.safetensors.index.json
Normal file
File diff suppressed because one or more lines are too long
3
pytorch_files/added_tokens.json
Normal file
3
pytorch_files/added_tokens.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"<pad>": 32000
|
||||
}
|
||||
26
pytorch_files/config.json
Normal file
26
pytorch_files/config.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"_name_or_path": "op_amadeus_models/MythoMax-Kurisu-13b-e1/",
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 5120,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 13824,
|
||||
"max_position_embeddings": 4096,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 40,
|
||||
"num_hidden_layers": 40,
|
||||
"num_key_value_heads": 40,
|
||||
"pad_token_id": 0,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.31.0",
|
||||
"use_cache": false,
|
||||
"vocab_size": 32000
|
||||
}
|
||||
3
pytorch_files/model-00001-of-00006.safetensors
Normal file
3
pytorch_files/model-00001-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d10bf79d311e67dc0113808fecf538e402a5d11e4c35f99a8d68e92bb23674b3
|
||||
size 5305947768
|
||||
3
pytorch_files/model-00002-of-00006.safetensors
Normal file
3
pytorch_files/model-00002-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2bb1e0472d33b1424fc8dbc723f92a1e81ae886518bd22c8074c9239edbb51e0
|
||||
size 4970423944
|
||||
3
pytorch_files/model-00003-of-00006.safetensors
Normal file
3
pytorch_files/model-00003-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:abe3e88038b6566e6f803e552a32d3cb01c42af81567f0d29081b70e55526de1
|
||||
size 4970424216
|
||||
3
pytorch_files/model-00004-of-00006.safetensors
Normal file
3
pytorch_files/model-00004-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ecc5231773a496a89531d98b831372a1f7d51ce19b09ed9178d88a2427dc010d
|
||||
size 4933703456
|
||||
3
pytorch_files/model-00005-of-00006.safetensors
Normal file
3
pytorch_files/model-00005-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5ed28a9941620d9e64503be4bb986cbd4dd14aeca099a01c03dcc37edc581db6
|
||||
size 4933724168
|
||||
3
pytorch_files/model-00006-of-00006.safetensors
Normal file
3
pytorch_files/model-00006-of-00006.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1c2f754ecf9f0daca797bc52000ec53724663d0f450dac02f77ced62daa0469d
|
||||
size 1572917168
|
||||
1
pytorch_files/model.safetensors.index.json
Normal file
1
pytorch_files/model.safetensors.index.json
Normal file
File diff suppressed because one or more lines are too long
24
pytorch_files/special_tokens_map.json
Normal file
24
pytorch_files/special_tokens_map.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<unk>",
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
93400
pytorch_files/tokenizer.json
Normal file
93400
pytorch_files/tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
3
pytorch_files/tokenizer.model
Normal file
3
pytorch_files/tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
|
||||
size 499723
|
||||
33
pytorch_files/tokenizer_config.json
Normal file
33
pytorch_files/tokenizer_config.json
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"legacy": false,
|
||||
"model_max_length": 4096,
|
||||
"pad_token": null,
|
||||
"padding_side": "right",
|
||||
"sp_model_kwargs": {},
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"unk_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
24
special_tokens_map.json
Normal file
24
special_tokens_map.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<unk>",
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
93400
tokenizer.json
Normal file
93400
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
3
tokenizer.model
Normal file
3
tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
|
||||
size 499723
|
||||
33
tokenizer_config.json
Normal file
33
tokenizer_config.json
Normal file
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "<s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "</s>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"legacy": false,
|
||||
"model_max_length": 4096,
|
||||
"pad_token": null,
|
||||
"padding_side": "right",
|
||||
"sp_model_kwargs": {},
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"unk_token": {
|
||||
"__type": "AddedToken",
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user