LeroyDyer/Mixtral_AI_Cyber_4.0
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: LeroyDyer/Mixtral_AI_Cyber_4.0
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-27 14:46:40 +08:00
commit 4077642952
17 changed files with 91403 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

81
README.md Normal file
View File

@@ -0,0 +1,81 @@
---
base_model:
- LeroyDyer/Mixtral_AI_Cyber_3.m1
- LeroyDyer/Mixtral_AI_Cyber_3.1_SFT
- LeroyDyer/Mixtral_AI_Cyber_2.0
- LeroyDyer/Mixtral_AI_Cyber_1.0
- LeroyDyer/Mixtral_AI_Cyber_3.m2
- LeroyDyer/Mixtral_AI_Cyber_3.0
library_name: transformers
tags:
- biology
- chemistry
- medical
license: mit
language:
- en
metrics:
- bertscore
- code_eval
- character
---
### Models Merged
The following models were included in the merge:
* [LeroyDyer/Mixtral_AI_Cyber_3.m1](https://huggingface.co/LeroyDyer/Mixtral_AI_Cyber_3.m1)
* [LeroyDyer/Mixtral_AI_Cyber_3.1_SFT](https://huggingface.co/LeroyDyer/Mixtral_AI_Cyber_3.1_SFT)
* [LeroyDyer/Mixtral_AI_Cyber_2.0](https://huggingface.co/LeroyDyer/Mixtral_AI_Cyber_2.0)
* [LeroyDyer/Mixtral_AI_Cyber_1.0](https://huggingface.co/LeroyDyer/Mixtral_AI_Cyber_1.0)
* [LeroyDyer/Mixtral_AI_Cyber_3.0](https://huggingface.co/LeroyDyer/Mixtral_AI_Cyber_3.0)
A Merges of my best models !:
A very great model as it contains the deltas from all of the very hard trained models : all these models were heavy coders!
### Configuration
The following YAML configuration was used to produce this model:
```yaml
models:
- model: LeroyDyer/Mixtral_AI_Cyber_3.m2
parameters:
density: [0.256, 0.512, 0.128] # density gradient
weight: 0.382
- model: LeroyDyer/Mixtral_AI_Cyber_2.0
parameters:
density: 0.382
weight: [0.256, 0.128, 0.256, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_3.0
parameters:
density: 0.382
weight: [0.128, 0.512, 0.128, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_3.m1
parameters:
density: 0.382
weight: [0.256, 0.256, 0.512, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_1.0
parameters:
density: 0.382
weight: [0.128, 0.512, 0.128, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_3.1_SFT
parameters:
density: 0.382
weight:
- filter: mlp
value: 0.5
- value: 0
merge_method: ties
base_model: LeroyDyer/Mixtral_AI_Cyber_3.m2
parameters:
normalize: true
int8_mask: true
dtype: float16
```

28
config.json Normal file
View File

@@ -0,0 +1,28 @@
{
"_name_or_path": "LeroyDyer/Mixtral_AI_Cyber_4.0",
"architectures": [
"MistralForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 32768,
"model_type": "mistral",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"pad_token_id": 2,
"rms_norm_eps": 1e-05,
"rope_theta": 10000.0,
"sliding_window": 4096,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.38.2",
"unsloth_version": "2024.3",
"use_cache": true,
"vocab_size": 32000
}

35
mergekit_config.yml Normal file
View File

@@ -0,0 +1,35 @@
models:
- model: LeroyDyer/Mixtral_AI_Cyber_3.m2
parameters:
density: [0.256, 0.512, 0.128] # density gradient
weight: 0.382
- model: LeroyDyer/Mixtral_AI_Cyber_2.0
parameters:
density: 0.382
weight: [0.256, 0.128, 0.256, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_3.0
parameters:
density: 0.382
weight: [0.128, 0.512, 0.128, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_3.m1
parameters:
density: 0.382
weight: [0.256, 0.256, 0.512, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_1.0
parameters:
density: 0.382
weight: [0.128, 0.512, 0.128, 0.128] # weight gradient
- model: LeroyDyer/Mixtral_AI_Cyber_3.1_SFT
parameters:
density: 0.382
weight:
- filter: mlp
value: 0.5
- value: 0
merge_method: ties
base_model: LeroyDyer/Mixtral_AI_Cyber_3.m2
parameters:
normalize: true
int8_mask: true
dtype: float16

3
model-00001-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c18a99280a4c1abce0209e0d1e4e5fc8d66b434039397198307dff2a7a748bf2
size 1912664448

3
model-00002-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a66af77b2b597d77e22a65d82e10ae6c1f757f2b571289b81cd8545ca0bd01e
size 1895870400

3
model-00003-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2a3069bec3770d7da1862a39afdabb501156fdd3bf1c28571e4300cc921a6718
size 1946227288

3
model-00004-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a441e53812eb2359e038246e1df69814a3fe3a557edadd7e16943685f11ccea8
size 1979781416

3
model-00005-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:52943d518cccf7af24e1244eaa50f600ad0b623d25e58b95de2fc38f280a3ce4
size 1889645184

3
model-00006-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:732f00ed61c86ce5b340893a3f67184f87eaa871635e81e4656cee23dacb7222
size 1946210728

3
model-00007-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f6475e02c882c257cd6389a7ff032eb4fe5a340358c228aca53ecf6bbf3d9e50
size 1923182616

3
model-00008-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a593dde4cdf03f2d3664af7f629ab1c16650a19b4a4fd5f15dab6f4e3f3208e
size 989915608

1
model.safetensors.index.json Normal file
View File

File diff suppressed because one or more lines are too long

30
special_tokens_map.json Normal file
View File

@@ -0,0 +1,30 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

91122
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

44
tokenizer_config.json Normal file
View File

@@ -0,0 +1,44 @@
{
"add_bos_token": true,
"add_eos_token": false,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [],
"bos_token": "<s>",
"chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"model_max_length": 32768,
"pad_token": "<unk>",
"padding_side": "right",
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}