Cartinoe5930/SOLAR-DUS-implement
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Cartinoe5930/SOLAR-DUS-implement
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-10 06:39:01 +08:00
commit 1168c499e9
21 changed files with 91365 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

63
README.md Normal file
View File

@@ -0,0 +1,63 @@
---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- Cartinoe5930/Llama2_init_Mistral
- Cartinoe5930/Llama2_init_Mistral
---
# SOLAR-DUS-implement
SOLAR-DUS-implement is a merge of the following model using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [Cartinoe5930/Llama2_init_Mistral](https://huggingface.co/Cartinoe5930/Llama2_init_Mistral)
For more detailed information, please refer to GitHub Repository.
GitHub Repository: https://github.com/gauss5930/iDUS
## 🧩 Configuration
```yaml
slices:
- sources:
- model: Cartinoe5930/Llama2_init_Mistral
layer_range: [0, 24]
- sources:
- model: Cartinoe5930/Llama2_init_Mistral
layer_range: [8, 32]
merge_method: passthrough
dtype: float16
```
## 🏆 HuggingFace Open LLM Leaderboard
|Model|ARC|HellaSwag|MMLU|TruthfulQA|Winogrande|GSM8K|Average|
|---|---|---|---|---|---|---|---|
|SOLAR-10.7B-DUS-Implementation|59.56|81.18|63.68|40.72|76.48|26.99|58.1|
## 💻 Usage
```python
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "Cartinoe5930/SOLAR-DUS-implement"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```

28
config.json Normal file
View File

@@ -0,0 +1,28 @@
{
"_name_or_path": "Cartinoe5930/Llama2_init_Mistral",
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 4096,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 48,
"num_key_value_heads": 8,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"rope_theta": 10000.0,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.35.2",
"use_cache": true,
"vocab_size": 32000
}

7
generation_config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"_from_model_config": true,
"bos_token_id": 1,
"eos_token_id": 2,
"transformers_version": "4.35.2",
"use_cache": false
}

10
mergekit_config.yml Normal file
View File

@@ -0,0 +1,10 @@
slices:
- sources:
- model: Cartinoe5930/Llama2_init_Mistral
layer_range: [0, 24]
- sources:
- model: Cartinoe5930/Llama2_init_Mistral
layer_range: [8, 32]
merge_method: passthrough
dtype: float16

3
model-00001-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c72735ab0668a22b5e9fc40a12ec40bf83d662574372925bab7b11a79a97667c
size 1889578704

3
model-00002-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:428b95edc6580bb166a317fca62bfbbe3ddd1860d8c70c2323d53c5c5aa502c8
size 1988178416

3
model-00003-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:082d201c999d68cf2955f3bbfd3b4a5926e70f02318bc5a64cc917332e209320
size 1971392656

3
model-00004-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eff5b79e272a7be5f0296cad420477d751d04c27b4219dbb4788ef4f6b8a8516
size 1979814624

3
model-00005-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:88275ea64ede7733396ae1cab29e541d0dd324f095cf4c8b8997bcc8d0db1967
size 1979798480

3
model-00006-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:948b16452c02b4fb3dbe786e432f3e66da98f8dec5a71fb4af39632d685204fa
size 1912689384

3
model-00007-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5630491c641b57c6a4afe6e93199cfd6f2e3c60bbad1fc357dbe856330a9dd49
size 1979781416

3
model-00008-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:55eaeb9ef545710025be19c593546a567aa7e5abd943e417e91d5f97da832c12
size 1979781416

3
model-00009-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aa782024348212fb80a2e7e318cf6ed1b3cc75a4f73f1060b55314ba738ccdc8
size 1889603640

3
model-00010-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b91ad43f25ff6bbd9a404e18ef247353df3e9cf92fc511f17bbd7dd14e429973
size 1946235640

3
model-00011-of-00011.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2b2446cdb829cf6fc2bbb46bf4c08cf0a062ae5ef0fb72dcc8f476f19a30e3c3
size 1946243928

1
model.safetensors.index.json Normal file
View File

File diff suppressed because one or more lines are too long

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

91122
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

40
tokenizer_config.json Normal file
View File

@@ -0,0 +1,40 @@
{
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [],
"bos_token": "<s>",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"model_max_length": 1000000000000000019884624838656,
"pad_token": null,
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}