cstr/Spaetzle-v12-7b
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: cstr/Spaetzle-v12-7b
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-07 09:32:56 +08:00
commit e180d23db5
17 changed files with 91480 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

169
README.md Normal file
View File

@@ -0,0 +1,169 @@
---
tags:
- merge
- mergekit
- lazymergekit
- flemmingmiguel/NeuDist-Ro-7B
- Blizado/discolm-mfto-7b-german-v0.1
- ResplendentAI/Flora_DPO_7B
base_model:
- flemmingmiguel/NeuDist-Ro-7B
- Blizado/discolm-mfto-7b-german-v0.1
- ResplendentAI/Flora_DPO_7B
license: apache-2.0
---
# Spaetzle-v12-7b
Spaetzle-v12-7b is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [flemmingmiguel/NeuDist-Ro-7B](https://huggingface.co/flemmingmiguel/NeuDist-Ro-7B)
* [Blizado/discolm-mfto-7b-german-v0.1](https://huggingface.co/Blizado/discolm-mfto-7b-german-v0.1)
* [ResplendentAI/Flora_DPO_7B](https://huggingface.co/ResplendentAI/Flora_DPO_7B)
* on the basis of [mayflowergmbh/Wiedervereinigung-7b-dpo-laser](https://huggingface.co/mayflowergmbh/Wiedervereinigung-7b-dpo-laser)
As expected, this is a little bit worse in general English tasks over [cstr/spaetzle-v8-7b](https://huggingface.co/cstr/spaetzle-v8-7b), but a tiny little bit better on German tasks, at least some: e.g. it reaches an EQ-Bench (de)
score of 64.81, but only
| Metric |Value|
|---------------------------------|----:|
|Avg. |69.36|
|AI2 Reasoning Challenge (25-Shot)|65.96|
|HellaSwag (10-Shot) |86.16|
|MMLU (5-Shot) |63.48|
|TruthfulQA (0-shot) |57.84|
|Winogrande (5-shot) |80.03|
|GSM8k (5-shot) |62.70|
| Model |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
|--------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|[Spaetzle-v12-7b](https://huggingface.co/cstr/Spaetzle-v12-7b)| 42.64| 74.3| 58.44| 44.44| 54.95|
### AGIEval
| Task |Version| Metric |Value| |Stderr|
|------------------------------|------:|--------|----:|---|-----:|
|agieval_aqua_rat | 0|acc |24.02|± | 2.69|
| | |acc_norm|21.65|± | 2.59|
|agieval_logiqa_en | 0|acc |36.10|± | 1.88|
| | |acc_norm|37.63|± | 1.90|
|agieval_lsat_ar | 0|acc |24.35|± | 2.84|
| | |acc_norm|23.04|± | 2.78|
|agieval_lsat_lr | 0|acc |48.82|± | 2.22|
| | |acc_norm|47.25|± | 2.21|
|agieval_lsat_rc | 0|acc |60.59|± | 2.98|
| | |acc_norm|57.99|± | 3.01|
|agieval_sat_en | 0|acc |76.21|± | 2.97|
| | |acc_norm|74.76|± | 3.03|
|agieval_sat_en_without_passage| 0|acc |46.60|± | 3.48|
| | |acc_norm|45.63|± | 3.48|
|agieval_sat_math | 0|acc |37.27|± | 3.27|
| | |acc_norm|33.18|± | 3.18|
Average: 42.64%
### GPT4All
| Task |Version| Metric |Value| |Stderr|
|-------------|------:|--------|----:|---|-----:|
|arc_challenge| 0|acc |59.13|± | 1.44|
| | |acc_norm|61.26|± | 1.42|
|arc_easy | 0|acc |83.67|± | 0.76|
| | |acc_norm|80.89|± | 0.81|
|boolq | 1|acc |87.83|± | 0.57|
|hellaswag | 0|acc |66.45|± | 0.47|
| | |acc_norm|84.63|± | 0.36|
|openbookqa | 0|acc |37.40|± | 2.17|
| | |acc_norm|45.80|± | 2.23|
|piqa | 0|acc |82.15|± | 0.89|
| | |acc_norm|83.13|± | 0.87|
|winogrande | 0|acc |76.56|± | 1.19|
Average: 74.3%
### TruthfulQA
| Task |Version|Metric|Value| |Stderr|
|-------------|------:|------|----:|---|-----:|
|truthfulqa_mc| 1|mc1 |42.59|± | 1.73|
| | |mc2 |58.44|± | 1.58|
Average: 58.44%
### Bigbench
| Task |Version| Metric |Value| |Stderr|
|------------------------------------------------|------:|---------------------|----:|---|-----:|
|bigbench_causal_judgement | 0|multiple_choice_grade|55.26|± | 3.62|
|bigbench_date_understanding | 0|multiple_choice_grade|64.77|± | 2.49|
|bigbench_disambiguation_qa | 0|multiple_choice_grade|37.60|± | 3.02|
|bigbench_geometric_shapes | 0|multiple_choice_grade|32.31|± | 2.47|
| | |exact_str_match |21.45|± | 2.17|
|bigbench_logical_deduction_five_objects | 0|multiple_choice_grade|31.00|± | 2.07|
|bigbench_logical_deduction_seven_objects | 0|multiple_choice_grade|22.43|± | 1.58|
|bigbench_logical_deduction_three_objects | 0|multiple_choice_grade|53.00|± | 2.89|
|bigbench_movie_recommendation | 0|multiple_choice_grade|40.40|± | 2.20|
|bigbench_navigate | 0|multiple_choice_grade|51.30|± | 1.58|
|bigbench_reasoning_about_colored_objects | 0|multiple_choice_grade|68.50|± | 1.04|
|bigbench_ruin_names | 0|multiple_choice_grade|48.66|± | 2.36|
|bigbench_salient_translation_error_detection | 0|multiple_choice_grade|30.36|± | 1.46|
|bigbench_snarks | 0|multiple_choice_grade|70.17|± | 3.41|
|bigbench_sports_understanding | 0|multiple_choice_grade|70.39|± | 1.45|
|bigbench_temporal_sequences | 0|multiple_choice_grade|31.00|± | 1.46|
|bigbench_tracking_shuffled_objects_five_objects | 0|multiple_choice_grade|21.44|± | 1.16|
|bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|18.29|± | 0.92|
|bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|53.00|± | 2.89|
Average: 44.44%
Average score: 54.95%
Elapsed time: 02:50:51
## 🧩 Configuration
```yaml
models:
- model: mayflowergmbh/Wiedervereinigung-7b-dpo-laser
# no parameters necessary for base model
- model: flemmingmiguel/NeuDist-Ro-7B
parameters:
density: 0.60
weight: 0.30
- model: Blizado/discolm-mfto-7b-german-v0.1
parameters:
density: 0.65
weight: 0.40
- model: ResplendentAI/Flora_DPO_7B
parameters:
density: 0.6
weight: 0.3
merge_method: dare_ties
base_model: mayflowergmbh/Wiedervereinigung-7b-dpo-laser
parameters:
int8_mask: true
dtype: bfloat16
random_seed: 0
tokenizer_source: base
```
## 💻 Usage
```python
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "cstr/Spaetzle-v12-7b"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```

26
config.json Normal file
View File

@@ -0,0 +1,26 @@
{
"_name_or_path": "mayflowergmbh/Wiedervereinigung-7b-dpo-laser",
"architectures": [
"MistralForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 32768,
"model_type": "mistral",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"rms_norm_eps": 1e-05,
"rope_theta": 10000.0,
"sliding_window": 4096,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.37.0",
"use_cache": true,
"vocab_size": 32000
}

23
mergekit_config.yml Normal file
View File

@@ -0,0 +1,23 @@
models:
- model: mayflowergmbh/Wiedervereinigung-7b-dpo-laser
# no parameters necessary for base model
- model: flemmingmiguel/NeuDist-Ro-7B
parameters:
density: 0.60
weight: 0.30
- model: Blizado/discolm-mfto-7b-german-v0.1
parameters:
density: 0.65
weight: 0.40
- model: ResplendentAI/Flora_DPO_7B
parameters:
density: 0.6
weight: 0.3
merge_method: dare_ties
base_model: mayflowergmbh/Wiedervereinigung-7b-dpo-laser
parameters:
int8_mask: true
dtype: bfloat16
random_seed: 0
tokenizer_source: base

3
model-00001-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:05938d549d03e5fe0798d09aa60f7a2d6fc2bfc6ecdd48c3994319b72b1ec5e0
size 1889595352

3
model-00002-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:98e83fde0aa2f17479dd54eb75093b18fbefc50b748035dcd4a8f1dc2688482c
size 1979781416

3
model-00003-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:481ba0f27003cd5202db86e750cf8a31fce98e3ca4379ea30d431f79ee9dc1ff
size 1988195080

3
model-00004-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2fcab0dcfbb60df85a299fbe20c294c95ca5d1da03c34110a8a2b033b668ebb7
size 1937846944

3
model-00005-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:71b7472587d62ec7cce15e39358b281b926328f7015618dd053de3692c1cffa9
size 1988178496

3
model-00006-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9ad2e15a807e0e5e499d6b10870aefd7919f5f6ba6cf4d88f7a8757a887a8ae0
size 1998655576

3
model-00007-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fda1ce38eaf7705d850ec400cda0145bb0a8a75516dbe351fc4b950792a282f6
size 1946243944

3
model-00008-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a2419cab18dc8f4bdfd6e23b29356edb28862cc261c5a430fb7641a2fb918e99
size 755001176

1
model.safetensors.index.json Normal file
View File

File diff suppressed because one or more lines are too long

28
special_tokens_map.json Normal file
View File

@@ -0,0 +1,28 @@
{
"additional_special_tokens": [
"<unk>",
"<s>",
"</s>"
],
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

91122
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

49
tokenizer_config.json Normal file
View File

@@ -0,0 +1,49 @@
{
"add_bos_token": true,
"add_eos_token": false,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [
"<unk>",
"<s>",
"</s>"
],
"bos_token": "<s>",
"chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content']}}{% if (loop.last and add_generation_prompt) or not loop.last %}{{ '<|im_end|>' + '\n'}}{% endif %}{% endfor %}\n{% if add_generation_prompt and messages[-1]['role'] != 'assistant' %}{{ '<|im_start|>assistant\n' }}{% endif %}",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"model_max_length": 1000000000000000019884624838656,
"pad_token": null,
"padding_side": "left",
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"split_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": true
}