AbacusResearch/haLLawa4-7b
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: AbacusResearch/haLLawa4-7b
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-14 09:50:43 +08:00
commit c5ceab1eb2
17 changed files with 91462 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

162
README.md Normal file
View File

@@ -0,0 +1,162 @@
---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- mlabonne/Monarch-7B
- paulml/OGNO-7B
- AbacusResearch/haLLAwa3
model-index:
- name: haLLawa4-7b
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: AI2 Reasoning Challenge (25-Shot)
type: ai2_arc
config: ARC-Challenge
split: test
args:
num_few_shot: 25
metrics:
- type: acc_norm
value: 71.5
name: normalized accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=AbacusResearch/haLLawa4-7b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: HellaSwag (10-Shot)
type: hellaswag
split: validation
args:
num_few_shot: 10
metrics:
- type: acc_norm
value: 88.36
name: normalized accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=AbacusResearch/haLLawa4-7b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU (5-Shot)
type: cais/mmlu
config: all
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 64.49
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=AbacusResearch/haLLawa4-7b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: TruthfulQA (0-shot)
type: truthful_qa
config: multiple_choice
split: validation
args:
num_few_shot: 0
metrics:
- type: mc2
value: 74.27
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=AbacusResearch/haLLawa4-7b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: Winogrande (5-shot)
type: winogrande
config: winogrande_xl
split: validation
args:
num_few_shot: 5
metrics:
- type: acc
value: 82.4
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=AbacusResearch/haLLawa4-7b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GSM8k (5-shot)
type: gsm8k
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 70.51
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=AbacusResearch/haLLawa4-7b
name: Open LLM Leaderboard
---
# haLLawa4-7b
haLLawa4-7b is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
* [mlabonne/Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B)
* [paulml/OGNO-7B](https://huggingface.co/paulml/OGNO-7B)
* [AbacusResearch/haLLAwa3](https://huggingface.co/AbacusResearch/haLLAwa3)
## 🧩 Configuration
\```yaml
models:
- model: eren23/ogno-monarch-jaskier-merge-7b
# No parameters necessary for base model
- model: mlabonne/Monarch-7B
#Emphasize the beginning of Vicuna format models
parameters:
weight: 0.5
density: 0.59
- model: paulml/OGNO-7B
parameters:
weight: 0.2
density: 0.55
# Vicuna format
- model: AbacusResearch/haLLAwa3
parameters:
weight: 0.3
density: 0.55
merge_method: dare_ties
base_model: eren23/ogno-monarch-jaskier-merge-7b
parameters:
int8_mask: true
dtype: bfloat16
random_seed: 0
\```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_AbacusResearch__haLLawa4-7b)
| Metric |Value|
|---------------------------------|----:|
|Avg. |75.25|
|AI2 Reasoning Challenge (25-Shot)|71.50|
|HellaSwag (10-Shot) |88.36|
|MMLU (5-Shot) |64.49|
|TruthfulQA (0-shot) |74.27|
|Winogrande (5-shot) |82.40|
|GSM8k (5-shot) |70.51|

26
config.json Normal file
View File

@@ -0,0 +1,26 @@
{
"_name_or_path": "eren23/ogno-monarch-jaskier-merge-7b",
"architectures": [
"MistralForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 32768,
"model_type": "mistral",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"rms_norm_eps": 1e-05,
"rope_theta": 10000.0,
"sliding_window": 4096,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.37.2",
"use_cache": true,
"vocab_size": 32000
}

24
mergekit_config.yml Normal file
View File

@@ -0,0 +1,24 @@
models:
- model: eren23/ogno-monarch-jaskier-merge-7b
# No parameters necessary for base model
- model: mlabonne/Monarch-7B
#Emphasize the beginning of Vicuna format models
parameters:
weight: 0.5
density: 0.59
- model: paulml/OGNO-7B
parameters:
weight: 0.2
density: 0.55
# Vicuna format
- model: AbacusResearch/haLLAwa3
parameters:
weight: 0.3
density: 0.55
merge_method: dare_ties
base_model: eren23/ogno-monarch-jaskier-merge-7b
parameters:
int8_mask: true
dtype: bfloat16
random_seed: 0

3
model-00001-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e9d0821807c2dae5056496ea149fdd0715826fc3f679d58f144000cfcf0a7da1
size 1979773128

3
model-00002-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d937e7685d9f92e07f2a915e7166cc296771b1021cd86aca44410984facc8bef
size 1996567640

3
model-00003-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:464c7061d7b2f93ed9d4069f8737605d376a3752255236f8c0594b086312c7dc
size 1923158216

3
model-00004-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b57ae4189b8704afc71700d63cc1af5a9e62962c57e42512fba6d66bc29e4047
size 1979781464

3
model-00005-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:53f5f170ef630243e18ba32ea9cd529c7fcf4e59c60b85aa9cf034e28e6a1ae2
size 1946243984

3
model-00006-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5b49bae01a2ae83980c5a34ebacf52f62de166d5166da03ce3e7146f3f57e89c
size 1923166040

3
model-00007-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7ea7fd4c258d8c71fd413698f3793ae7b10df2793de8075c55310dfbb035aa7b
size 1946243984

3
model-00008-of-00008.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9567868210c6bccbc6a4fc54c2199cb15f7caa5bfc4e868c09c41b729ac67730
size 788563544

1
model.safetensors.index.json Normal file
View File

File diff suppressed because one or more lines are too long

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

91122
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

42
tokenizer_config.json Normal file
View File

@@ -0,0 +1,42 @@
{
"add_bos_token": true,
"add_eos_token": false,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [],
"bos_token": "<s>",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"model_max_length": 1000000000000000019884624838656,
"pad_token": null,
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}