RichardErkhov/Pretergeek_-_OpenChat-3.5-0106_10.7B_48Layers-Interleaved-gguf

Files

ModelHub XC 93455e5a5d 初始化项目，由ModelHub XC社区提供模型

Model: RichardErkhov/Pretergeek_-_OpenChat-3.5-0106_10.7B_48Layers-Interleaved-gguf
Source: Original Platform

2026-04-11 19:18:04 +08:00

15 KiB

Raw Permalink Blame History

Quantization made by Richard Erkhov.

Github

Discord

Request more models

OpenChat-3.5-0106_10.7B_48Layers-Interleaved - GGUF

Model creator: https://huggingface.co/Pretergeek/
Original model: https://huggingface.co/Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved/

Name	Quant method	Size
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q2_K.gguf	Q2_K	3.73GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q3_K_S.gguf	Q3_K_S	4.34GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q3_K.gguf	Q3_K	2.31GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q3_K_M.gguf	Q3_K_M	4.84GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q3_K_L.gguf	Q3_K_L	5.26GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.IQ4_XS.gguf	IQ4_XS	5.43GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q4_0.gguf	Q4_0	5.66GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.IQ4_NL.gguf	IQ4_NL	5.72GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q4_K_S.gguf	Q4_K_S	5.7GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q4_K.gguf	Q4_K	6.02GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q4_K_M.gguf	Q4_K_M	6.02GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q4_1.gguf	Q4_1	6.27GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q5_0.gguf	Q5_0	6.89GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q5_K_S.gguf	Q5_K_S	6.89GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q5_K.gguf	Q5_K	7.08GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q5_K_M.gguf	Q5_K_M	7.08GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q5_1.gguf	Q5_1	7.51GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q6_K.gguf	Q6_K	8.2GB
OpenChat-3.5-0106_10.7B_48Layers-Interleaved.Q8_0.gguf	Q8_0	10.62GB

Original model description:

license: apache-2.0 library_name: transformers tags:

mergekit
merge base_model:
openchat/openchat-3.5-0106 model-index:
name: OpenChat-3.5-0106_10.7B_48Layers-Interleaved results:
- task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics:
  - type: inst_level_strict_acc and prompt_level_strict_acc value: 59.61 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics:
  - type: acc_norm value: 24.06 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics:
  - type: exact_match value: 6.8 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics:
  - type: acc_norm value: 7.27 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics:
  - type: acc_norm value: 11.78 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved name: Open LLM Leaderboard
- task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics:
  - type: acc value: 25.54 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=Pretergeek/OpenChat-3.5-0106_10.7B_48Layers-Interleaved name: Open LLM Leaderboard

Buy me a Ko-Fi • Support my work using Patreon

OpenChat-3.5-0106_10.7B_48Layers-Interleaved

This is NOT your usual frankenmerge created using mergekit.

Merge Details

Merge Method

This model was merged using the passthrough merge method, but employing the Block Expansion method described in the paper LLaMA Pro: Progressive LLaMA with Block Expansion.

The authors of the paper added new layers interleaved in between the original layers of the model, setting the parameters of the o_proj and down_proj layers to zero. This effectively adds layers that will just output their input (as if they were "transparent") allowing the model to remain functional even without further training. These new layers can then be targeted during training or fine-tuning without risking catastrophic forgetting, if you follow the author's training method to freeze the original layers and only train the new layers.

This model has not yet received additional training, so it should perform close to the original model.

Models Merged

The following models were included in the merge:

openchat/openchat-3.5-0106

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [0, 2]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [1, 2]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [2, 4]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [3, 4]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [4, 6]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [5, 6]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [6, 8]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [7, 8]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [8, 10]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [9, 10]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [10, 12]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [11, 12]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [12, 14]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [13, 14]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [14, 16]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [15, 16]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [16, 18]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [17, 18]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [18, 20]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [19, 20]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [20, 22]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [21, 22]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [22, 24]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [23, 24]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [24, 26]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [25, 26]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [26, 28]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [27, 28]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [28, 30]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [29, 30]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [30, 32]
  - sources:
    - model: openchat/openchat-3.5-0106
      layer_range: [31, 32]
      parameters:
        scale:
          - filter: o_proj
            value: 0.0
          - filter: down_proj
            value: 0.0
          - value: 1.0
merge_method: passthrough
dtype: bfloat16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	22.51
IFEval (0-Shot)	59.61
BBH (3-Shot)	24.06
MATH Lvl 5 (4-Shot)	6.80
GPQA (0-shot)	7.27
MuSR (0-shot)	11.78
MMLU-PRO (5-shot)	25.54

Citation

@misc{wu2024llamaproprogressivellama,
      title={LLaMA Pro: Progressive LLaMA with Block Expansion}, 
      author={Chengyue Wu and Yukang Gan and Yixiao Ge and Zeyu Lu and Jiahao Wang and Ye Feng and Ying Shan and Ping Luo},
      year={2024},
      eprint={2401.02415},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2401.02415}, 
}

15 KiB Raw Permalink Blame History