初始化项目，由ModelHub XC社区提供模型

Model: Euraika/EuroLLM-22B-Instruct-GPTQ
Source: Original Platform

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,161 @@
+---
+license: apache-2.0
+language:
+- en
+- de
+- fr
+- it
+- es
+- pt
+- pl
+- nl
+- tr
+- sv
+- cs
+- el
+- hu
+- ro
+- fi
+- uk
+- sk
+- da
+- bg
+- no
+- sl
+- lt
+- lv
+- et
+- hr
+- ga
+- mt
+base_model: utter-project/EuroLLM-22B-Instruct-2512
+library_name: transformers
+tags:
+- gptq
+- 4bit
+- quantized
+- eurollm
+- multilingual
+- european
+- instruct
+pipeline_tag: text-generation
+model_creator: UTTER Project
+quantized_by: Euraika
+---
+
+# EuroLLM-22B-Instruct GPTQ (4-bit)
+
+This is a 4-bit GPTQ quantized version of [utter-project/EuroLLM-22B-Instruct-2512](https://huggingface.co/utter-project/EuroLLM-22B-Instruct-2512).
+
+## Model Details
+
+| Attribute | Value |
+|-----------|-------|
+| Original Model | EuroLLM-22B-Instruct-2512 |
+| Quantization | GPTQ 4-bit |
+| Group Size | 128 |
+| Activation Order | desc_act=True |
+| Calibration Samples | 512 |
+| Sequence Length | 1024 |
+| Calibration Dataset | OpenHermes-2.5 (ChatML format) |
+| Size | ~14GB |
+
+## Quantization Configuration
+
+```json
+{
+  "bits": 4,
+  "group_size": 128,
+  "desc_act": true,
+  "damp_percent": 0.01,
+  "sym": true,
+  "true_sequential": true
+}
+```
+
+## Quantization Quality
+
+The quantization achieved excellent loss values across all 53 transformer layers:
+- Attention projections: 10⁻⁵ to 10⁻⁶ range
+- MLP layers: 10⁻⁵ range
+- No error explosion or instability
+
+## Languages Supported
+
+EuroLLM supports all 24 official EU languages plus additional European languages:
+Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Irish, Italian, Latvian, Lithuanian, Maltese, Norwegian, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Turkish, Ukrainian
+
+## Usage
+
+### With GPTQModel (Recommended)
+
+```python
+from gptqmodel import GPTQModel
+from transformers import AutoTokenizer
+
+model = GPTQModel.load("Euraika/EuroLLM-22B-Instruct-GPTQ", device="cuda:0")
+tokenizer = AutoTokenizer.from_pretrained("Euraika/EuroLLM-22B-Instruct-GPTQ")
+
+prompt = """<|im_start|>system
+You are a helpful AI assistant.<|im_end|>
+<|im_start|>user
+Explain quantum computing in simple terms.<|im_end|>
+<|im_start|>assistant
+"""
+
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda:0")
+outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### With vLLM
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(model="Euraika/EuroLLM-22B-Instruct-GPTQ", quantization="gptq")
+sampling_params = SamplingParams(temperature=0.7, top_p=0.9, max_tokens=256)
+
+outputs = llm.generate(["Your prompt here"], sampling_params)
+```
+
+### With Transformers + AutoGPTQ
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "Euraika/EuroLLM-22B-Instruct-GPTQ",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("Euraika/EuroLLM-22B-Instruct-GPTQ")
+```
+
+## Chat Template
+
+This model uses the ChatML format:
+
+```
+<|im_start|>system
+{system_message}<|im_end|>
+<|im_start|>user
+{user_message}<|im_end|>
+<|im_start|>assistant
+{assistant_message}<|im_end|>
+```
+
+## Hardware Requirements
+
+- **Minimum**: 16GB VRAM (RTX 4080, RTX A4000)
+- **Recommended**: 24GB VRAM (RTX 4090, RTX A5000, RTX 6000)
+- Supports multi-GPU inference via device_map="auto"
+
+## Credits
+
+- **Original Model**: [UTTER Project](https://huggingface.co/utter-project) - EuroLLM team
+- **Quantization**: Euraika using GPTQModel v5.6.12
+- **Calibration Dataset**: [OpenHermes-2.5](https://huggingface.co/datasets/teknium/OpenHermes-2.5)
+
+## License
+
+Apache 2.0 (same as base model)

chat_template.jinja

@@ -0,0 +1,4 @@
+{% for message in messages %}{% if message['role'] == 'assistant' %}{% set role = 'assistant' %}{% else %}{% set role = message['role'] %}{% endif %}<|im_start|>{{ role }}
+{{ message['content'] | trim }}<|im_end|>
+{% endfor %}{% if add_generation_prompt %}{{'<|im_start|>assistant
+'}}{% endif %}

config.json

@@ -0,0 +1,55 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 4,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 6144,
+  "initializer_range": 0.02,
+  "intermediate_size": 16384,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 48,
+  "num_hidden_layers": 54,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "quantization_config": {
+    "bits": 4,
+    "checkpoint_format": "gptq",
+    "desc_act": true,
+    "format": "gptq",
+    "group_size": 128,
+    "lm_head": false,
+    "meta": {
+      "act_group_aware": false,
+      "damp_auto_increment": 0.01,
+      "damp_percent": 0.01,
+      "gptaq": false,
+      "gptaq_alpha": 0.25,
+      "mse": 0.0,
+      "quantizer": [
+        "gptqmodel:5.6.12"
+      ],
+      "static_groups": false,
+      "true_sequential": true,
+      "uri": "https://github.com/modelcloud/gptqmodel"
+    },
+    "pack_dtype": "int32",
+    "pack_impl": "cpu",
+    "quant_method": "gptq",
+    "sym": true
+  },
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.6",
+  "use_cache": false,
+  "vocab_size": 128000
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 3,
+  "do_sample": true,
+  "eos_token_id": 4,
+  "transformers_version": "4.57.6"
+}

model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5e200a445d63a4ebb7615369b31654450030f87d12d308ff5fb99faccce3c2b
+size 4261210433

model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d9eabc93cb7dd373046298d38c2a1ef7041078fd228b306c10d03b391858086
+size 4260266546

model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:882717fa222455d151d288e1af94dcfd348d79e64fc8ee8e2b39b7197dcca9cf
+size 4260266664

model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:607c9ded4b5140ed1d444a9da2b889eeb8ca68b9935b1d524421e8abe3967ecd
+size 1320307669

quant_log.csv

@@ -0,0 +1,379 @@
+layer,module,loss,samples,damp,time
+0,self_attn.q_proj,0.0000008913,0.01000,6.138
+0,self_attn.v_proj,0.0000000100,0.01000,6.138
+0,self_attn.k_proj,0.0000017337,0.01000,6.164
+0,self_attn.o_proj,0.0000000037,0.01000,1.578
+0,mlp.gate_proj,0.0000001901,0.01000,3.440
+0,mlp.up_proj,0.0000001856,0.01000,3.456
+0,mlp.down_proj,0.0000000166,0.01000,5.620
+1,self_attn.q_proj,0.0000003933,0.01000,4.576
+1,self_attn.k_proj,0.0000002099,0.01000,4.613
+1,self_attn.v_proj,0.0000000120,0.01000,4.671
+1,self_attn.o_proj,0.0000000045,0.01000,1.574
+1,mlp.up_proj,0.0000005750,0.01000,3.232
+1,mlp.gate_proj,0.0000006601,0.01000,3.234
+1,mlp.down_proj,0.0000000506,0.01000,5.622
+2,self_attn.v_proj,0.0000000347,0.01000,4.567
+2,self_attn.q_proj,0.0000005629,0.01000,4.628
+2,self_attn.k_proj,0.0000001695,0.01000,4.645
+2,self_attn.o_proj,0.0000000264,0.01000,1.567
+2,mlp.up_proj,0.0000013546,0.01000,3.520
+2,mlp.gate_proj,0.0000016016,0.01000,3.726
+2,mlp.down_proj,0.0000000479,0.01000,5.599
+3,self_attn.q_proj,0.0000012548,0.01000,4.518
+3,self_attn.k_proj,0.0000003074,0.01000,4.541
+3,self_attn.v_proj,0.0000000672,0.01000,4.547
+3,self_attn.o_proj,0.0000000413,0.01000,1.530
+3,mlp.up_proj,0.0000023189,0.01000,3.232
+3,mlp.gate_proj,0.0000025972,0.01000,3.243
+3,mlp.down_proj,0.0000000890,0.01000,5.554
+4,self_attn.q_proj,0.0000027218,0.01000,4.525
+4,self_attn.k_proj,0.0000008105,0.01000,4.566
+4,self_attn.v_proj,0.0000001387,0.01000,4.572
+4,self_attn.o_proj,0.0000000382,0.01000,1.526
+4,mlp.gate_proj,0.0000035640,0.01000,3.192
+4,mlp.up_proj,0.0000032485,0.01000,3.196
+4,mlp.down_proj,0.0000001381,0.01000,5.408
+5,self_attn.q_proj,0.0000028091,0.01000,4.644
+5,self_attn.k_proj,0.0000008442,0.01000,4.659
+5,self_attn.v_proj,0.0000001876,0.01000,4.669
+5,self_attn.o_proj,0.0000000697,0.01000,1.523
+5,mlp.up_proj,0.0000044974,0.01000,3.193
+5,mlp.gate_proj,0.0000048562,0.01000,3.195
+5,mlp.down_proj,0.0000131428,0.01000,5.395
+6,self_attn.q_proj,0.0000052526,0.01000,4.537
+6,self_attn.k_proj,0.0000018290,0.01000,4.557
+6,self_attn.v_proj,0.0000003574,0.01000,4.575
+6,self_attn.o_proj,0.0000001497,0.01000,1.521
+6,mlp.up_proj,0.0000063462,0.01000,3.194
+6,mlp.gate_proj,0.0000066522,0.01000,3.195
+6,mlp.down_proj,0.0000003866,0.01000,5.493
+7,self_attn.q_proj,0.0000072205,0.01000,4.526
+7,self_attn.v_proj,0.0000005591,0.01000,4.558
+7,self_attn.k_proj,0.0000023830,0.01000,4.754
+7,self_attn.o_proj,0.0000002008,0.01000,1.552
+7,mlp.gate_proj,0.0000095508,0.01000,3.405
+7,mlp.up_proj,0.0000091268,0.01000,3.627
+7,mlp.down_proj,0.0000006677,0.01000,5.470
+8,self_attn.q_proj,0.0000114084,0.01000,4.681
+8,self_attn.k_proj,0.0000031581,0.01000,4.699
+8,self_attn.v_proj,0.0000009229,0.01000,4.716
+8,self_attn.o_proj,0.0000001933,0.01000,2.097
+8,mlp.gate_proj,0.0000121788,0.01000,3.179
+8,mlp.up_proj,0.0000116188,0.01000,3.184
+8,mlp.down_proj,0.0000009437,0.01000,5.483
+9,self_attn.q_proj,0.0000156803,0.01000,4.508
+9,self_attn.v_proj,0.0000011937,0.01000,4.518
+9,self_attn.k_proj,0.0000042363,0.01000,4.746
+9,self_attn.o_proj,0.0000003874,0.01000,1.528
+9,mlp.gate_proj,0.0000159241,0.01000,3.432
+9,mlp.up_proj,0.0000149667,0.01000,3.607
+9,mlp.down_proj,0.0000014772,0.01000,5.454
+10,self_attn.k_proj,0.0000040977,0.01000,4.554
+10,self_attn.q_proj,0.0000145022,0.01000,4.584
+10,self_attn.v_proj,0.0000012288,0.01000,4.587
+10,self_attn.o_proj,0.0000004555,0.01000,1.615
+10,mlp.up_proj,0.0000178941,0.01000,3.440
+10,mlp.gate_proj,0.0000192809,0.01000,3.624
+10,mlp.down_proj,0.0000022575,0.01000,5.459
+11,self_attn.q_proj,0.0000168993,0.01000,4.524
+11,self_attn.v_proj,0.0000013383,0.01000,4.548
+11,self_attn.k_proj,0.0000044527,0.01000,4.641
+11,self_attn.o_proj,0.0000007151,0.01000,1.512
+11,mlp.gate_proj,0.0000223744,0.01000,3.272
+11,mlp.up_proj,0.0000209215,0.01000,3.326
+11,mlp.down_proj,0.0000031872,0.01000,5.535
+12,self_attn.k_proj,0.0000046569,0.01000,4.660
+12,self_attn.q_proj,0.0000181386,0.01000,4.764
+12,self_attn.v_proj,0.0000015525,0.01000,4.821
+12,self_attn.o_proj,0.0000011414,0.01000,1.546
+12,mlp.up_proj,0.0000247787,0.01000,3.290
+12,mlp.gate_proj,0.0000268349,0.01000,3.439
+12,mlp.down_proj,0.0000043331,0.01000,5.436
+13,self_attn.q_proj,0.0000233999,0.01000,4.523
+13,self_attn.v_proj,0.0000017773,0.01000,4.595
+13,self_attn.k_proj,0.0000057049,0.01000,4.649
+13,self_attn.o_proj,0.0000014400,0.01000,1.534
+13,mlp.up_proj,0.0000294066,0.01000,3.293
+13,mlp.gate_proj,0.0000312232,0.01000,3.328
+13,mlp.down_proj,0.0000056300,0.01000,6.001
+14,self_attn.q_proj,0.0000261586,0.01000,5.173
+14,self_attn.k_proj,0.0000066188,0.01000,5.212
+14,self_attn.v_proj,0.0000021216,0.01000,5.231
+14,self_attn.o_proj,0.0000018467,0.01000,1.538
+14,mlp.up_proj,0.0000330952,0.01000,3.399
+14,mlp.gate_proj,0.0000342462,0.01000,3.615
+14,mlp.down_proj,0.0000091821,0.01000,5.479
+15,self_attn.q_proj,0.0000223956,0.01000,5.307
+15,self_attn.v_proj,0.0000020811,0.01000,5.426
+15,self_attn.k_proj,0.0000056626,0.01000,5.445
+15,self_attn.o_proj,0.0000022301,0.01000,1.520
+15,mlp.gate_proj,0.0000347192,0.01000,3.406
+15,mlp.up_proj,0.0000351988,0.01000,3.634
+15,mlp.down_proj,0.0000077845,0.01000,6.912
+16,self_attn.v_proj,0.0000029328,0.01000,4.544
+16,self_attn.k_proj,0.0000076021,0.01000,4.568
+16,self_attn.q_proj,0.0000341581,0.01000,4.649
+16,self_attn.o_proj,0.0000029574,0.01000,1.566
+16,mlp.up_proj,0.0000392853,0.01000,3.208
+16,mlp.gate_proj,0.0000383945,0.01000,3.214
+16,mlp.down_proj,0.0000100357,0.01000,5.512
+17,self_attn.q_proj,0.0000354530,0.01000,4.517
+17,self_attn.v_proj,0.0000032930,0.01000,4.533
+17,self_attn.k_proj,0.0000075661,0.01000,4.551
+17,self_attn.o_proj,0.0000044376,0.01000,1.509
+17,mlp.gate_proj,0.0000417208,0.01000,3.201
+17,mlp.up_proj,0.0000431488,0.01000,3.208
+17,mlp.down_proj,0.0000128390,0.01000,5.423
+18,self_attn.q_proj,0.0000325484,0.01000,4.493
+18,self_attn.v_proj,0.0000036216,0.01000,4.508
+18,self_attn.k_proj,0.0000069052,0.01000,4.522
+18,self_attn.o_proj,0.0000045402,0.01000,1.529
+18,mlp.gate_proj,0.0000436114,0.01000,3.217
+18,mlp.up_proj,0.0000459943,0.01000,3.232
+18,mlp.down_proj,0.0000143081,0.01000,5.420
+19,self_attn.q_proj,0.0000389733,0.01000,4.467
+19,self_attn.v_proj,0.0000042069,0.01000,4.495
+19,self_attn.k_proj,0.0000081779,0.01000,4.523
+19,self_attn.o_proj,0.0000049657,0.01000,1.522
+19,mlp.gate_proj,0.0000456067,0.01000,3.177
+19,mlp.up_proj,0.0000487316,0.01000,3.184
+19,mlp.down_proj,0.0000162543,0.01000,5.404
+20,self_attn.q_proj,0.0000394778,0.01000,5.646
+20,self_attn.v_proj,0.0000048935,0.01000,5.670
+20,self_attn.k_proj,0.0000079125,0.01000,5.712
+20,self_attn.o_proj,0.0000074877,0.01000,1.554
+20,mlp.gate_proj,0.0000466254,0.01000,3.202
+20,mlp.up_proj,0.0000507956,0.01000,3.200
+20,mlp.down_proj,0.0000190122,0.01000,5.437
+21,self_attn.q_proj,0.0000425122,0.01000,4.633
+21,self_attn.v_proj,0.0000054937,0.01000,4.734
+21,self_attn.k_proj,0.0000084130,0.01000,4.829
+21,self_attn.o_proj,0.0000087002,0.01000,1.527
+21,mlp.gate_proj,0.0000512173,0.01000,3.390
+21,mlp.up_proj,0.0000561848,0.01000,3.614
+21,mlp.down_proj,0.0000240307,0.01000,5.434
+22,self_attn.q_proj,0.0000426710,0.01000,4.411
+22,self_attn.k_proj,0.0000084810,0.01000,4.457
+22,self_attn.v_proj,0.0000054290,0.01000,4.463
+22,self_attn.o_proj,0.0000088443,0.01000,2.071
+22,mlp.gate_proj,0.0000557083,0.01000,3.178
+22,mlp.up_proj,0.0000607494,0.01000,3.185
+22,mlp.down_proj,0.0000272708,0.01000,5.403
+23,self_attn.q_proj,0.0000450702,0.01000,4.429
+23,self_attn.k_proj,0.0000091971,0.01000,4.454
+23,self_attn.v_proj,0.0000064182,0.01000,4.456
+23,self_attn.o_proj,0.0000108490,0.01000,1.528
+23,mlp.gate_proj,0.0000583830,0.01000,3.210
+23,mlp.up_proj,0.0000642057,0.01000,3.216
+23,mlp.down_proj,0.0000326036,0.01000,5.404
+24,self_attn.q_proj,0.0000472265,0.01000,4.426
+24,self_attn.v_proj,0.0000070015,0.01000,4.464
+24,self_attn.k_proj,0.0000094841,0.01000,4.472
+24,self_attn.o_proj,0.0000130456,0.01000,1.549
+24,mlp.gate_proj,0.0000658193,0.01000,3.195
+24,mlp.up_proj,0.0000721073,0.01000,3.199
+24,mlp.down_proj,0.0000394027,0.01000,5.393
+25,self_attn.k_proj,0.0000099797,0.01000,4.542
+25,self_attn.v_proj,0.0000084211,0.01000,4.574
+25,self_attn.q_proj,0.0000518252,0.01000,4.751
+25,self_attn.o_proj,0.0000112846,0.01000,1.528
+25,mlp.up_proj,0.0000784803,0.01000,3.309
+25,mlp.gate_proj,0.0000712152,0.01000,3.409
+25,mlp.down_proj,0.0000474415,0.01000,5.389
+26,self_attn.q_proj,0.0000525890,0.01000,4.498
+26,self_attn.k_proj,0.0000100229,0.01000,4.550
+26,self_attn.v_proj,0.0000088433,0.01000,4.573
+26,self_attn.o_proj,0.0000152159,0.01000,1.528
+26,mlp.gate_proj,0.0000789596,0.01000,3.292
+26,mlp.up_proj,0.0000868882,0.01000,3.367
+26,mlp.down_proj,0.0000558948,0.01000,6.902
+27,self_attn.k_proj,0.0000098061,0.01000,5.597
+27,self_attn.q_proj,0.0000516858,0.01000,5.658
+27,self_attn.v_proj,0.0000085428,0.01000,5.663
+27,self_attn.o_proj,0.0000154769,0.01000,1.519
+27,mlp.gate_proj,0.0000842932,0.01000,3.147
+27,mlp.up_proj,0.0000933967,0.01000,3.148
+27,mlp.down_proj,0.0000697337,0.01000,6.840
+28,self_attn.k_proj,0.0000096209,0.01000,4.523
+28,self_attn.v_proj,0.0000105874,0.01000,4.536
+28,self_attn.q_proj,0.0000543645,0.01000,4.558
+28,self_attn.o_proj,0.0000256079,0.01000,1.521
+28,mlp.gate_proj,0.0000956455,0.01000,3.191
+28,mlp.up_proj,0.0001060709,0.01000,3.192
+28,mlp.down_proj,0.0000868129,0.01000,5.381
+29,self_attn.k_proj,0.0000105517,0.01000,4.459
+29,self_attn.q_proj,0.0000600444,0.01000,4.473
+29,self_attn.v_proj,0.0000127455,0.01000,4.478
+29,self_attn.o_proj,0.0000294386,0.01000,1.498
+29,mlp.up_proj,0.0001240473,0.01000,3.173
+29,mlp.gate_proj,0.0001110740,0.01000,3.178
+29,mlp.down_proj,0.0001162564,0.01000,5.380
+30,self_attn.q_proj,0.0000613605,0.01000,4.478
+30,self_attn.v_proj,0.0000136211,0.01000,4.521
+30,self_attn.k_proj,0.0000101663,0.01000,4.528
+30,self_attn.o_proj,0.0000277450,0.01000,1.515
+30,mlp.up_proj,0.0001366665,0.01000,3.192
+30,mlp.gate_proj,0.0001220095,0.01000,3.194
+30,mlp.down_proj,0.0001533648,0.01000,5.372
+31,self_attn.q_proj,0.0000629197,0.01000,4.667
+31,self_attn.v_proj,0.0000150755,0.01000,4.716
+31,self_attn.k_proj,0.0000100236,0.01000,4.764
+31,self_attn.o_proj,0.0000394865,0.01000,1.541
+31,mlp.gate_proj,0.0001309123,0.01000,3.410
+31,mlp.up_proj,0.0001460293,0.01000,3.633
+31,mlp.down_proj,0.0001916053,0.01000,5.504
+32,self_attn.q_proj,0.0000688052,0.01000,4.727
+32,self_attn.v_proj,0.0000199689,0.01000,4.771
+32,self_attn.k_proj,0.0000101955,0.01000,4.908
+32,self_attn.o_proj,0.0000414511,0.01000,1.675
+32,mlp.gate_proj,0.0001506769,0.01000,3.193
+32,mlp.up_proj,0.0001681236,0.01000,3.198
+32,mlp.down_proj,0.0002321370,0.01000,5.405
+33,self_attn.q_proj,0.0000790759,0.01000,5.877
+33,self_attn.k_proj,0.0000119265,0.01000,6.008
+33,self_attn.v_proj,0.0000245031,0.01000,6.027
+33,self_attn.o_proj,0.0000481164,0.01000,1.537
+33,mlp.gate_proj,0.0001631006,0.01000,3.384
+33,mlp.up_proj,0.0001819707,0.01000,3.558
+33,mlp.down_proj,0.0002729572,0.01000,5.446
+34,self_attn.q_proj,0.0000711151,0.01000,4.468
+34,self_attn.k_proj,0.0000106548,0.01000,4.487
+34,self_attn.v_proj,0.0000241654,0.01000,4.530
+34,self_attn.o_proj,0.0000437769,0.01000,2.056
+34,mlp.up_proj,0.0001999646,0.01000,3.210
+34,mlp.gate_proj,0.0001800606,0.01000,3.243
+34,mlp.down_proj,0.0002995736,0.01000,6.643
+35,self_attn.v_proj,0.0000267306,0.01000,4.621
+35,self_attn.q_proj,0.0000763110,0.01000,4.682
+35,self_attn.k_proj,0.0000114702,0.01000,4.714
+35,self_attn.o_proj,0.0000432579,0.01000,1.510
+35,mlp.up_proj,0.0002159799,0.01000,3.189
+35,mlp.gate_proj,0.0001948314,0.01000,3.190
+35,mlp.down_proj,0.0003204718,0.01000,5.387
+36,self_attn.k_proj,0.0000131923,0.01000,4.720
+36,self_attn.v_proj,0.0000260484,0.01000,4.742
+36,self_attn.q_proj,0.0000835342,0.01000,4.761
+36,self_attn.o_proj,0.0000440268,0.01000,1.542
+36,mlp.gate_proj,0.0002087337,0.01000,3.193
+36,mlp.up_proj,0.0002316518,0.01000,3.194
+36,mlp.down_proj,0.0003474331,0.01000,5.490
+37,self_attn.k_proj,0.0000136178,0.01000,4.508
+37,self_attn.q_proj,0.0000840837,0.01000,4.556
+37,self_attn.v_proj,0.0000244343,0.01000,4.574
+37,self_attn.o_proj,0.0000274780,0.01000,1.536
+37,mlp.gate_proj,0.0002214634,0.01000,3.221
+37,mlp.up_proj,0.0002454306,0.01000,3.239
+37,mlp.down_proj,0.0003771611,0.01000,5.486
+38,self_attn.q_proj,0.0000895742,0.01000,4.848
+38,self_attn.k_proj,0.0000140327,0.01000,4.926
+38,self_attn.v_proj,0.0000301099,0.01000,4.934
+38,self_attn.o_proj,0.0000425441,0.01000,1.517
+38,mlp.up_proj,0.0002633508,0.01000,3.263
+38,mlp.gate_proj,0.0002368640,0.01000,3.270
+38,mlp.down_proj,0.0003942181,0.01000,6.156
+39,self_attn.k_proj,0.0000140179,0.01000,4.522
+39,self_attn.v_proj,0.0000313321,0.01000,4.537
+39,self_attn.q_proj,0.0000908200,0.01000,4.553
+39,self_attn.o_proj,0.0000534199,0.01000,1.533
+39,mlp.up_proj,0.0002788311,0.01000,3.178
+39,mlp.gate_proj,0.0002501276,0.01000,3.185
+39,mlp.down_proj,0.0004236163,0.01000,5.421
+40,self_attn.q_proj,0.0000999253,0.01000,4.997
+40,self_attn.v_proj,0.0000328509,0.01000,5.026
+40,self_attn.k_proj,0.0000157782,0.01000,5.058
+40,self_attn.o_proj,0.0000260055,0.01000,1.529
+40,mlp.up_proj,0.0002908479,0.01000,3.212
+40,mlp.gate_proj,0.0002605559,0.01000,3.213
+40,mlp.down_proj,0.0004451857,0.01000,5.371
+41,self_attn.q_proj,0.0001023891,0.01000,5.047
+41,self_attn.v_proj,0.0000318231,0.01000,5.110
+41,self_attn.k_proj,0.0000167836,0.01000,5.250
+41,self_attn.o_proj,0.0000375266,0.01000,1.543
+41,mlp.gate_proj,0.0002773313,0.01000,3.425
+41,mlp.up_proj,0.0003107931,0.01000,3.632
+41,mlp.down_proj,0.0004739352,0.01000,5.388
+42,self_attn.q_proj,0.0001094544,0.01000,4.987
+42,self_attn.v_proj,0.0000319344,0.01000,5.083
+42,self_attn.k_proj,0.0000180468,0.01000,5.092
+42,self_attn.o_proj,0.0000348270,0.01000,2.082
+42,mlp.gate_proj,0.0002944671,0.01000,3.209
+42,mlp.up_proj,0.0003314464,0.01000,3.216
+42,mlp.down_proj,0.0005044182,0.01000,6.817
+43,self_attn.q_proj,0.0001125766,0.01000,4.619
+43,self_attn.v_proj,0.0000335141,0.01000,4.636
+43,self_attn.k_proj,0.0000183378,0.01000,4.664
+43,self_attn.o_proj,0.0000649387,0.01000,1.550
+43,mlp.gate_proj,0.0003134010,0.01000,3.325
+43,mlp.up_proj,0.0003545903,0.01000,3.343
+43,mlp.down_proj,0.0005684485,0.01000,5.398
+44,self_attn.q_proj,0.0001224409,0.01000,4.553
+44,self_attn.v_proj,0.0000358007,0.01000,4.602
+44,self_attn.k_proj,0.0000201812,0.01000,4.633
+44,self_attn.o_proj,0.0000679611,0.01000,1.541
+44,mlp.gate_proj,0.0003327907,0.01000,3.233
+44,mlp.up_proj,0.0003794263,0.01000,3.264
+44,mlp.down_proj,0.0006130660,0.01000,5.382
+45,self_attn.q_proj,0.0001163339,0.01000,4.507
+45,self_attn.v_proj,0.0000354407,0.01000,4.566
+45,self_attn.k_proj,0.0000192487,0.01000,4.578
+45,self_attn.o_proj,0.0000531762,0.01000,2.069
+45,mlp.gate_proj,0.0003525082,0.01000,3.193
+45,mlp.up_proj,0.0004050766,0.01000,3.199
+45,mlp.down_proj,0.0006635655,0.01000,5.426
+46,self_attn.v_proj,0.0000432381,0.01000,4.585
+46,self_attn.k_proj,0.0000209618,0.01000,4.626
+46,self_attn.q_proj,0.0001321122,0.01000,4.648
+46,self_attn.o_proj,0.0000832375,0.01000,1.539
+46,mlp.gate_proj,0.0003708685,0.01000,3.260
+46,mlp.up_proj,0.0004293002,0.01000,3.261
+46,mlp.down_proj,0.0007252367,0.01000,5.401
+47,self_attn.q_proj,0.0001240910,0.01000,4.474
+47,self_attn.v_proj,0.0000418121,0.01000,4.520
+47,self_attn.k_proj,0.0000194023,0.01000,4.527
+47,self_attn.o_proj,0.0001032710,0.01000,1.540
+47,mlp.up_proj,0.0004477853,0.01000,3.191
+47,mlp.gate_proj,0.0003832070,0.01000,3.192
+47,mlp.down_proj,0.0007992553,0.01000,5.531
+48,self_attn.q_proj,0.0001193619,0.01000,4.535
+48,self_attn.v_proj,0.0000391486,0.01000,4.563
+48,self_attn.k_proj,0.0000189018,0.01000,4.652
+48,self_attn.o_proj,0.0000801096,0.01000,1.545
+48,mlp.up_proj,0.0004526175,0.01000,3.296
+48,mlp.gate_proj,0.0003852706,0.01000,3.302
+48,mlp.down_proj,0.0008187147,0.01000,5.403
+49,self_attn.k_proj,0.0000191849,0.01000,4.540
+49,self_attn.q_proj,0.0001215081,0.01000,4.546
+49,self_attn.v_proj,0.0000401676,0.01000,4.561
+49,self_attn.o_proj,0.0000807410,0.01000,1.535
+49,mlp.up_proj,0.0004751309,0.01000,3.198
+49,mlp.gate_proj,0.0004003226,0.01000,3.204
+49,mlp.down_proj,0.0009328053,0.01000,5.450
+50,self_attn.q_proj,0.0001148517,0.01000,4.624
+50,self_attn.k_proj,0.0000170242,0.01000,4.670
+50,self_attn.v_proj,0.0000410779,0.01000,4.674
+50,self_attn.o_proj,0.0001186695,0.01000,1.545
+50,mlp.gate_proj,0.0003979134,0.01000,3.191
+50,mlp.up_proj,0.0004742058,0.01000,3.198
+50,mlp.down_proj,0.0010534921,0.01000,5.385
+51,self_attn.q_proj,0.0000908077,0.01000,4.519
+51,self_attn.v_proj,0.0000311765,0.01000,4.532
+51,self_attn.k_proj,0.0000149527,0.01000,4.540
+51,self_attn.o_proj,0.0000697661,0.01000,1.545
+51,mlp.up_proj,0.0004750221,0.01000,3.179
+51,mlp.gate_proj,0.0003956558,0.01000,3.189
+51,mlp.down_proj,0.0010364314,0.01000,5.418
+52,self_attn.q_proj,0.0000817674,0.01000,4.758
+52,self_attn.v_proj,0.0000265048,0.01000,4.802
+52,self_attn.k_proj,0.0000131860,0.01000,4.860
+52,self_attn.o_proj,0.0000599252,0.01000,1.541
+52,mlp.gate_proj,0.0003909920,0.01000,3.215
+52,mlp.up_proj,0.0004676292,0.01000,3.258
+52,mlp.down_proj,0.0011807736,0.01000,5.416
+53,self_attn.q_proj,0.0000785972,0.01000,4.487
+53,self_attn.v_proj,0.0000265034,0.01000,4.546
+53,self_attn.k_proj,0.0000117989,0.01000,4.755
+53,self_attn.o_proj,0.0000894996,0.01000,1.527
+53,mlp.up_proj,0.0004224250,0.01000,3.457
+53,mlp.gate_proj,0.0003560778,0.01000,3.463
+53,mlp.down_proj,0.0022309641,0.01000,5.440

quantize_config.json

@@ -0,0 +1,26 @@
+{
+  "bits": 4,
+  "group_size": 128,
+  "desc_act": true,
+  "sym": true,
+  "lm_head": false,
+  "quant_method": "gptq",
+  "checkpoint_format": "gptq",
+  "pack_dtype": "int32",
+  "meta": {
+    "quantizer": [
+      "gptqmodel:5.6.12"
+    ],
+    "uri": "https://github.com/modelcloud/gptqmodel",
+    "damp_percent": 0.01,
+    "damp_auto_increment": 0.01,
+    "static_groups": false,
+    "true_sequential": true,
+    "mse": 0.0,
+    "gptaq": false,
+    "gptaq_alpha": 0.25,
+    "act_group_aware": false
+  },
+  "pack_impl": "cpu",
+  "format": "gptq"
+}

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25c2a74f04a35dff665adfc22e18af151c4d559e763e1c1697180437253a3339
+size 15783174

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42957aec88804838805b335da1bbcce9d116024a20c6c4b56757648b9e348254
+size 2408875