初始化项目，由ModelHub XC社区提供模型

Model: sliuau/Llama-3.2-3B_4bits_128group_size
Source: Original Platform

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,57 @@
+{
+  "_name_or_path": "/mnt/home/shihyangl/.cache/huggingface/hub/models--meta-llama--Llama-3.2-3B/snapshots/13afe5124825b4f3751f836b40dafda64c1ed062",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": 128001,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 24,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "quantization_config": {
+    "bits": 4,
+    "checkpoint_format": "gptq",
+    "desc_act": true,
+    "group_size": 128,
+    "lm_head": false,
+    "meta": {
+      "damp_auto_increment": 0.0025,
+      "damp_percent": 0.01,
+      "mse": 0.0,
+      "quantizer": [
+        "gptqmodel:2.0.0-dev"
+      ],
+      "static_groups": false,
+      "true_sequential": true,
+      "uri": "https://github.com/modelcloud/gptqmodel"
+    },
+    "pack_dtype": "int32",
+    "quant_method": "gptq",
+    "sym": true
+  },
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 32.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.49.0",
+  "use_cache": true,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": 128001,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.49.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a672e344fd76c16e7fe2b02e9f789beca1338efe11ee40d04c016bddd2015b5a
+size 2255767328

quant_log.csv

@@ -0,0 +1,197 @@
+layer,module,loss,damp,time
+0,self_attn.k_proj,0.29946,0.01000,0.807
+0,self_attn.v_proj,0.00675,0.01000,0.400
+0,self_attn.q_proj,0.52337,0.01000,0.405
+0,self_attn.o_proj,0.00071,0.01000,0.400
+0,mlp.up_proj,0.35710,0.01000,0.406
+0,mlp.gate_proj,0.40111,0.01000,0.403
+0,mlp.down_proj,0.00492,0.01000,1.127
+1,self_attn.k_proj,0.27738,0.01000,0.411
+1,self_attn.v_proj,0.02259,0.01000,0.408
+1,self_attn.q_proj,0.45884,0.01000,0.407
+1,self_attn.o_proj,0.00270,0.01000,0.410
+1,mlp.up_proj,0.48707,0.01000,0.415
+1,mlp.gate_proj,0.55758,0.01000,0.411
+1,mlp.down_proj,0.11645,0.01000,1.126
+2,self_attn.k_proj,1.48578,0.01000,0.408
+2,self_attn.v_proj,0.11819,0.01000,0.407
+2,self_attn.q_proj,2.27496,0.01000,0.410
+2,self_attn.o_proj,0.00322,0.01000,0.407
+2,mlp.up_proj,0.79056,0.01000,0.412
+2,mlp.gate_proj,0.93259,0.01000,0.412
+2,mlp.down_proj,0.01728,0.01000,1.119
+3,self_attn.k_proj,1.08448,0.01000,0.407
+3,self_attn.v_proj,0.18229,0.01000,0.404
+3,self_attn.q_proj,1.94974,0.01000,0.405
+3,self_attn.o_proj,0.00642,0.01000,0.407
+3,mlp.up_proj,1.02323,0.01000,0.413
+3,mlp.gate_proj,1.35199,0.01000,0.415
+3,mlp.down_proj,0.02748,0.01000,1.121
+4,self_attn.k_proj,0.96294,0.01000,0.408
+4,self_attn.v_proj,0.16686,0.01000,0.407
+4,self_attn.q_proj,1.82451,0.01000,0.408
+4,self_attn.o_proj,0.01145,0.01000,0.410
+4,mlp.up_proj,1.19055,0.01000,0.411
+4,mlp.gate_proj,1.78874,0.01000,0.406
+4,mlp.down_proj,0.03888,0.01000,1.115
+5,self_attn.k_proj,1.57038,0.01000,0.407
+5,self_attn.v_proj,0.15284,0.01000,0.407
+5,self_attn.q_proj,2.49632,0.01000,0.406
+5,self_attn.o_proj,0.01166,0.01000,0.413
+5,mlp.up_proj,1.38595,0.01000,0.410
+5,mlp.gate_proj,1.96359,0.01000,0.411
+5,mlp.down_proj,0.04957,0.01000,1.121
+6,self_attn.k_proj,1.23547,0.01000,0.409
+6,self_attn.v_proj,0.16476,0.01000,0.409
+6,self_attn.q_proj,2.25663,0.01000,0.408
+6,self_attn.o_proj,0.01764,0.01000,0.409
+6,mlp.up_proj,1.44220,0.01000,0.411
+6,mlp.gate_proj,2.01956,0.01000,0.411
+6,mlp.down_proj,0.05499,0.01000,1.120
+7,self_attn.k_proj,1.18016,0.01000,0.408
+7,self_attn.v_proj,0.16010,0.01000,0.407
+7,self_attn.q_proj,1.95431,0.01000,0.406
+7,self_attn.o_proj,0.02251,0.01000,0.407
+7,mlp.up_proj,1.51327,0.01000,0.413
+7,mlp.gate_proj,1.95106,0.01000,0.411
+7,mlp.down_proj,0.06075,0.01000,1.129
+8,self_attn.k_proj,1.46581,0.01000,0.408
+8,self_attn.v_proj,0.19094,0.01000,0.408
+8,self_attn.q_proj,2.43620,0.01000,0.408
+8,self_attn.o_proj,0.02746,0.01000,0.410
+8,mlp.up_proj,1.55921,0.01000,0.411
+8,mlp.gate_proj,2.04694,0.01000,0.411
+8,mlp.down_proj,0.06329,0.01000,1.121
+9,self_attn.k_proj,1.46827,0.01000,0.409
+9,self_attn.v_proj,0.25290,0.01000,0.409
+9,self_attn.q_proj,2.45840,0.01000,0.406
+9,self_attn.o_proj,0.03244,0.01000,0.406
+9,mlp.up_proj,1.57092,0.01000,0.413
+9,mlp.gate_proj,1.97575,0.01000,0.412
+9,mlp.down_proj,0.06281,0.01000,1.127
+10,self_attn.k_proj,1.48356,0.01000,0.408
+10,self_attn.v_proj,0.19161,0.01000,0.406
+10,self_attn.q_proj,2.42431,0.01000,0.408
+10,self_attn.o_proj,0.02705,0.01000,0.410
+10,mlp.up_proj,1.63709,0.01000,0.412
+10,mlp.gate_proj,1.92312,0.01000,0.411
+10,mlp.down_proj,0.06795,0.01000,1.124
+11,self_attn.k_proj,1.17969,0.01000,0.410
+11,self_attn.v_proj,0.24321,0.01000,0.410
+11,self_attn.q_proj,2.09870,0.01000,0.408
+11,self_attn.o_proj,0.03406,0.01000,0.409
+11,mlp.up_proj,1.71431,0.01000,0.414
+11,mlp.gate_proj,1.92656,0.01000,0.413
+11,mlp.down_proj,0.07648,0.01000,1.126
+12,self_attn.k_proj,1.71231,0.01000,0.409
+12,self_attn.v_proj,0.24980,0.01000,0.407
+12,self_attn.q_proj,2.82739,0.01000,0.408
+12,self_attn.o_proj,0.03858,0.01000,0.408
+12,mlp.up_proj,1.77227,0.01000,0.413
+12,mlp.gate_proj,1.96278,0.01000,0.412
+12,mlp.down_proj,0.08552,0.01000,1.123
+13,self_attn.k_proj,1.77657,0.01000,0.409
+13,self_attn.v_proj,0.27030,0.01000,0.408
+13,self_attn.q_proj,2.72148,0.01000,0.408
+13,self_attn.o_proj,0.04566,0.01000,0.409
+13,mlp.up_proj,1.91986,0.01000,0.414
+13,mlp.gate_proj,2.22166,0.01000,0.412
+13,mlp.down_proj,0.10679,0.01000,1.125
+14,self_attn.k_proj,1.56657,0.01000,0.408
+14,self_attn.v_proj,0.32393,0.01000,0.406
+14,self_attn.q_proj,3.15069,0.01000,0.406
+14,self_attn.o_proj,0.05882,0.01000,0.407
+14,mlp.up_proj,2.07567,0.01000,0.411
+14,mlp.gate_proj,2.44127,0.01000,0.411
+14,mlp.down_proj,0.13384,0.01000,1.123
+15,self_attn.k_proj,1.84475,0.01000,0.408
+15,self_attn.v_proj,0.33284,0.01000,0.406
+15,self_attn.q_proj,3.30065,0.01000,0.409
+15,self_attn.o_proj,0.04057,0.01000,0.408
+15,mlp.up_proj,2.18649,0.01000,0.414
+15,mlp.gate_proj,2.79509,0.01000,0.412
+15,mlp.down_proj,0.14568,0.01000,1.124
+16,self_attn.k_proj,2.06072,0.01000,0.410
+16,self_attn.v_proj,0.36225,0.01000,0.408
+16,self_attn.q_proj,3.38652,0.01000,0.407
+16,self_attn.o_proj,0.03020,0.01000,0.408
+16,mlp.up_proj,2.29235,0.01000,0.412
+16,mlp.gate_proj,3.01989,0.01000,0.411
+16,mlp.down_proj,0.14246,0.01000,1.125
+17,self_attn.k_proj,1.91421,0.01000,0.411
+17,self_attn.v_proj,0.37361,0.01000,0.405
+17,self_attn.q_proj,3.36408,0.01000,0.405
+17,self_attn.o_proj,0.02733,0.01000,0.405
+17,mlp.up_proj,2.40934,0.01000,0.408
+17,mlp.gate_proj,3.22124,0.01000,0.407
+17,mlp.down_proj,0.15129,0.01000,1.117
+18,self_attn.k_proj,2.13512,0.01000,0.407
+18,self_attn.v_proj,0.41803,0.01000,0.407
+18,self_attn.q_proj,3.59549,0.01000,0.406
+18,self_attn.o_proj,0.02947,0.01000,0.407
+18,mlp.up_proj,2.59439,0.01000,0.412
+18,mlp.gate_proj,3.41548,0.01000,0.411
+18,mlp.down_proj,0.16239,0.01000,1.123
+19,self_attn.k_proj,2.15473,0.01000,0.409
+19,self_attn.v_proj,0.44331,0.01000,0.407
+19,self_attn.q_proj,3.37589,0.01000,0.406
+19,self_attn.o_proj,0.04266,0.01000,0.407
+19,mlp.up_proj,2.79805,0.01000,0.412
+19,mlp.gate_proj,3.59804,0.01000,0.412
+19,mlp.down_proj,0.19594,0.01000,1.121
+20,self_attn.k_proj,2.17133,0.01000,0.410
+20,self_attn.v_proj,0.53146,0.01000,0.408
+20,self_attn.q_proj,3.50423,0.01000,0.408
+20,self_attn.o_proj,0.03867,0.01000,0.408
+20,mlp.up_proj,2.88202,0.01000,0.412
+20,mlp.gate_proj,3.56189,0.01000,0.411
+20,mlp.down_proj,0.19890,0.01000,1.122
+21,self_attn.k_proj,2.10406,0.01000,0.408
+21,self_attn.v_proj,0.66344,0.01000,0.406
+21,self_attn.q_proj,3.46277,0.01000,0.407
+21,self_attn.o_proj,0.03712,0.01000,0.408
+21,mlp.up_proj,3.08798,0.01000,0.413
+21,mlp.gate_proj,3.84153,0.01000,0.411
+21,mlp.down_proj,0.21074,0.01000,1.124
+22,self_attn.k_proj,1.95893,0.01000,0.409
+22,self_attn.v_proj,0.68801,0.01000,0.407
+22,self_attn.q_proj,3.48500,0.01000,0.406
+22,self_attn.o_proj,0.03578,0.01000,0.410
+22,mlp.up_proj,3.33403,0.01000,0.412
+22,mlp.gate_proj,4.16879,0.01000,0.410
+22,mlp.down_proj,0.23738,0.01000,1.123
+23,self_attn.k_proj,2.16770,0.01000,0.409
+23,self_attn.v_proj,0.64082,0.01000,0.407
+23,self_attn.q_proj,3.44272,0.01000,0.407
+23,self_attn.o_proj,0.05867,0.01000,0.408
+23,mlp.up_proj,3.67624,0.01000,0.412
+23,mlp.gate_proj,4.77981,0.01000,0.411
+23,mlp.down_proj,0.27588,0.01000,1.125
+24,self_attn.k_proj,2.36331,0.01000,0.408
+24,self_attn.v_proj,0.93491,0.01000,0.407
+24,self_attn.q_proj,3.71457,0.01000,0.407
+24,self_attn.o_proj,0.07586,0.01000,0.407
+24,mlp.up_proj,4.14072,0.01000,0.413
+24,mlp.gate_proj,5.50643,0.01000,0.412
+24,mlp.down_proj,0.33052,0.01000,1.126
+25,self_attn.k_proj,1.87947,0.01000,0.415
+25,self_attn.v_proj,0.89642,0.01000,0.407
+25,self_attn.q_proj,3.67515,0.01000,0.407
+25,self_attn.o_proj,0.08351,0.01000,0.407
+25,mlp.up_proj,4.47591,0.01000,0.413
+25,mlp.gate_proj,5.87787,0.01000,0.418
+25,mlp.down_proj,0.43629,0.01000,1.127
+26,self_attn.k_proj,2.02957,0.01000,0.409
+26,self_attn.v_proj,1.15193,0.01000,0.406
+26,self_attn.q_proj,3.48187,0.01000,0.408
+26,self_attn.o_proj,0.14784,0.01000,0.409
+26,mlp.up_proj,4.72783,0.01000,0.412
+26,mlp.gate_proj,6.28630,0.01000,0.412
+26,mlp.down_proj,0.59028,0.01000,1.127
+27,self_attn.k_proj,1.47995,0.01000,0.409
+27,self_attn.v_proj,0.80023,0.01000,0.407
+27,self_attn.q_proj,2.80143,0.01000,0.408
+27,self_attn.o_proj,0.26216,0.01000,0.408
+27,mlp.up_proj,4.78022,0.01000,0.412
+27,mlp.gate_proj,5.72947,0.01000,0.411
+27,mlp.down_proj,1.25502,0.01000,1.126

quantize_config.json

@@ -0,0 +1,21 @@
+{
+  "bits": 4,
+  "group_size": 128,
+  "desc_act": true,
+  "sym": true,
+  "lm_head": false,
+  "quant_method": "gptq",
+  "checkpoint_format": "gptq",
+  "pack_dtype": "int32",
+  "meta": {
+    "quantizer": [
+      "gptqmodel:2.0.0-dev"
+    ],
+    "uri": "https://github.com/modelcloud/gptqmodel",
+    "damp_percent": 0.01,
+    "damp_auto_increment": 0.0025,
+    "static_groups": false,
+    "true_sequential": true,
+    "mse": 0.0
+  }
+}

special_tokens_map.json

@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|finetune_right_pad_id|>"
+}