初始化项目，由ModelHub XC社区提供模型

Model: duyntnet/Qwen2.5-Coder-7B-Instruct-imatrix-GGUF Source: Original Platform
2026-05-30 12:21:16 +08:00
commit 81962c6024
29 changed files with 252 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,62 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen2.5-Coder-7B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
--- a/Qwen2.5-Coder-7B-Instruct-IQ1_M.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ1_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7bd45d04e42ac8eec774ca6a745b8d2fd66e7ccf4a9da6c04ac91968bf16c85
+size 2042197152
--- a/Qwen2.5-Coder-7B-Instruct-IQ1_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ1_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:596dae781245e65b37b452c1475b5871c4486c00deebea51d692dddb9d17a08b
+size 1903668384
--- a/Qwen2.5-Coder-7B-Instruct-IQ2_M.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ2_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7246b8e6e6604700268b89f0bdbbeff5f23be90b87b285fa9734c5c489e12d3
+size 2780343456
--- a/Qwen2.5-Coder-7B-Instruct-IQ2_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ2_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecf02648fab3a8a73d86025ca9c03299b508448199846d4a48fafcfc9a2b4447
+size 2595638432
--- a/Qwen2.5-Coder-7B-Instruct-IQ2_XS.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ2_XS.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d3e8f4b4c88f1be837540b0cebdfaf451d9352915214a7a410aa5a5ec0453f0
+size 2469022880
--- a/Qwen2.5-Coder-7B-Instruct-IQ2_XXS.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ2_XXS.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ef5d11082d774fa47e38b4dd43516862ddd7b82adc532513ef091d21520a2ed
+size 2273078432
--- a/Qwen2.5-Coder-7B-Instruct-IQ3_M.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ3_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:851af6af5d3d21c7377efca571830868886e8a096584df45b109c414414522a9
+size 3574013088
--- a/Qwen2.5-Coder-7B-Instruct-IQ3_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ3_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17547af54d54c4871b8cc1232150f53b5b779b8cc138ab10c967e3990d335135
+size 3499193504
--- a/Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ3_XS.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d37b78ea47ecab4807ba0346159191dc5df8ccd1aa47bebfd8cb16645dc413b4
+size 3346257056
--- a/Qwen2.5-Coder-7B-Instruct-IQ3_XXS.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ3_XXS.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe921676a0bb893b423ca72dad27c30bb4eded7f50e57fa6b2d0a80777e46ead
+size 3114515616
--- a/Qwen2.5-Coder-7B-Instruct-IQ4_NL.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ4_NL.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dafad5f6661d94fdb1f0025a492de69999653e4f9a61f9c978508d4155e30ce3
+size 4437814432
--- a/Qwen2.5-Coder-7B-Instruct-IQ4_XS.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-IQ4_XS.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f823fce748ae5ca14e568e4ed41b26812a391d0801e2805ae307e13c223f4294
+size 4218473632
--- a/Qwen2.5-Coder-7B-Instruct-Q2_K.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q2_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8dd4ee4dc1c5c0aea694d1aa0f2738253a9bb2bd800cb78299ea72a1c8d7e0
+size 3015941280
--- a/Qwen2.5-Coder-7B-Instruct-Q2_K_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q2_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87e564898bcdebb10ea6a280030b51d98e9ebbae4c2493a0adbcb650fbed05f6
+size 2834074784
--- a/Qwen2.5-Coder-7B-Instruct-Q3_K_L.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q3_K_L.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50573e8a56a71ad802a366d4f7953288cebd7612c19374d2e97375b4e4504a4c
+size 4088460448
--- a/Qwen2.5-Coder-7B-Instruct-Q3_K_M.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q3_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0eec65c103ac34314cca4369651fa00b732e86076b45f10c47935a5c2364c16c
+size 3808392352
--- a/Qwen2.5-Coder-7B-Instruct-Q3_K_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q3_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8853d5d6c7fa6cbd7d14503e183de8dcddb208fcb10da3dfa18c879b3ca96d90
+size 3492369568
--- a/Qwen2.5-Coder-7B-Instruct-Q4_0.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q4_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:816a631fd8e796cbfadf8e486add6b1725c51399d7367224560976bbb88ec8fa
+size 4444122272
--- a/Qwen2.5-Coder-7B-Instruct-Q4_1.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q4_1.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bd2c1fb817bdec1599c3e2c378c39a6e4a754bb1709ee560b6a5b76760f1140
+size 4873284768
--- a/Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d19d30d2b8c219cd2022d75c76eae79b9f7532917f851001cd0aa087e0b2c7e
+size 4683074720
--- a/Qwen2.5-Coder-7B-Instruct-Q4_K_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q4_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c4d8742f0f622980c7470a3d4204a1ef94c4671ad9d5d01bba2a90ac2184903
+size 4457770144
--- a/Qwen2.5-Coder-7B-Instruct-Q5_0.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q5_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5c1094db9e018003818fc285627153c471ed0199c19f445a783910fcf6a7b45
+size 5327908000
--- a/Qwen2.5-Coder-7B-Instruct-Q5_1.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q5_1.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:418dc072833a54a686771638905d25258bbd608246cacf88fd7710964988e10f
+size 5757070496
--- a/Qwen2.5-Coder-7B-Instruct-Q5_K_M.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f75ae8efe4d15917c885350ea70f49535222c47e73945c76d8f76eb242864c0
+size 5444832416
--- a/Qwen2.5-Coder-7B-Instruct-Q5_K_S.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q5_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39b51b82adad74c09c1d6d1c25a49b8445d99db56945d22bba0dc2c8f80fd99d
+size 5315177632
--- a/Qwen2.5-Coder-7B-Instruct-Q6_K.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:018ebbc3c88805e83309f80da3931474b40323180b911eba206a31f8b2110e33
+size 6254199968
--- a/Qwen2.5-Coder-7B-Instruct-Q8_0.gguf
+++ b/Qwen2.5-Coder-7B-Instruct-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3ed1587d44957b1d0469454783e735eb1c4f460398d841991c7c8c9f7131238
+size 8098526368
--- a/README.md
+++ b/README.md
@@ -0,0 +1,109 @@
+---
+license: other
+language:
+- en
+pipeline_tag: text-generation
+inference: false
+tags:
+- transformers
+- gguf
+- imatrix
+- Qwen2.5-Coder-7B-Instruct
+---
+Quantizations of https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct
+
+
+### Inference Clients/UIs
+* [llama.cpp](https://github.com/ggerganov/llama.cpp)
+* [KoboldCPP](https://github.com/LostRuins/koboldcpp)
+* [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
+* [ollama](https://github.com/ollama/ollama)
+
+
+---
+
+# From original readme
+
+Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). For Qwen2.5-Coder, we release three base language models and instruction-tuned language models, 1.5, 7 and 32 (coming soon) billion parameters. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
+
+- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. 
+- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
+- **Long-context Support** up to 128K tokens.
+
+**This repo contains the instruction-tuned 7B Qwen2.5-Coder model**, which has the following features:
+- Type: Causal Language Models
+- Training Stage: Pretraining & Post-training
+- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
+- Number of Parameters: 7.61B
+- Number of Paramaters (Non-Embedding): 6.53B
+- Number of Layers: 28
+- Number of Attention Heads (GQA): 28 for Q and 4 for KV
+- Context Length: Full 131,072 tokens
+  - Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts.
+  
+For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
+
+## Requirements
+
+The code of Qwen2.5-Coder has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
+
+With `transformers<4.37.0`, you will encounter the following error:
+```
+KeyError: 'qwen2'
+```
+
+## Quickstart
+
+Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "Qwen/Qwen2.5-Coder-7B-Instruct"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "write a quick sort algorithm."
+messages = [
+    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+```
+
+### Processing Long Texts
+
+The current `config.json` is set for context length up to 32,768 tokens.
+To handle extensive inputs exceeding 32,768 tokens, we utilize [YaRN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
+
+For supported frameworks, you could add the following to `config.json` to enable YaRN:
+```json
+{
+  ...,
+  "rope_scaling": {
+    "factor": 4.0,
+    "original_max_position_embeddings": 32768,
+    "type": "yarn"
+  }
+}
+```