ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-19 05:02:16 +08:00
commit ae5c2ea4af
32 changed files with 337 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
.gitattributes vendored Normal file
View File

@@ -0,0 +1,64 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-F16.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-Coder-0.5B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

3
Qwen2.5-Coder-0.5B-Instruct-F16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f44732b10856f1137245774806a1ba1d762cd4ae6bd559aabe9e6503cad0b466
size 994156928

3
Qwen2.5-Coder-0.5B-Instruct-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:49bef31f1f643d31b7f76171e12c30ebcea822711921c9df00b495353216f83b
size 317974720

3
Qwen2.5-Coder-0.5B-Instruct-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73d05cde348da6fc394e503a094da5931640d6a11182dcb8b581e49d5f9043b9
size 315829696

3
Qwen2.5-Coder-0.5B-Instruct-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f36614878ab02784ae744da74b0169ea60bc4a3ac1dba79a87758c675234561d
size 328597696

3
Qwen2.5-Coder-0.5B-Instruct-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ce154a05457340a1a12c19d5207a1d9d368e23db7745142b131442e64785424f
size 325737664

3
Qwen2.5-Coder-0.5B-Instruct-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e39157ada3a2971efdcd375ad77fc62f3a7dbdf58a5f2364db64c855618a4126
size 324409792

3
Qwen2.5-Coder-0.5B-Instruct-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:edae399b439ce96cdb87f751505d6f32f39970e1b981a18b14277b1ebc5eac01
size 321549760

3
Qwen2.5-Coder-0.5B-Instruct-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:55e0c4e50c79773cdc85bf0ae757289099c5fdc872076089224618938c398db8
size 342752704

3
Qwen2.5-Coder-0.5B-Instruct-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b673e1433e6080b3afa0e111313bbdfbd096d514728b2192a2ec5f36297c9732
size 338607808

3
Qwen2.5-Coder-0.5B-Instruct-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cdddb6c28677d6e70cdfded0dc59cf9e0dd04108720cdd7fc2b351f40c8af7a1
size 338607808

3
Qwen2.5-Coder-0.5B-Instruct-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5b7108a24d512fd761fa28a2d02b273e1bb26262e66e82b2784300c4f392ded0
size 333704896

3
Qwen2.5-Coder-0.5B-Instruct-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7e73a9a6c03e2122226f05eb47241a96fee02982956831e30aa4d8e2d5d60f1d
size 352671424

3
Qwen2.5-Coder-0.5B-Instruct-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2943e9d986fc0d4366d25dde70af043e2abf6471baa4a7f82db5317a134b9f23
size 349402816

3
Qwen2.5-Coder-0.5B-Instruct-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0d3714efb30e45ba2aa038e9b1c1c44b19df0b7b7f3ec3c7bb3c4e61ae2edf6f
size 338607808

3
Qwen2.5-Coder-0.5B-Instruct-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b1d7a8c90e3b66d4012b9cd56a3901b38ec3acc135cab3a70564d07e0003bd3
size 331049152

3
Qwen2.5-Coder-0.5B-Instruct-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aad5e93267ef18a04680a2ebaa84dba767bebcb7448d3530d67b443ce2797bdd
size 369358528

3
Qwen2.5-Coder-0.5B-Instruct-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7682806a7a03febbbc5f9063a7b42e1daca9b9cc2fac5deaa049a24a8b58cce6
size 355466944

3
Qwen2.5-Coder-0.5B-Instruct-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:30b5c6abe621f5efbdffd0685319f7b5d2392fd0a5a527d2daedc2681c5b0d9e
size 338263744

3
Qwen2.5-Coder-0.5B-Instruct-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a1b100f062a23a5d18d766d8458c161bb62c06cd113a95e8795d6abf6a21c1a7
size 352972480

3
Qwen2.5-Coder-0.5B-Instruct-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a040196b8e0734a959df002ae6d1a3c27b18c7dbead6d39f3498ba5254402b89
size 374519488

3
Qwen2.5-Coder-0.5B-Instruct-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:45fcabeb488e1f3becc0d93e10d38554355de6d379a04fecd3f21fc59316efac
size 397808320

3
Qwen2.5-Coder-0.5B-Instruct-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:010d0119d7a69630293dc3e34bbacaf007037050021b370c218d73fbe5335b18
size 385472192

3
Qwen2.5-Coder-0.5B-Instruct-Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:79cfe02b976ff99558524cab1a30da39467f4238c25fc80d670f69aca9034263
size 397700800

3
Qwen2.5-Coder-0.5B-Instruct-Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b70868f1dd61110166892c5127c29fa9c1c3241a94335bcbecae57e0e6d3e640
size 419247808

3
Qwen2.5-Coder-0.5B-Instruct-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c71d887fd3660eb936ea56d85a020b13237ed01df30467c4aea88ca4a4d90036
size 420086464

3
Qwen2.5-Coder-0.5B-Instruct-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3f15ba69c55945fc897eb204fd578c80aee9efffbaa36052c19c87d859587d5
size 412710592

3
Qwen2.5-Coder-0.5B-Instruct-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e390cf18f2e67afd4d5ddd91a8ecfc07bd4b91470d2e68c8bc585456ef167d61
size 505736896

3
Qwen2.5-Coder-0.5B-Instruct-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9e0dbb28c09e92c8b85dcd44ee39f06434a6578e9eef9511760b96ffe1b90531
size 531068608

156
README.md Normal file
View File

@@ -0,0 +1,156 @@
---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct/blob/main/LICENSE
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-0.5B
pipeline_tag: text-generation
library_name: transformers
tags:
- code
- codeqwen
- chat
- qwen
- qwen-coder
---
<br><img src="https://cdn-uploads.huggingface.co/production/uploads/646410e04bf9122922289dc7/wHRc2ZKSErOs92ZcdKQrY.webp" width="720"><br>
# Llama.cpp imatrix quantizations of [Qwen/Qwen2.5-Coder-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct)
Using llama.cpp commit [3ad5451](https://github.com/ggerganov/llama.cpp/commit/3ad5451) for quantization.
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
<hr>
# Perplexity table (the lower the better)
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------------------------------------------------------------------------------------------------------------------------------------ | --------- | ------- | -------- | ------------ | -------------- |
| [IQ1_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ1_S.gguf) | 301 | 32.6390 | 31.75 | 69.27 | 0.59 |
| [IQ1_M](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ1_M.gguf) | 303 | 29.9721 | 31.96 | 75.43 | 0.53 |
| [IQ2_XXS](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ2_XXS.gguf) | 306 | 28.2728 | 32.28 | 79.97 | 0.52 |
| [IQ2_XS](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ2_XS.gguf) | 309 | 26.5764 | 32.59 | 85.07 | 0.48 |
| [IQ2_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ2_S.gguf) | 310 | 26.3276 | 32.70 | 85.88 | 0.48 |
| [IQ2_M](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ2_M.gguf) | 313 | 25.6364 | 33.02 | 88.19 | 0.46 |
| [Q2_K_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q2_K_S.gguf) | 315 | 25.6897 | 33.23 | 88.01 | 0.46 |
| [IQ3_XXS](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ3_XXS.gguf) | 318 | 24.6070 | 33.54 | 91.88 | 0.44 |
| [Q3_K_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q3_K_S.gguf) | 322 | 24.5354 | 33.97 | 92.15 | 0.44 |
| [IQ3_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ3_S.gguf) | 322 | 24.0335 | 33.97 | 94.07 | 0.43 |
| [IQ3_XS](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ3_XS.gguf) | 322 | 24.0335 | 33.97 | 94.07 | 0.43 |
| [Q2_K](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q2_K.gguf) | 322 | 24.4565 | 33.97 | 92.45 | 0.44 |
| [IQ3_M](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ3_M.gguf) | 326 | 23.8652 | 34.39 | 94.74 | 0.43 |
| [IQ4_XS](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ4_XS.gguf) | 333 | 23.6261 | 35.13 | 95.70 | 0.42 |
| [IQ4_NL](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-IQ4_NL.gguf) | 336 | 23.6102 | 35.44 | 95.76 | 0.42 |
| [Q4_0](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q4_0.gguf) | 336 | 23.8547 | 35.44 | 94.78 | 0.42 |
| [Q3_K_M](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q3_K_M.gguf) | 338 | 23.5268 | 35.65 | 96.10 | 0.42 |
| [Q3_K_L](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q3_K_L.gguf) | 352 | 23.3214 | 37.13 | 96.95 | 0.42 |
| [Q4_1](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q4_1.gguf) | 357 | 23.3396 | 37.66 | 96.87 | 0.42 |
| [Q4_K_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q4_K_S.gguf) | 367 | 23.2008 | 38.71 | 97.45 | 0.41 |
| [Q5_0](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q5_0.gguf) | 379 | 23.1914 | 39.98 | 97.49 | 0.41 |
| [Q4_K_M](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q4_K_M.gguf) | 379 | 23.1306 | 39.98 | 97.75 | 0.41 |
| [Q5_K_S](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q5_K_S.gguf) | 393 | 22.8146 | 41.46 | 99.10 | 0.41 |
| [Q5_1](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q5_1.gguf) | 399 | 22.8381 | 42.09 | 99.00 | 0.41 |
| [Q5_K_M](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q5_K_M.gguf) | 400 | 22.7761 | 42.19 | 99.27 | 0.41 |
| [Q6_K](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q6_K.gguf) | 482 | 22.6302 | 50.84 | 99.91 | 0.40 |
| [Q8_0](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-Q8_0.gguf) | 506 | 22.6359 | 53.38 | 99.88 | 0.40 |
| [F16](https://huggingface.co/ThomasBaruzier/Qwen2.5-Coder-0.5B-Instruct-GGUF/blob/main/Qwen2.5-Coder-0.5B-Instruct-F16.gguf) | 948 | 22.6094 | 100 | 100 | 0.40 |
---
# Qwen2.5-Coder-0.5B-Instruct
## Introduction
Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
**This repo contains the instruction-tuned 0.5B Qwen2.5-Coder model**, which has the following features:
- Type: Causal Language Models
- Training Stage: Pretraining & Post-training
- Architecture: transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias and tied word embeddings
- Number of Parameters: 0.49B
- Number of Paramaters (Non-Embedding): 0.36B
- Number of Layers: 24
- Number of Attention Heads (GQA): 14 for Q and 2 for KV
- Context Length: Full 32,768 tokens
For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
## Requirements
The code of Qwen2.5-Coder has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`.
With `transformers<4.37.0`, you will encounter the following error:
```
KeyError: 'qwen2'
```
## Quickstart
Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen2.5-Coder-0.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "write a quick sort algorithm."
messages = [
{"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```
## Evaluation & Performance
Detailed evaluation results are reported in this [📑 blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/).
For requirements on GPU memory and the respective throughput, see results [here](https://qwen.readthedocs.io/en/latest/benchmark/speed_benchmark.html).
## Citation
If you find our work helpful, feel free to give us a cite.
```
@article{hui2024qwen2,
title={Qwen2. 5-Coder Technical Report},
author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
journal={arXiv preprint arXiv:2409.12186},
year={2024}
}
@article{qwen2,
title={Qwen2 Technical Report},
author={An Yang and Baosong Yang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Zhou and Chengpeng Li and Chengyuan Li and Dayiheng Liu and Fei Huang and Guanting Dong and Haoran Wei and Huan Lin and Jialong Tang and Jialin Wang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Ma and Jin Xu and Jingren Zhou and Jinze Bai and Jinzheng He and Junyang Lin and Kai Dang and Keming Lu and Keqin Chen and Kexin Yang and Mei Li and Mingfeng Xue and Na Ni and Pei Zhang and Peng Wang and Ru Peng and Rui Men and Ruize Gao and Runji Lin and Shijie Wang and Shuai Bai and Sinan Tan and Tianhang Zhu and Tianhao Li and Tianyu Liu and Wenbin Ge and Xiaodong Deng and Xiaohuan Zhou and Xingzhang Ren and Xinyu Zhang and Xipin Wei and Xuancheng Ren and Yang Fan and Yang Yao and Yichang Zhang and Yu Wan and Yunfei Chu and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zhihao Fan},
journal={arXiv preprint arXiv:2407.10671},
year={2024}
}
```

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:df2c4e54f457f8a2e2b4d54432e046c476d14e9185aa02f9507cde643ff68cdd
size 988628

30
perplexity.md Normal file
View File

@@ -0,0 +1,30 @@
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------ | --------- | ------- | -------- | ------------ | -------------- |
| IQ1_S | 301 | 32.6390 | 31.75 | 69.27 | 0.59 |
| IQ1_M | 303 | 29.9721 | 31.96 | 75.43 | 0.53 |
| IQ2_XXS | 306 | 28.2728 | 32.28 | 79.97 | 0.52 |
| IQ2_XS | 309 | 26.5764 | 32.59 | 85.07 | 0.48 |
| IQ2_S | 310 | 26.3276 | 32.70 | 85.88 | 0.48 |
| IQ2_M | 313 | 25.6364 | 33.02 | 88.19 | 0.46 |
| Q2_K_S | 315 | 25.6897 | 33.23 | 88.01 | 0.46 |
| IQ3_XXS | 318 | 24.6070 | 33.54 | 91.88 | 0.44 |
| Q3_K_S | 322 | 24.5354 | 33.97 | 92.15 | 0.44 |
| IQ3_S | 322 | 24.0335 | 33.97 | 94.07 | 0.43 |
| IQ3_XS | 322 | 24.0335 | 33.97 | 94.07 | 0.43 |
| Q2_K | 322 | 24.4565 | 33.97 | 92.45 | 0.44 |
| IQ3_M | 326 | 23.8652 | 34.39 | 94.74 | 0.43 |
| IQ4_XS | 333 | 23.6261 | 35.13 | 95.70 | 0.42 |
| IQ4_NL | 336 | 23.6102 | 35.44 | 95.76 | 0.42 |
| Q4_0 | 336 | 23.8547 | 35.44 | 94.78 | 0.42 |
| Q3_K_M | 338 | 23.5268 | 35.65 | 96.10 | 0.42 |
| Q3_K_L | 352 | 23.3214 | 37.13 | 96.95 | 0.42 |
| Q4_1 | 357 | 23.3396 | 37.66 | 96.87 | 0.42 |
| Q4_K_S | 367 | 23.2008 | 38.71 | 97.45 | 0.41 |
| Q5_0 | 379 | 23.1914 | 39.98 | 97.49 | 0.41 |
| Q4_K_M | 379 | 23.1306 | 39.98 | 97.75 | 0.41 |
| Q5_K_S | 393 | 22.8146 | 41.46 | 99.10 | 0.41 |
| Q5_1 | 399 | 22.8381 | 42.09 | 99.00 | 0.41 |
| Q5_K_M | 400 | 22.7761 | 42.19 | 99.27 | 0.41 |
| Q6_K | 482 | 22.6302 | 50.84 | 99.91 | 0.40 |
| Q8_0 | 506 | 22.6359 | 53.38 | 99.88 | 0.40 |
| F16 | 948 | 22.6094 | 100 | 100 | 0.40 |