RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-17 03:46:23 +08:00
commit d2187d31ab
24 changed files with 208 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q3_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2.5-0.5B-Instruct-MLX.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

3
Qwen2.5-0.5B-Instruct-MLX.IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9693aa3d3bb80f42d3fad0200307e192e5569921adc49294045c19a75196d2bc
size 342752352

3
Qwen2.5-0.5B-Instruct-MLX.IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c9b5d492844ec70ecd5c4eea440f809a3e0167c8e9002f84347f29241ed57512
size 338607456

3
Qwen2.5-0.5B-Instruct-MLX.IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:831370c12f72b9af30aed4b168a5d368d68397e5811436e6b315161f44ad64d1
size 338607456

3
Qwen2.5-0.5B-Instruct-MLX.IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cbd862e570c7d53abbbe0abb871271b8d3e8ee8f28845ba8471fdca76be3c974
size 354305376

3
Qwen2.5-0.5B-Instruct-MLX.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b1e269b950d7f8ca36f5c8ec2e4a4337ae4c8c2b05bc2c0128924c1857dfcaf3
size 351445344

3
Qwen2.5-0.5B-Instruct-MLX.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:119b4de48663bb6f17db890b507ce102ba2ff736cf4da98a181455ed741c5fef
size 338607456

3
Qwen2.5-0.5B-Instruct-MLX.Q3_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ad829748348821d354805aa11323fe5e3015994e1ad8212c0dce85be790d14d0
size 355466592

3
Qwen2.5-0.5B-Instruct-MLX.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9312c598f4c8a696b63ecdde8f98cf86cf12416ee7b4995c2372ed484aaaade7
size 369358176

3
Qwen2.5-0.5B-Instruct-MLX.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ad829748348821d354805aa11323fe5e3015994e1ad8212c0dce85be790d14d0
size 355466592

3
Qwen2.5-0.5B-Instruct-MLX.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c79ecca768e3a8c703b0f9e20eb807e24bf2dd0169b62429c66974f544afc0ce
size 338263392

3
Qwen2.5-0.5B-Instruct-MLX.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3e676f7aa188b8516d154babe40708111d6170f05bcbfc674aef491e42b3c5b
size 352154976

3
Qwen2.5-0.5B-Instruct-MLX.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:af257a05e8cdf674bb4644e5d6c798d427323d0b57f7a52eba4b00eaf785281e
size 374519136

3
Qwen2.5-0.5B-Instruct-MLX.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4c826e9b1a61274c32811f7560389c266405cff6867b5aded39cd70a4bb0b466
size 397807968

3
Qwen2.5-0.5B-Instruct-MLX.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4c826e9b1a61274c32811f7560389c266405cff6867b5aded39cd70a4bb0b466
size 397807968

3
Qwen2.5-0.5B-Instruct-MLX.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c4caf77d07e566e9d3399046c956b05d44f357eeb35f6be36c4a252e37ca98ca
size 385471840

3
Qwen2.5-0.5B-Instruct-MLX.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fc2de4266e702ced97872d5d589ee74aa80d60282cce900668ca03ae18ce662a
size 396883296

3
Qwen2.5-0.5B-Instruct-MLX.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d0a5d56dd27f34d72f32258845a26a82aa6af155f3e53ac96807ed12e804c15
size 419247456

3
Qwen2.5-0.5B-Instruct-MLX.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96b5fd412e2e406519202040de7692d6b5f4d9bb226f86cdcde3dbca699c64ef
size 420086112

3
Qwen2.5-0.5B-Instruct-MLX.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96b5fd412e2e406519202040de7692d6b5f4d9bb226f86cdcde3dbca699c64ef
size 420086112

3
Qwen2.5-0.5B-Instruct-MLX.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:08350c8178fd47356560cf909a71ae84a33108c171676346285e4f63fe4ca9b9
size 412710240

3
Qwen2.5-0.5B-Instruct-MLX.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b7502bbac5da2bdab4c878b8ac9601555554e7b44191b15e60f8ac2180ae28c3
size 505736544

3
Qwen2.5-0.5B-Instruct-MLX.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:558e4eb78187bf7997fa092b98f132aa40114b0fcb207de4abb4d12c53889237
size 531068256

85
README.md Normal file
View File

@@ -0,0 +1,85 @@
Quantization made by Richard Erkhov.
[Github](https://github.com/RichardErkhov)
[Discord](https://discord.gg/pvy7H8DZMG)
[Request more models](https://github.com/RichardErkhov/quant_request)
Qwen2.5-0.5B-Instruct-MLX - GGUF
- Model creator: https://huggingface.co/TheBlueObserver/
- Original model: https://huggingface.co/TheBlueObserver/Qwen2.5-0.5B-Instruct-MLX/
| Name | Quant method | Size |
| ---- | ---- | ---- |
| [Qwen2.5-0.5B-Instruct-MLX.Q2_K.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q2_K.gguf) | Q2_K | 0.32GB |
| [Qwen2.5-0.5B-Instruct-MLX.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.IQ3_XS.gguf) | IQ3_XS | 0.32GB |
| [Qwen2.5-0.5B-Instruct-MLX.IQ3_S.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.IQ3_S.gguf) | IQ3_S | 0.32GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q3_K_S.gguf) | Q3_K_S | 0.32GB |
| [Qwen2.5-0.5B-Instruct-MLX.IQ3_M.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.IQ3_M.gguf) | IQ3_M | 0.32GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q3_K.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q3_K.gguf) | Q3_K | 0.33GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q3_K_M.gguf) | Q3_K_M | 0.33GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q3_K_L.gguf) | Q3_K_L | 0.34GB |
| [Qwen2.5-0.5B-Instruct-MLX.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.IQ4_XS.gguf) | IQ4_XS | 0.33GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q4_0.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q4_0.gguf) | Q4_0 | 0.33GB |
| [Qwen2.5-0.5B-Instruct-MLX.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.IQ4_NL.gguf) | IQ4_NL | 0.33GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q4_K_S.gguf) | Q4_K_S | 0.36GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q4_K.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q4_K.gguf) | Q4_K | 0.37GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q4_K_M.gguf) | Q4_K_M | 0.37GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q4_1.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q4_1.gguf) | Q4_1 | 0.35GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q5_0.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q5_0.gguf) | Q5_0 | 0.37GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q5_K_S.gguf) | Q5_K_S | 0.38GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q5_K.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q5_K.gguf) | Q5_K | 0.39GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q5_K_M.gguf) | Q5_K_M | 0.39GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q5_1.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q5_1.gguf) | Q5_1 | 0.39GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q6_K.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q6_K.gguf) | Q6_K | 0.47GB |
| [Qwen2.5-0.5B-Instruct-MLX.Q8_0.gguf](https://huggingface.co/RichardErkhov/TheBlueObserver_-_Qwen2.5-0.5B-Instruct-MLX-gguf/blob/main/Qwen2.5-0.5B-Instruct-MLX.Q8_0.gguf) | Q8_0 | 0.49GB |
Original model description:
---
base_model: Qwen/Qwen2.5-0.5B-Instruct
language:
- en
library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE
pipeline_tag: text-generation
tags:
- chat
- mlx
---
# TheBlueObserver/Qwen2.5-0.5B-Instruct-MLX
The Model [TheBlueObserver/Qwen2.5-0.5B-Instruct-MLX](https://huggingface.co/TheBlueObserver/Qwen2.5-0.5B-Instruct-MLX) was
converted to MLX format from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
using mlx-lm version **0.20.2**.
## Use with mlx
```bash
pip install mlx-lm
```
```python
from mlx_lm import load, generate
model, tokenizer = load("TheBlueObserver/Qwen2.5-0.5B-Instruct-MLX")
prompt="hello"
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)
```