SanctumAI/Phi-3.5-mini-instruct-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: SanctumAI/Phi-3.5-mini-instruct-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-18 05:38:16 +08:00
commit d7093b834c
20 changed files with 170 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
.gitattributes vendored Normal file
View File

@@ -0,0 +1,52 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.f16.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
phi-3.5-mini-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

64
README.md Normal file
View File

@@ -0,0 +1,64 @@
---
license: mit
license_link: https://huggingface.co/microsoft/Phi-3.5-mini-instruct/resolve/main/LICENSE
language:
- multilingual
pipeline_tag: text-generation
tags:
- nlp
- code
---
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64a28db2f1968b7d7f357182/u6tSdh8u1tucYkSr2bNnV.png)
*This model was quantized by [SanctumAI](https://sanctum.ai). To leave feedback, join our community in [Discord](https://discord.gg/7ZNE78HJKh).*
# Phi 3 Mini 4K Instruct GGUF
**Model creator:** [microsoft](https://huggingface.co/microsoft)<br>
**Original model**: [Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)<br>
## Model Summary:
Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.
## Prompt Template:
If you're using Sanctum app, simply use `Phi 3` model preset.
Prompt template:
```
<|system|>
{system_prompt}<|end|>
<|user|>
{prompt}<|end|>
<|assistant|>
```
## Hardware Requirements Estimate
| Name | Quant method | Size | Memory (RAM, vRAM) required |
| ---- | ---- | ---- | ---- |
| [phi-3.5-mini-instruct.Q2_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q2_K.gguf) | Q2_K | 1.42 GB | 5.02 GB |
| [phi-3.5-mini-instruct.Q3_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_S.gguf) | Q3_K_S | 1.68 GB | 5.27 GB |
| [phi-3.5-mini-instruct.Q3_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_M.gguf) | Q3_K_M | 1.96 GB | 5.52 GB |
| [phi-3.5-mini-instruct.Q3_K_L.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q3_K_L.gguf) | Q3_K_L | 2.09 GB | 5.65 GB |
| [phi-3.5-mini-instruct.Q4_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_0.gguf) | Q4_0 | 2.18 GB | 5.73 GB |
| [phi-3.5-mini-instruct.Q4_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K_S.gguf) | Q4_K_S | 2.19 GB | 5.74 GB |
| [phi-3.5-mini-instruct.Q4_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K_M.gguf) | Q4_K_M | 2.39 GB | 5.93 GB |
| [phi-3.5-mini-instruct.Q4_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_K.gguf) | Q4_K | 2.39 GB | 5.93 GB |
| [phi-3.5-mini-instruct.Q4_1.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q4_1.gguf) | Q4_1 | 2.41 GB | 5.94 GB |
| [phi-3.5-mini-instruct.Q5_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_0.gguf) | Q5_0 | 2.64 GB | 6.16 GB |
| [phi-3.5-mini-instruct.Q5_K_S.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K_S.gguf) | Q5_K_S | 2.64 GB | 6.16 GB |
| [phi-3.5-mini-instruct.Q5_K_M.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K_M.gguf) | Q5_K_M | 2.82 GB | 6.32 GB |
| [phi-3.5-mini-instruct.Q5_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_K.gguf) | Q5_K | 2.82 GB | 6.32 GB |
| [phi-3.5-mini-instruct.Q5_1.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q5_1.gguf) | Q5_1 | 2.87 GB | 6.38 GB |
| [phi-3.5-mini-instruct.Q6_K.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q6_K.gguf) | Q6_K | 3.14 GB | 6.62 GB |
| [phi-3.5-mini-instruct.Q8_0.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.Q8_0.gguf) | Q8_0 | 4.06 GB | 7.48 GB |
| [phi-3.5-mini-instruct.f16.gguf](https://huggingface.co/SanctumAI/Phi-3.5-mini-instruct-GGUF/blob/main/phi-3.5-mini-instruct.f16.gguf) | f16 | 7.64 GB | 10.82 GB |
## Disclaimer
Sanctum is not the creator, originator, or owner of any Model featured in the Models section of the Sanctum application. Each Model is created and provided by third parties. Sanctum does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Model listed there. You understand that supported Models can produce content that might be offensive, harmful, inaccurate or otherwise inappropriate, or deceptive. Each Model is the sole responsibility of the person or entity who originated such Model. Sanctum may not monitor or control the Models supported and cannot, and does not, take responsibility for any such Model. Sanctum disclaims all warranties or guarantees about the accuracy, reliability or benefits of the Models. Sanctum further disclaims any warranty that the Model will meet your requirements, be secure, uninterrupted or available at any time or location, or error-free, viruses-free, or that any errors will be corrected, or otherwise. You will be solely responsible for any damage resulting from your use of or access to the Models, your downloading of any Model, or use of any other Model provided by or through Sanctum.

3
config.json Normal file
View File

@@ -0,0 +1,3 @@
{
"model_type": "phi3"
}

3
phi-3.5-mini-instruct.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54d47caa8bf30afce66f6945667b03df2df4deb94711c21d854706481c9c4c4b
size 1416204288

3
phi-3.5-mini-instruct.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cda3c938fc13343a80f60881f4987bbbb990a59403aafce7f81d4380b76fd2f1
size 2087597568

3
phi-3.5-mini-instruct.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:abcfd2f187d16cb29ba3e5d1fd9b05b860e9a5afdd4d609e40ec0596fdbdc8c8
size 1955476992

3
phi-3.5-mini-instruct.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:81c2147a6c5a94b01185b14ae424a26163543afa2333560c7d2297f000cb31ee
size 1681798656

3
phi-3.5-mini-instruct.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b5374915da534cb93df39f03bd4f2cd5a0c533df0d5e21957dc9556c260be9eb
size 2176177152

3
phi-3.5-mini-instruct.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3a0533a94963547beef098b5813d60fe89a577d53fd7a9043e5c20ba4326b69a
size 2408825856

3
phi-3.5-mini-instruct.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ef532675b6684b28e0a1a0dca8b3bf4132673bf2294b7d87b41b1d64bb03bca
size 2393232384

3
phi-3.5-mini-instruct.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ef532675b6684b28e0a1a0dca8b3bf4132673bf2294b7d87b41b1d64bb03bca
size 2393232384

3
phi-3.5-mini-instruct.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5db3e683309003f16a47246b8e4b610204c16c0d3abe0de86ccf971569493d68
size 2188760064

3
phi-3.5-mini-instruct.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b75226072ac8ef8c1e279fd7de436b5aedca7b6fcae6bed3e2d6c933ca2d55a
size 2641474560

3
phi-3.5-mini-instruct.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2a937d3f6c1507f19dbc54ff126b7156ba38d7c8a1b02d2bd76d476731c7a068
size 2874123264

3
phi-3.5-mini-instruct.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ef57be356e37cdf6045f7ef9149f7f6974e7c18f23b384296608029466c48f01
size 2815276032

3
phi-3.5-mini-instruct.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ef57be356e37cdf6045f7ef9149f7f6974e7c18f23b384296608029466c48f01
size 2815276032

3
phi-3.5-mini-instruct.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50b8dc3b4ccc9c28778f43df7356709463d96a1bd35e942ed46ee88a34a4b22f
size 2641474560

3
phi-3.5-mini-instruct.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:86e5e74919d7935fe84e29a2e525feb9f1cce5beb6d550bc8cb2098f0f931410
size 3135853056

3
phi-3.5-mini-instruct.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:81fd0b829a6ff15b3df3cc792de3f3e75a9fb2f1f55c0059af55a698b1173150
size 4061222400

3
phi-3.5-mini-instruct.f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04381ade8b69632b564f1dbc473dac52f7819349ceb3cc76d268dee91f762fe2
size 7643297280