初始化项目，由ModelHub XC社区提供模型

Model: brittlewis12/Phi-3-mini-4k-instruct-GGUF
Source: Original Platform

.gitattributes

@@ -0,0 +1,60 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+phi-3-mini-4k-instruct.fp16.gguf filter=lfs diff=lfs merge=lfs -text
+Phi-3-mini-4k-instruct.imatrix filter=lfs diff=lfs merge=lfs -text

Phi-3-mini-4k-instruct.imatrix

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8add4e97cd4e5e494d0bc2156b8a503a289100fe7ae556625708adb458b323f
+size 2232597

README.md

@@ -0,0 +1,112 @@
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+inference: false
+license: mit
+license_link: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/LICENSE
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- nlp
+- code
+model_creator: microsoft
+model_name: Phi-3-mini-4k-instruct
+model_type: phi3
+quantized_by: brittlewis12
+---
+
+# Phi 3 Mini 4K Instruct GGUF
+
+***Updated with Microsoft’s [latest model changes](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/commit/4f818b18e097c9ae8f93a29a57027cad54b75304) as of July 21, 2024***
+
+**Original model**: [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
+
+**Model creator**: [Microsoft](https://huggingface.co/microsoft)
+
+This repo contains GGUF format model files for Microsoft’s Phi 3 Mini 4K Instruct.
+
+> The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties.
+
+Learn more on Microsoft’s [Model page](https://azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/).
+
+### What is GGUF?
+
+GGUF is a file format for representing AI models. It is the third version of the format, 
+introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. 
+Converted with llama.cpp build 3432 (revision [45f2c19](https://github.com/ggerganov/llama.cpp/commit/45f2c19cc57286eead7b232ce8028273a817aa4d)),
+using [autogguf](https://github.com/brittlewis12/autogguf).
+
+### Prompt template
+
+```
+<|system|>
+{{system_prompt}}<|end|>
+<|user|>
+{{prompt}}<|end|>
+<|assistant|>
+
+```
+
+---
+
+## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!
+
+![cnvrs.ai](https://pbs.twimg.com/profile_images/1744049151241797632/0mIP-P9e_400x400.jpg)
+
+[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
+- create & save **Characters** with custom system prompts & temperature settings
+- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
+- make it your own with custom **Theme colors**
+- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
+- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
+- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
+
+---
+
+## Original Model Evaluation
+
+Comparison of July update vs original April release:
+
+| Benchmarks | Original | June 2024 Update |
+|------------|----------|------------------|
+| Instruction Extra Hard | 5.7 | 6.0 |
+| Instruction Hard | 4.9 | 5.1 |
+| Instructions Challenge | 24.6 | 42.3 |
+| JSON Structure Output | 11.5 | 52.3 |
+| XML Structure Output | 14.4 | 49.8 |
+| GPQA	| 23.7	| 30.6 |
+| MMLU	| 68.8	| 70.9 |
+| **Average**	| **21.9**	| **36.7** |
+
+
+---
+
+### Original April release
+
+> As is now standard, we use few-shot prompts to evaluate the models, at temperature 0. 
+> The prompts and number of shots are part of a Microsoft internal tool to evaluate language models, and in particular we did no optimization to the pipeline for Phi-3.
+> More specifically, we do not change prompts, pick different few-shot examples, change prompt format, or do any other form of optimization for the model.
+> 
+> The number of k–shot examples is listed per-benchmark. 
+
+|   | Phi-3-Mini-4K-In<br>3.8b | Phi-2<br>2.7b | Mistral<br>7b | Gemma<br>7b | Llama-3-In<br>8b | Mixtral<br>8x7b | GPT-3.5<br>version 1106 |
+|---|---|---|---|---|---|---|---|
+| MMLU <br>5-Shot | 68.8 | 56.3 | 61.7 | 63.6 | 66.5 | 68.4 | 71.4 |
+| HellaSwag <br> 5-Shot | 76.7 | 53.6 | 58.5 | 49.8 | 71.1 | 70.4 | 78.8 |
+| ANLI <br> 7-Shot | 52.8 | 42.5 | 47.1 | 48.7 | 57.3 | 55.2 | 58.1 |
+| GSM-8K <br> 0-Shot; CoT | 82.5 | 61.1 | 46.4 | 59.8 | 77.4 | 64.7 | 78.1 |
+| MedQA <br> 2-Shot | 53.8 | 40.9 | 49.6 | 50.0 | 60.5 | 62.2 | 63.4 |
+| AGIEval <br> 0-Shot | 37.5 | 29.8 | 35.1 | 42.1 | 42.0 | 45.2 | 48.4 |
+| TriviaQA <br> 5-Shot | 64.0 | 45.2 | 72.3 | 75.2 | 67.7 | 82.2 | 85.8 |
+| Arc-C <br> 10-Shot | 84.9 | 75.9 | 78.6 | 78.3 | 82.8 | 87.3 | 87.4 |
+| Arc-E <br> 10-Shot | 94.6 | 88.5 | 90.6 | 91.4 | 93.4 | 95.6 | 96.3 |
+| PIQA <br> 5-Shot | 84.2 | 60.2 | 77.7 | 78.1 | 75.7 | 86.0 | 86.6 |
+| SociQA <br> 5-Shot | 76.6 | 68.3 | 74.6 | 65.5 | 73.9 | 75.9 | 68.3 |
+| BigBench-Hard <br> 0-Shot | 71.7 | 59.4 | 57.3 | 59.6 | 51.5 | 69.7 | 68.32 |
+| WinoGrande <br> 5-Shot | 70.8 | 54.7 | 54.2 | 55.6 | 65 | 62.0 | 68.8 |
+| OpenBookQA <br> 10-Shot | 83.2 | 73.6 | 79.8 | 78.6 | 82.6 | 85.8 | 86.0 |
+| BoolQ <br> 0-Shot | 77.6 | -- | 72.2 | 66.0 | 80.9 | 77.6 | 79.1 |
+| CommonSenseQA <br> 10-Shot | 80.2 | 69.3 | 72.6 | 76.2 | 79 | 78.1 | 79.6 |
+| TruthfulQA <br> 10-Shot | 65.0 | -- | 52.1 | 53.0 | 63.2 | 60.1 | 85.8 |
+| HumanEval <br> 0-Shot | 59.1 | 47.0 | 28.0 | 34.1 | 60.4 | 37.8 | 62.2 |
+| MBPP <br> 3-Shot | 53.8 | 60.6 | 50.8 | 51.5 | 67.7 | 60.2 | 77.8 |

phi-3-mini-4k-instruct.IQ1_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd96028169dd5552ef46b17a2a5b299b453d6c446ab4ef6e7ee9629f453e138d
+size 917106816

phi-3-mini-4k-instruct.IQ1_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ee273fef5d2fa1fd2b5dd1303af4fa070e3a6aac7b9bf73dfc71da83108e005
+size 841609344

phi-3-mini-4k-instruct.IQ2_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83b5fe1174d93fcdd418d5c1479596688f551d69396c35fbf476079cb0fca4fc
+size 1316394624

phi-3-mini-4k-instruct.IQ2_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a04a0dd436bb48d70cce735eb08cd78c2a72072b90d60a926e161760ba10aa50
+size 1215731328

phi-3-mini-4k-instruct.IQ2_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91bb96a94609fbb806f1fb5cd53c8d20b95e3d410f2e7122ad387f4d5f826109
+size 1153036416

phi-3-mini-4k-instruct.IQ2_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac8f44b73a443ad74b0741bd5bb4d8ce9fd1e384d1589b2ba7a987d5986e2d3d
+size 1042935936

phi-3-mini-4k-instruct.IQ3_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:113759538d92fb1bf67fea0c6d6d8dc3a4bd5f9340dff950eeaf439e9b20e19d
+size 1855599744

phi-3-mini-4k-instruct.IQ3_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58bcb964b44cadecff5ddbcf106dcf54a840bec698970b74233380699cbdb81b
+size 1681798272

phi-3-mini-4k-instruct.IQ3_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c240141bab30fcdfb9993e76698b7b882a25f062a595fc27c29fcc2c6e8c4449
+size 1625175168

phi-3-mini-4k-instruct.IQ3_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39f9173a5410422abc5efd4748c35a5032492bdee065b8ffb2b93515ad4c1c89
+size 1513002624

phi-3-mini-4k-instruct.IQ4_NL.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c05610e38fe29b16f2f48e00240b0e8ff382ff557face17951e08d00f0430fc
+size 2176176768

phi-3-mini-4k-instruct.IQ4_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4d9958a5b031aec6a479ae25c532da8f44a2b66042270373b7cc798525d0b82
+size 2059852416

phi-3-mini-4k-instruct.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:962eabd0f201d9d6ff0b811132d8a59d79457d85ac64f936442ab004594038ab
+size 1416203680

phi-3-mini-4k-instruct.Q2_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a46e4b86f3fc4a3d92e7cb2ec1799c4e83f115e4ed87b3c00627da87fbfe06a
+size 1327337088

phi-3-mini-4k-instruct.Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:857870688589c0fff41bf0dd4a5a33ad16e18faadcbc721abcead497cc838e1c
+size 2087596960

phi-3-mini-4k-instruct.Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a059430cbf281abb1dfbf2d139d9c39b224d17d56bcad4e569b54c314c7d11e6
+size 1955476384

phi-3-mini-4k-instruct.Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd297929a9763c9b663b11a184fe85b3205add634a10dd0656076728a6fc2f41
+size 1681798048

phi-3-mini-4k-instruct.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2d23f7e8e7e5d441abf847e1604dd57b3a9a3d962c6011538ae4d8ea99070aa
+size 2393231776

phi-3-mini-4k-instruct.Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25dd99c428774ae199d7eebc5f11e3cc6be12c3d2797edd3bdd62264a4133b22
+size 2188759456

phi-3-mini-4k-instruct.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9cece78508b767a8a5682de87926a2616c2dfdaa4fd5c4a1fa7b36e9da0e4a53
+size 2815275424

phi-3-mini-4k-instruct.Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88999dc5cadfa7da97a9bdfd173da9a5b9238a21f5a04bbbfa097683b155f2cc
+size 2641473952

phi-3-mini-4k-instruct.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0374c03e747ce5d0cfa8d77d0464ab3fc737b797789e240998298c52f89334a
+size 3135852448

phi-3-mini-4k-instruct.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1ff580bd7d046d6036a68bf6b1139ba3d852af66c0cd80a1f5de8849bc3dfd3
+size 4061221792

phi-3-mini-4k-instruct.fp16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f0007c83dac511c10d465fc96ed1a3bbc76b9d16390bafe7268a04298697786
+size 7643296672