113 lines
5.0 KiB
Markdown
113 lines
5.0 KiB
Markdown
---
|
||
base_model: microsoft/Phi-3-mini-4k-instruct
|
||
inference: false
|
||
license: mit
|
||
license_link: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/LICENSE
|
||
language:
|
||
- en
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- nlp
|
||
- code
|
||
model_creator: microsoft
|
||
model_name: Phi-3-mini-4k-instruct
|
||
model_type: phi3
|
||
quantized_by: brittlewis12
|
||
---
|
||
|
||
# Phi 3 Mini 4K Instruct GGUF
|
||
|
||
***Updated with Microsoft’s [latest model changes](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/commit/4f818b18e097c9ae8f93a29a57027cad54b75304) as of July 21, 2024***
|
||
|
||
**Original model**: [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)
|
||
|
||
**Model creator**: [Microsoft](https://huggingface.co/microsoft)
|
||
|
||
This repo contains GGUF format model files for Microsoft’s Phi 3 Mini 4K Instruct.
|
||
|
||
> The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties.
|
||
|
||
Learn more on Microsoft’s [Model page](https://azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/).
|
||
|
||
### What is GGUF?
|
||
|
||
GGUF is a file format for representing AI models. It is the third version of the format,
|
||
introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
|
||
Converted with llama.cpp build 3432 (revision [45f2c19](https://github.com/ggerganov/llama.cpp/commit/45f2c19cc57286eead7b232ce8028273a817aa4d)),
|
||
using [autogguf](https://github.com/brittlewis12/autogguf).
|
||
|
||
### Prompt template
|
||
|
||
```
|
||
<|system|>
|
||
{{system_prompt}}<|end|>
|
||
<|user|>
|
||
{{prompt}}<|end|>
|
||
<|assistant|>
|
||
|
||
```
|
||
|
||
---
|
||
|
||
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!
|
||
|
||
|
||
|
||
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
|
||
- create & save **Characters** with custom system prompts & temperature settings
|
||
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
|
||
- make it your own with custom **Theme colors**
|
||
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
|
||
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
|
||
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
|
||
|
||
---
|
||
|
||
## Original Model Evaluation
|
||
|
||
Comparison of July update vs original April release:
|
||
|
||
| Benchmarks | Original | June 2024 Update |
|
||
|------------|----------|------------------|
|
||
| Instruction Extra Hard | 5.7 | 6.0 |
|
||
| Instruction Hard | 4.9 | 5.1 |
|
||
| Instructions Challenge | 24.6 | 42.3 |
|
||
| JSON Structure Output | 11.5 | 52.3 |
|
||
| XML Structure Output | 14.4 | 49.8 |
|
||
| GPQA | 23.7 | 30.6 |
|
||
| MMLU | 68.8 | 70.9 |
|
||
| **Average** | **21.9** | **36.7** |
|
||
|
||
|
||
---
|
||
|
||
### Original April release
|
||
|
||
> As is now standard, we use few-shot prompts to evaluate the models, at temperature 0.
|
||
> The prompts and number of shots are part of a Microsoft internal tool to evaluate language models, and in particular we did no optimization to the pipeline for Phi-3.
|
||
> More specifically, we do not change prompts, pick different few-shot examples, change prompt format, or do any other form of optimization for the model.
|
||
>
|
||
> The number of k–shot examples is listed per-benchmark.
|
||
|
||
| | Phi-3-Mini-4K-In<br>3.8b | Phi-2<br>2.7b | Mistral<br>7b | Gemma<br>7b | Llama-3-In<br>8b | Mixtral<br>8x7b | GPT-3.5<br>version 1106 |
|
||
|---|---|---|---|---|---|---|---|
|
||
| MMLU <br>5-Shot | 68.8 | 56.3 | 61.7 | 63.6 | 66.5 | 68.4 | 71.4 |
|
||
| HellaSwag <br> 5-Shot | 76.7 | 53.6 | 58.5 | 49.8 | 71.1 | 70.4 | 78.8 |
|
||
| ANLI <br> 7-Shot | 52.8 | 42.5 | 47.1 | 48.7 | 57.3 | 55.2 | 58.1 |
|
||
| GSM-8K <br> 0-Shot; CoT | 82.5 | 61.1 | 46.4 | 59.8 | 77.4 | 64.7 | 78.1 |
|
||
| MedQA <br> 2-Shot | 53.8 | 40.9 | 49.6 | 50.0 | 60.5 | 62.2 | 63.4 |
|
||
| AGIEval <br> 0-Shot | 37.5 | 29.8 | 35.1 | 42.1 | 42.0 | 45.2 | 48.4 |
|
||
| TriviaQA <br> 5-Shot | 64.0 | 45.2 | 72.3 | 75.2 | 67.7 | 82.2 | 85.8 |
|
||
| Arc-C <br> 10-Shot | 84.9 | 75.9 | 78.6 | 78.3 | 82.8 | 87.3 | 87.4 |
|
||
| Arc-E <br> 10-Shot | 94.6 | 88.5 | 90.6 | 91.4 | 93.4 | 95.6 | 96.3 |
|
||
| PIQA <br> 5-Shot | 84.2 | 60.2 | 77.7 | 78.1 | 75.7 | 86.0 | 86.6 |
|
||
| SociQA <br> 5-Shot | 76.6 | 68.3 | 74.6 | 65.5 | 73.9 | 75.9 | 68.3 |
|
||
| BigBench-Hard <br> 0-Shot | 71.7 | 59.4 | 57.3 | 59.6 | 51.5 | 69.7 | 68.32 |
|
||
| WinoGrande <br> 5-Shot | 70.8 | 54.7 | 54.2 | 55.6 | 65 | 62.0 | 68.8 |
|
||
| OpenBookQA <br> 10-Shot | 83.2 | 73.6 | 79.8 | 78.6 | 82.6 | 85.8 | 86.0 |
|
||
| BoolQ <br> 0-Shot | 77.6 | -- | 72.2 | 66.0 | 80.9 | 77.6 | 79.1 |
|
||
| CommonSenseQA <br> 10-Shot | 80.2 | 69.3 | 72.6 | 76.2 | 79 | 78.1 | 79.6 |
|
||
| TruthfulQA <br> 10-Shot | 65.0 | -- | 52.1 | 53.0 | 63.2 | 60.1 | 85.8 |
|
||
| HumanEval <br> 0-Shot | 59.1 | 47.0 | 28.0 | 34.1 | 60.4 | 37.8 | 62.2 |
|
||
| MBPP <br> 3-Shot | 53.8 | 60.6 | 50.8 | 51.5 | 67.7 | 60.2 | 77.8 |
|