102 lines
5.8 KiB
Markdown
102 lines
5.8 KiB
Markdown
---
|
||
base_model: google/gemma-7b-it
|
||
inference: false
|
||
language:
|
||
- en
|
||
model_creator: google
|
||
model_name: gemma-7b-it
|
||
model_type: gemma
|
||
pipeline_tag: text-generation
|
||
license: other
|
||
license_name: gemma-terms-of-use
|
||
license_link: https://ai.google.dev/gemma/terms
|
||
quantized_by: brittlewis12
|
||
---
|
||
|
||
# Gemma 7B Instruct GGUF
|
||
|
||
**Original model**: [gemma-7b-it](https://huggingface.co/google/gemma-7b-it)
|
||
|
||
**Model creator**: [google](https://huggingface.co/google)
|
||
|
||
This repo contains GGUF format model files for Google’s Gemma-7B-it.
|
||
|
||
> Gemma is a family of lightweight, state-of-the-art open models from Google,
|
||
> built from the same research and technology used to create the Gemini models.
|
||
> They are text-to-text, decoder-only large language models, available in English,
|
||
> with open weights, pre-trained variants, and instruction-tuned variants. Gemma
|
||
> models are well-suited for a variety of text generation tasks, including
|
||
> question answering, summarization, and reasoning. Their relatively small size
|
||
> makes it possible to deploy them in environments with limited resources such as
|
||
> a laptop, desktop or your own cloud infrastructure, democratizing access to
|
||
> state of the art AI models and helping foster innovation for everyone.
|
||
|
||
Learn more on Google’s [Model page](https://ai.google.dev/gemma/docs).
|
||
|
||
### What is GGUF?
|
||
|
||
GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
|
||
Converted using llama.cpp build 2226 (revision [eccd7a2](https://github.com/ggerganov/llama.cpp/commit/eccd7a26ddbff19e4b8805648f5f14c501957859))
|
||
|
||
### Prompt template: Gemma Instruct
|
||
|
||
```
|
||
<start_of_turn>user
|
||
{{prompt}}<end_of_turn>
|
||
<start_of_turn>model
|
||
|
||
```
|
||
|
||
---
|
||
|
||
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!
|
||
|
||
|
||
|
||
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
|
||
- create & save **Characters** with custom system prompts & temperature settings
|
||
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
|
||
- make it your own with custom **Theme colors**
|
||
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
|
||
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
|
||
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
|
||
|
||
---
|
||
|
||
## Original Model Evaluation
|
||
|
||
| Benchmark | Metric | 2B Params | 7B Params |
|
||
| ------------------------------ | ------------- | ----------- | --------- |
|
||
| [MMLU](https://arxiv.org/abs/2009.03300) | 5-shot, top-1 | 42.3 | 64.3 |
|
||
| [HellaSwag](https://arxiv.org/abs/1905.07830) | 0-shot |71.4 | 81.2 |
|
||
| [PIQA](https://arxiv.org/abs/1911.11641) | 0-shot | 77.3 | 81.2 |
|
||
| [SocialIQA](https://arxiv.org/abs/1904.09728) | 0-shot | 59.7 | 51.8 |
|
||
| [BooIQ](https://arxiv.org/abs/1905.10044) | 0-shot | 69.4 | 83.2 |
|
||
| [WinoGrande](https://arxiv.org/abs/1907.10641) | partial score | 65.4 | 72.3 |
|
||
| [CommonsenseQA](https://arxiv.org/abs/1811.00937) | 7-shot | 65.3 | 71.3 |
|
||
| [OpenBookQA](https://arxiv.org/abs/1809.02789) | | 47.8 | 52.8 |
|
||
| [ARC-e](https://arxiv.org/abs/1911.01547) | | 73.2 | 81.5 |
|
||
| [ARC-c](https://arxiv.org/abs/1911.01547) | | 42.1 | 53.2 |
|
||
| [TriviaQA](https://arxiv.org/abs/1705.03551) | 5-shot | 53.2 | 63.4 |
|
||
| [Natural Questions](https://github.com/google-research-datasets/natural-questions) | 5-shot | - | 23 |
|
||
| [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 | 22.0 | 32.3 |
|
||
| [MBPP](https://arxiv.org/abs/2108.07732) | 3-shot | 29.2 | 44.4 |
|
||
| [GSM8K](https://arxiv.org/abs/2110.14168) | maj@1 | 17.7 | 46.4 |
|
||
| [MATH](https://arxiv.org/abs/2108.07732) | 4-shot | 11.8 | 24.3 |
|
||
| [AGIEval](https://arxiv.org/abs/2304.06364) | | 24.2 | 41.7 |
|
||
| [BIG-Bench](https://arxiv.org/abs/2206.04615) | | 35.2 | 55.1 |
|
||
| **Average** | | **54.0** | **56.4** |
|
||
|
||
|
||
| Benchmark | Metric | 2B Params | 7B Params |
|
||
| ------------------------------ | ------------- | ----------- | --------- |
|
||
| [RealToxicity](https://arxiv.org/abs/2009.11462) | average | 6.86 | 7.90 |
|
||
| [BOLD](https://arxiv.org/abs/2101.11718) | | 45.57 | 49.08 |
|
||
| [CrowS-Pairs](https://aclanthology.org/2020.emnlp-main.154/) | top-1 | 45.82 | 51.33 |
|
||
| [BBQ Ambig](https://arxiv.org/abs/2110.08193v2) | 1-shot, top-1 | 62.58 | 92.54 |
|
||
| [BBQ Disambig](https://arxiv.org/abs/2110.08193v2) | top-1 | 54.62 | 71.99 |
|
||
| [Winogender](https://arxiv.org/abs/1804.09301) | top-1 | 51.25 | 54.17 |
|
||
| [TruthfulQA](https://arxiv.org/abs/2109.07958) | | 44.84 | 31.81 |
|
||
| [Winobias 1_2](https://arxiv.org/abs/1804.06876) | | 56.12 | 59.09 |
|
||
| [Winobias 2_2](https://arxiv.org/abs/1804.06876) | | 91.10 | 92.23 |
|
||
| [Toxigen](https://arxiv.org/abs/2203.09509) | | 29.77 | 39.59 | |