bartowski/Teuken-7B-instruct-research-v0.4-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/Teuken-7B-instruct-research-v0.4-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-06 05:07:12 +08:00
commit 6b77e9e331
28 changed files with 324 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4-f16.gguf filter=lfs diff=lfs merge=lfs -text
Teuken-7B-instruct-research-v0.4.imatrix filter=lfs diff=lfs merge=lfs -text

188
README.md Normal file
View File

@@ -0,0 +1,188 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model: openGPT-X/Teuken-7B-instruct-research-v0.4
metrics:
- accuracy
- bleu
license: other
language:
- de
- bg
- cs
- da
- el
- en
- es
- et
- fi
- fr
- ga
- hr
- hu
- it
- lt
- lv
- mt
- nl
- pl
- pt
- ro
- sl
- sv
- sk
---
## Llamacpp imatrix Quantizations of Teuken-7B-instruct-research-v0.4
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4132">b4132</a> for quantization.
Original model: https://huggingface.co/openGPT-X/Teuken-7B-instruct-research-v0.4
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
## Prompt format
No prompt format found, check original model page
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Teuken-7B-instruct-research-v0.4-f16.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-f16.gguf) | f16 | 14.91GB | false | Full F16 weights. |
| [Teuken-7B-instruct-research-v0.4-Q8_0.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q8_0.gguf) | Q8_0 | 7.93GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Teuken-7B-instruct-research-v0.4-Q6_K_L.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q6_K_L.gguf) | Q6_K_L | 6.80GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q6_K.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q6_K.gguf) | Q6_K | 6.55GB | false | Very high quality, near perfect, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q5_K_L.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q5_K_L.gguf) | Q5_K_L | 5.90GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q5_K_M.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q5_K_M.gguf) | Q5_K_M | 5.65GB | false | High quality, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q5_K_S.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q5_K_S.gguf) | Q5_K_S | 5.38GB | false | High quality, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q4_K_L.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_K_L.gguf) | Q4_K_L | 5.27GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q4_K_M.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_K_M.gguf) | Q4_K_M | 5.02GB | false | Good quality, default size for most use cases, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q4_K_S.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_K_S.gguf) | Q4_K_S | 4.70GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q3_K_XL.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q3_K_XL.gguf) | Q3_K_XL | 4.57GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Teuken-7B-instruct-research-v0.4-Q4_0.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_0.gguf) | Q4_0 | 4.48GB | false | Legacy format, generally not worth using over similarly sized formats |
| [Teuken-7B-instruct-research-v0.4-Q4_0_8_8.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_0_8_8.gguf) | Q4_0_8_8 | 4.46GB | false | Optimized for ARM and AVX inference. Requires 'sve' support for ARM (see details below). *Don't use on Mac*. |
| [Teuken-7B-instruct-research-v0.4-Q4_0_4_8.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_0_4_8.gguf) | Q4_0_4_8 | 4.46GB | false | Optimized for ARM inference. Requires 'i8mm' support (see details below). *Don't use on Mac*. |
| [Teuken-7B-instruct-research-v0.4-Q4_0_4_4.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q4_0_4_4.gguf) | Q4_0_4_4 | 4.46GB | false | Optimized for ARM inference. Should work well on all ARM chips, not for use with GPUs. *Don't use on Mac*. |
| [Teuken-7B-instruct-research-v0.4-IQ4_XS.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-IQ4_XS.gguf) | IQ4_XS | 4.32GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Teuken-7B-instruct-research-v0.4-Q3_K_L.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q3_K_L.gguf) | Q3_K_L | 4.32GB | false | Lower quality but usable, good for low RAM availability. |
| [Teuken-7B-instruct-research-v0.4-Q3_K_M.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q3_K_M.gguf) | Q3_K_M | 4.15GB | false | Low quality. |
| [Teuken-7B-instruct-research-v0.4-IQ3_M.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-IQ3_M.gguf) | IQ3_M | 3.95GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [Teuken-7B-instruct-research-v0.4-Q3_K_S.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q3_K_S.gguf) | Q3_K_S | 3.84GB | false | Low quality, not recommended. |
| [Teuken-7B-instruct-research-v0.4-IQ3_XS.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-IQ3_XS.gguf) | IQ3_XS | 3.70GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [Teuken-7B-instruct-research-v0.4-Q2_K_L.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q2_K_L.gguf) | Q2_K_L | 3.68GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [Teuken-7B-instruct-research-v0.4-Q2_K.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-Q2_K.gguf) | Q2_K | 3.43GB | false | Very low quality but surprisingly usable. |
| [Teuken-7B-instruct-research-v0.4-IQ2_M.gguf](https://huggingface.co/bartowski/Teuken-7B-instruct-research-v0.4-GGUF/blob/main/Teuken-7B-instruct-research-v0.4-IQ2_M.gguf) | IQ2_M | 3.26GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/Teuken-7B-instruct-research-v0.4-GGUF --include "Teuken-7B-instruct-research-v0.4-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/Teuken-7B-instruct-research-v0.4-GGUF --include "Teuken-7B-instruct-research-v0.4-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (Teuken-7B-instruct-research-v0.4-Q8_0) or download them all in place (./)
</details>
## Q4_0_X_X information
<details>
<summary>Click to view Q4_0_X_X information</summary>
These are *NOT* for Metal (Apple) or GPU (nvidia/AMD/intel) offloading, only ARM chips (and certain AVX2/AVX512 CPUs).
If you're using an ARM chip, the Q4_0_X_X quants will have a substantial speedup. Check out Q4_0_4_4 speed comparisons [on the original pull request](https://github.com/ggerganov/llama.cpp/pull/5780#pullrequestreview-21657544660)
To check which one would work best for your ARM chip, you can check [AArch64 SoC features](https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html) (thanks EloyOn!).
If you're using a CPU that supports AVX2 or AVX512 (typically server CPUs and AMD's latest Zen5 CPUs) and are not offloading to a GPU, the Q4_0_8_8 may offer a nice speed as well:
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

3
Teuken-7B-instruct-research-v0.4-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d7a6088cb64258d547ae65e7a0289d38afd4614b64b6542fe989a3d8a6fe31c6
size 3264938848

3
Teuken-7B-instruct-research-v0.4-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ef34b42dc7ee32ddf73311dc98fd7f7d96eb91a1c2d82a2441c66ab36bf02d19
size 3947879008

3
Teuken-7B-instruct-research-v0.4-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b2df1c683971904cfbeddb7cc31781932f0cc0af0a6513882ec7ac182ba58b90
size 3698448992

3
Teuken-7B-instruct-research-v0.4-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:479081aef17b3a4d58038e7a9f801d9c9013706db296fc4440dcad9a062b43e1
size 4323531360

3
Teuken-7B-instruct-research-v0.4-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b7c7f15b520ee6b48555e7291b0e64d49ed968f7c1a7907702292b5aac3da4c4
size 3433290336

3
Teuken-7B-instruct-research-v0.4-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0db7f1318bdf316033c80756c41a8cb970070b459a2e432f940bb958c8e1f841
size 3681964896

3
Teuken-7B-instruct-research-v0.4-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2b34e203b708d8ac21831379994150f42a860f8453626c97c19ceb661099729f
size 4321958496

3
Teuken-7B-instruct-research-v0.4-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0033c4f458da168f38eabce07018fcf0cb2345bd985a83d782b2510aea40a1ca
size 4147698272

3
Teuken-7B-instruct-research-v0.4-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f62db3737fe272d60a8ea8c4ec4bdcd7845828fb0af8920af43c145c13e20b36
size 3844594272

3
Teuken-7B-instruct-research-v0.4-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5506001ddd2bd67cca4a1c248618cac6f8d52f6eb295b81455ab3c7e5cb3c07a
size 4570633056

3
Teuken-7B-instruct-research-v0.4-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b7238f9f36680ba3c4ed3e34b19c4d5b94213fd85dfbd831cea285bad0580746
size 4477803104

3
Teuken-7B-instruct-research-v0.4-Q4_0_4_4.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:795f0a9b027bbee371a1b9c30756edc76738b2eb91e6e82d974ff438ab0c7b45
size 4464040544

3
Teuken-7B-instruct-research-v0.4-Q4_0_4_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:15e80e8e1ec7487b802167d43b97c55b22f0abd3db80c83b54a57011b0eeff5c
size 4464040544

3
Teuken-7B-instruct-research-v0.4-Q4_0_8_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3c8fa1fbcb7e9893417e05bebdac2445802f089d5ddf9b282d94df43249dae20
size 4464040544

3
Teuken-7B-instruct-research-v0.4-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ade4cfe1a19bc023e5234b46962a3e0c4739664a642d9c53078b1eac8a3dd33f
size 5267542880

3
Teuken-7B-instruct-research-v0.4-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:99d2c70d1a7b3fdc3e3221bc6852d4d3a540ca8a7016c804c84635bba50ac023
size 5018868320

3
Teuken-7B-instruct-research-v0.4-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8aa8985fa73d7c75f6a5350ee80e233a4a4dcbbf9fc530c002e23f8d2110bdc8
size 4698528352

3
Teuken-7B-instruct-research-v0.4-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4f19cd746b61d0711b9596d42abeea952532535020d4f444938039de36f18da
size 5903504224

3
Teuken-7B-instruct-research-v0.4-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:feeb0af21b007855db3535f6cf947ce4b47f715203961fb197482da3d170c470
size 5654829664

3
Teuken-7B-instruct-research-v0.4-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ca6608531e1b58071ef31a9d99aed16e8e42fdbdd53d9a885226369ad3715f0
size 5377350240

3
Teuken-7B-instruct-research-v0.4-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e671709fe6a61b96210c9a2a85c81494842d3e4f5c9e6e65d584dc8d6f1732e8
size 6547298912

3
Teuken-7B-instruct-research-v0.4-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:36de525b44c1febd1968f212fcd3d7e4d2089532ae7195cf2f2f23817c11f4e6
size 6795973472

3
Teuken-7B-instruct-research-v0.4-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:628fcbdd2961fe282368e92c24f2e2878b4fe392163a224aaf7a4aada69c7278
size 7925551968

3
Teuken-7B-instruct-research-v0.4-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ca0b83abfcc5ed2158a06af5d73a09d260eb0b84ef1af1a85062a17955b37a21
size 14912232000

3
Teuken-7B-instruct-research-v0.4.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5738130af502a5fa540659c2e2fae731e2e5bd627c136cbf34c91d557ef9f307
size 4873482

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}