bartowski/arcee-ai_Homunculus-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/arcee-ai_Homunculus-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-23 21:16:06 +08:00
commit 71614e8675
30 changed files with 306 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

49
.gitattributes vendored Normal file
View File

@@ -0,0 +1,49 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
arcee-ai_Homunculus.imatrix filter=lfs diff=lfs merge=lfs -text

175
README.md Normal file
View File

@@ -0,0 +1,175 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model: arcee-ai/Homunculus
base_model_relation: quantized
license: apache-2.0
language:
- en
---
## Llamacpp imatrix Quantizations of Homunculus by arcee-ai
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b5568">b5568</a> for quantization.
Original model: https://huggingface.co/arcee-ai/Homunculus
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
## Prompt format
```
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Homunculus-bf16.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-bf16.gguf) | bf16 | 24.92GB | false | Full BF16 weights. |
| [Homunculus-Q8_0.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q8_0.gguf) | Q8_0 | 13.24GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Homunculus-Q6_K_L.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q6_K_L.gguf) | Q6_K_L | 10.60GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Homunculus-Q6_K.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q6_K.gguf) | Q6_K | 10.23GB | false | Very high quality, near perfect, *recommended*. |
| [Homunculus-Q5_K_L.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q5_K_L.gguf) | Q5_K_L | 9.36GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Homunculus-Q5_K_M.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q5_K_M.gguf) | Q5_K_M | 8.88GB | false | High quality, *recommended*. |
| [Homunculus-Q5_K_S.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q5_K_S.gguf) | Q5_K_S | 8.68GB | false | High quality, *recommended*. |
| [Homunculus-Q4_K_L.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q4_K_L.gguf) | Q4_K_L | 8.20GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Homunculus-Q4_1.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q4_1.gguf) | Q4_1 | 7.95GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [Homunculus-Q4_K_M.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q4_K_M.gguf) | Q4_K_M | 7.62GB | false | Good quality, default size for most use cases, *recommended*. |
| [Homunculus-Q3_K_XL.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q3_K_XL.gguf) | Q3_K_XL | 7.37GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Homunculus-Q4_K_S.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q4_K_S.gguf) | Q4_K_S | 7.26GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Homunculus-Q4_0.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q4_0.gguf) | Q4_0 | 7.24GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [Homunculus-IQ4_NL.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ4_NL.gguf) | IQ4_NL | 7.24GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [Homunculus-IQ4_XS.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ4_XS.gguf) | IQ4_XS | 6.88GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Homunculus-Q3_K_L.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q3_K_L.gguf) | Q3_K_L | 6.69GB | false | Lower quality but usable, good for low RAM availability. |
| [Homunculus-Q3_K_M.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q3_K_M.gguf) | Q3_K_M | 6.21GB | false | Low quality. |
| [Homunculus-IQ3_M.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ3_M.gguf) | IQ3_M | 5.85GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [Homunculus-Q2_K_L.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q2_K_L.gguf) | Q2_K_L | 5.67GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [Homunculus-Q3_K_S.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q3_K_S.gguf) | Q3_K_S | 5.66GB | false | Low quality, not recommended. |
| [Homunculus-IQ3_XS.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ3_XS.gguf) | IQ3_XS | 5.44GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [Homunculus-IQ3_XXS.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ3_XXS.gguf) | IQ3_XXS | 5.06GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [Homunculus-Q2_K.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-Q2_K.gguf) | Q2_K | 4.91GB | false | Very low quality but surprisingly usable. |
| [Homunculus-IQ2_M.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ2_M.gguf) | IQ2_M | 4.55GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
| [Homunculus-IQ2_S.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ2_S.gguf) | IQ2_S | 4.25GB | false | Low quality, uses SOTA techniques to be usable. |
| [Homunculus-IQ2_XS.gguf](https://huggingface.co/bartowski/arcee-ai_Homunculus-GGUF/blob/main/arcee-ai_Homunculus-IQ2_XS.gguf) | IQ2_XS | 4.02GB | false | Low quality, uses SOTA techniques to be usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/arcee-ai_Homunculus-GGUF --include "arcee-ai_Homunculus-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/arcee-ai_Homunculus-GGUF --include "arcee-ai_Homunculus-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (arcee-ai_Homunculus-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggerganov/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggerganov/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggerganov/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

3
arcee-ai_Homunculus-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5fed319513e69959188e1cd15ab2c4ddaa31dd70c6daab7cf59d969eecf715a5
size 4550968448

3
arcee-ai_Homunculus-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47923a69a02b8d0caa73bbb32cbe1af7f16b2ee9cf5e2d97aaf7116c4c92ec60
size 4254418048

3
arcee-ai_Homunculus-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6f3bd07d3319c216caea50eb2bb1add77b89ed51b60f541b00b97a08d27e0f97
size 4020306048

3
arcee-ai_Homunculus-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fb62d6fd7eddad6574ad0a136f36c9dc5c1e361f95b288ded09488838db7853
size 5852190848

3
arcee-ai_Homunculus-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:af296e3ed36d78e120172440642e5566feb41d0a5480d5f8c883a875a27806b6
size 5436446848

3
arcee-ai_Homunculus-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:617e754857cb5ebd4bc6915f107734f284d60d3c88ff7bf2359d234362867501
size 5061330048

3
arcee-ai_Homunculus-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ece00ad0da5806f5dc0be106a8b1c3dd51a2af6f42677afb3c7f40f26809879
size 7241886848

3
arcee-ai_Homunculus-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:01ee55b735c19c69da26f99d444391a50b8c36685879c3c80c3f61715c26217a
size 6883384448

3
arcee-ai_Homunculus-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:14bd7a18cbdc413812a6180460ceb49b0e0458b5d908afd49428d755f6e15384
size 4910290048

3
arcee-ai_Homunculus-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c0b7a73c9405f6a1fa7a43e90a90fc7b03ce113f14e370697b3359f1583c8b2a
size 5668690048

3
arcee-ai_Homunculus-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d4ffd0cb568ec37aaf1ab702db31aa23c0a80de68825df947cbfc3ba50392e50
size 6691461248

3
arcee-ai_Homunculus-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:659a734395174cf70d265ad0aa388d58f8abb7bbdc4ea0613bd3e0356b37b96e
size 6213048448

3
arcee-ai_Homunculus-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:465e35a70b2923a2d8ba283afd939b355e217c87f53ccd5e28957af6a6d64d24
size 5664184448

3
arcee-ai_Homunculus-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:601e6949b4d425fc09134da1a1dcf91b6ae1ab6daf315c56a30616b5af606921
size 7370987648

3
arcee-ai_Homunculus-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:76348564095c147dc82868f1382551c0a22b0333f1dc4741924d545543470473
size 7238610048

3
arcee-ai_Homunculus-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b613140688eceb145cb7e6400d0c95d74fe2e8302900ab7658d6620fe52e6bf4
size 7945784448

3
arcee-ai_Homunculus-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0bd676e091f33385cd2b40f5681dd5062ab21d7812232074662fe218772a3c18
size 8197560448

3
arcee-ai_Homunculus-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:243a41543cc239612465b0474afb782a5cde130d836b7cbd60d1120295269318
size 7621176448

3
arcee-ai_Homunculus-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:37fda4c1765c15802b4d517db4330ddd37415bbdc3bafb2c58eb8f377239abdf
size 7264169088

3
arcee-ai_Homunculus-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:37a4e88a40a0a2f550fca761c6eeb2a73bf446012652677d7eb22637b1ada40b
size 9364101248

3
arcee-ai_Homunculus-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:483acfbde4dde7a7e605c9db2ced7665da7cd176037fe1a89f907198602220cf
size 8884792448

3
arcee-ai_Homunculus-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:91a5a114cc68fc0f043168f02486fdf3c63777547c5379cb6c6b29821d55a9f8
size 8675896448

3
arcee-ai_Homunculus-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d3c3aa362632feae6c275d31a8ca9456c40f67aa31f16b710e5f9b66dd8724aa
size 10227384448

3
arcee-ai_Homunculus-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9b6426e7d9d3e773bfd5d10fc415f8cec4ab5ca6b60dc5d9f3b697593ed2324e
size 10603550848

3
arcee-ai_Homunculus-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c440bdce03ed4c2d387021031334f70e8a7a167b9fa11cf1f1e40004cc5ef323
size 13244651648

3
arcee-ai_Homunculus-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:207bcffc2921e61a4e537f2c7ac0789f714253188b3defacaaf3e56d011c974e
size 24924395360

3
arcee-ai_Homunculus.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9ae89cb322f7d0e8c7e79cb46ddf93d837b1a5163d09a5e6df653ee6186dbf7b
size 7054418

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}