bartowski/katanemo_Arch-Function-Chat-3B-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/katanemo_Arch-Function-Chat-3B-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-22 14:24:14 +08:00
commit 699a572c17
28 changed files with 295 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
.gitattributes vendored Normal file
View File

@@ -0,0 +1,47 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

175
README.md Normal file
View File

@@ -0,0 +1,175 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model_relation: quantized
license_name: katanemo-research
base_model: katanemo/Arch-Function-Chat-3B
language:
- en
license: other
license_link: https://huggingface.co/katanemo/Arch-Function-Chat-3B/blob/main/LICENSE
---
## Llamacpp imatrix Quantizations of Arch-Function-Chat-3B by katanemo
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b5010">b5010</a> for quantization.
Original model: https://huggingface.co/katanemo/Arch-Function-Chat-3B
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
## Prompt format
```
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Arch-Function-Chat-3B-bf16.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-bf16.gguf) | bf16 | 6.18GB | false | Full BF16 weights. |
| [Arch-Function-Chat-3B-Q8_0.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q8_0.gguf) | Q8_0 | 3.29GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Arch-Function-Chat-3B-Q6_K_L.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q6_K_L.gguf) | Q6_K_L | 2.61GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Arch-Function-Chat-3B-Q6_K.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q6_K.gguf) | Q6_K | 2.54GB | false | Very high quality, near perfect, *recommended*. |
| [Arch-Function-Chat-3B-Q5_K_L.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q5_K_L.gguf) | Q5_K_L | 2.30GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Arch-Function-Chat-3B-Q5_K_M.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q5_K_M.gguf) | Q5_K_M | 2.22GB | false | High quality, *recommended*. |
| [Arch-Function-Chat-3B-Q5_K_S.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q5_K_S.gguf) | Q5_K_S | 2.17GB | false | High quality, *recommended*. |
| [Arch-Function-Chat-3B-Q4_K_L.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q4_K_L.gguf) | Q4_K_L | 2.01GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Arch-Function-Chat-3B-Q4_1.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q4_1.gguf) | Q4_1 | 2.00GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [Arch-Function-Chat-3B-Q4_K_M.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q4_K_M.gguf) | Q4_K_M | 1.93GB | false | Good quality, default size for most use cases, *recommended*. |
| [Arch-Function-Chat-3B-Q4_K_S.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q4_K_S.gguf) | Q4_K_S | 1.83GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Arch-Function-Chat-3B-Q4_0.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q4_0.gguf) | Q4_0 | 1.83GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [Arch-Function-Chat-3B-IQ4_NL.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-IQ4_NL.gguf) | IQ4_NL | 1.83GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [Arch-Function-Chat-3B-Q3_K_XL.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q3_K_XL.gguf) | Q3_K_XL | 1.78GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Arch-Function-Chat-3B-IQ4_XS.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-IQ4_XS.gguf) | IQ4_XS | 1.74GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Arch-Function-Chat-3B-Q3_K_L.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q3_K_L.gguf) | Q3_K_L | 1.71GB | false | Lower quality but usable, good for low RAM availability. |
| [Arch-Function-Chat-3B-Q3_K_M.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q3_K_M.gguf) | Q3_K_M | 1.59GB | false | Low quality. |
| [Arch-Function-Chat-3B-IQ3_M.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-IQ3_M.gguf) | IQ3_M | 1.49GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [Arch-Function-Chat-3B-Q3_K_S.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q3_K_S.gguf) | Q3_K_S | 1.45GB | false | Low quality, not recommended. |
| [Arch-Function-Chat-3B-IQ3_XS.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-IQ3_XS.gguf) | IQ3_XS | 1.39GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [Arch-Function-Chat-3B-Q2_K_L.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q2_K_L.gguf) | Q2_K_L | 1.35GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [Arch-Function-Chat-3B-IQ3_XXS.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-IQ3_XXS.gguf) | IQ3_XXS | 1.28GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [Arch-Function-Chat-3B-Q2_K.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-Q2_K.gguf) | Q2_K | 1.27GB | false | Very low quality but surprisingly usable. |
| [Arch-Function-Chat-3B-IQ2_M.gguf](https://huggingface.co/bartowski/katanemo_Arch-Function-Chat-3B-GGUF/blob/main/katanemo_Arch-Function-Chat-3B-IQ2_M.gguf) | IQ2_M | 1.14GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/katanemo_Arch-Function-Chat-3B-GGUF --include "katanemo_Arch-Function-Chat-3B-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/katanemo_Arch-Function-Chat-3B-GGUF --include "katanemo_Arch-Function-Chat-3B-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (katanemo_Arch-Function-Chat-3B-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggerganov/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggerganov/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggerganov/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
katanemo_Arch-Function-Chat-3B-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac67c641baab04c1ba6af7d8c6c3dc8864fb4d314577a661f6bfc2f136a5e4f2
size 1140516256

3
katanemo_Arch-Function-Chat-3B-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e28456542cd4e3f13c01092c47bcf583642ff50be49ce31d581052d31a0c19e5
size 1488895392

3
katanemo_Arch-Function-Chat-3B-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d7f71cffd0993e17f4662be5b9e15147b7fb1fbf7734ad749cb79d08eed0bdc1
size 1391836576

3
katanemo_Arch-Function-Chat-3B-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:482dd5e9d61334f665b2704b71958133909041183f522ec15f48fa38e67d264a
size 1282827680

3
katanemo_Arch-Function-Chat-3B-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22d7efb46eaa4dffca75bb4cad4a4093fc65d7140e0a93efbf377ca46b0574be
size 1825209760

3
katanemo_Arch-Function-Chat-3B-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0e19bbf1fb968df94264c9298f6b0dbc4cdb40ffb202101c573a8f3e331c2f25
size 1739095456

3
katanemo_Arch-Function-Chat-3B-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:21eabb4c183749a1b7d241db1dddc5099f865a47eada4b712a9550760b19fa2f
size 1274756512

3
katanemo_Arch-Function-Chat-3B-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b21dcc6c16324bbdd43a9293431eddfcc289e0c15c0c6743d5baa6ad4321f2e
size 1350116768

3
katanemo_Arch-Function-Chat-3B-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ece8d0acf17427be9921197f674f89af5475698057d6f918451bff0252b19b6d
size 1707392416

3
katanemo_Arch-Function-Chat-3B-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5ca21e9dff281a7bc886dc8630c0bf697acf2112c04281bb6068e6da303a44b1
size 1590476192

3
katanemo_Arch-Function-Chat-3B-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae1db7519c71e985d07ecd21ea7e8848ea61624e8f9d70cc951ac7bd465529d1
size 1454357920

3
katanemo_Arch-Function-Chat-3B-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7bbcb9ccd39bfadfb8ede381291ba95e48d28f332073a30429508286cdbb4af5
size 1782752672

3
katanemo_Arch-Function-Chat-3B-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:70c8f1b2d37c6901bcfcad8abd355dc7a68b942f550181a5de06ff1c15c271bf
size 1828486560

3
katanemo_Arch-Function-Chat-3B-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8f6d059bdaca30965516e59abe4fdb6076ff423e0db452ee8d9cd7e1d84baaab
size 1996258720

3
katanemo_Arch-Function-Chat-3B-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:51d82c2e88bd27021e88ea7c51b492cc1b9f2f6fb50cf1aed86a3713b5a93e4d
size 2005263776

3
katanemo_Arch-Function-Chat-3B-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f59dbef397bf1364b5f0a2c23a7f67c40ec63cc666036c4e7615fa7d79d4e1a0
size 1929903520

3
katanemo_Arch-Function-Chat-3B-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:789847559604e0d29fa84b81293716e0a026763c77b43c8fb8a7086227bb5d01
size 1834384800

3
katanemo_Arch-Function-Chat-3B-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c202a7f2f3b262ffec3c7e029e47bd3dc47d6a6810b2db448500cd2a461e310e
size 2300175776

3
katanemo_Arch-Function-Chat-3B-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96a00313b3890135acf0d0861d2723913bb7fa4c00d1f65ee82ac0ade0150d78
size 2224815520

3
katanemo_Arch-Function-Chat-3B-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:29e301f85ff03ed1a5a798f8a222d3e957bbbc4bfeadd92e182ba5e4e8e0bcaf
size 2169666976

3
katanemo_Arch-Function-Chat-3B-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:465ced08f114d999b612f641aa10570abab1ee74c34303966298e6c6e9f09653
size 2538159520

3
katanemo_Arch-Function-Chat-3B-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:89fe399e93edc0bc5a3382440952542f7b951f0f8dd3c9d30d4204139ddd5833
size 2613519776

3
katanemo_Arch-Function-Chat-3B-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bff95422400d54f045018dc44a58c90ba14729b8d3d013b237bef98ed021cb27
size 3285476768

3
katanemo_Arch-Function-Chat-3B-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:041a47f892b6508a81f76a3e1530277a4e3e38bc5f4687ce1b1a7de58dcd289f
size 6178317440

BIN
katanemo_Arch-Function-Chat-3B.imatrix Normal file
View File

Binary file not shown.