bartowski/Arch-Function-1.5B-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update metadata with huggingface_hub

Browse Source
This commit is contained in:
ai-modelscope
2024-11-27 18:40:18 +08:00
parent 98200d1d8a
commit 49f3567f03
22 changed files with 204 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
.gitattributes vendored
View File

@@ -1,47 +1,54 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B-f16.gguf filter=lfs diff=lfs merge=lfs -text
Arch-Function-1.5B.imatrix filter=lfs diff=lfs merge=lfs -text

3
Arch-Function-1.5B-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:81e059451e5f56832a059cad29162e4dc44b6405d5382d10f7afada2dbe156e9
size 776664416

3
Arch-Function-1.5B-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5bba28d7f3f98baf4376d2064c63c0c85ffc8f1df09d8775333f173a3a33ad0b
size 895732064

3
Arch-Function-1.5B-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7adb9f3eeb9ce147348ac1901d0725bea69453c9bbc12411bcf37e8b68e286cf
size 880163168

3
Arch-Function-1.5B-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2cead0c827e868a48c4ea6adfe8330051f39d74388c3fcf5ffb9ecef360e1dcb
size 936683360

3
Arch-Function-1.5B-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fad39bc70a5f33eaf916cf58d23d8b278f9bda9a236922062ceb8321c4fabdea
size 937535840

3
Arch-Function-1.5B-Q4_0_4_4.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a56fb68e0929a24a31d75e00338af91f9ec99db47776bf589373c9edf22a27e5
size 934955360

3
Arch-Function-1.5B-Q4_0_4_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b1c65de547a01535d6993f4b68a073ae89c5910957e2d7f985ce7617c97928e
size 934955360

3
Arch-Function-1.5B-Q4_0_8_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a67442f5692fa9f47ebff87f2319b54a7f4306ac241a3243a364090d33b1719e
size 934955360

3
Arch-Function-1.5B-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1caa30cd14abe5169eb096a2664ea728d10f0f7b5236836943e13905ced655cf
size 1042569056

3
Arch-Function-1.5B-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f095addb7215ba3880dca0da7a98df76f3e6438869779c9648ca715374df1eda
size 986048864

3
Arch-Function-1.5B-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c0cdbc3d03fb3b252b58cf8748e8ef93c4482ac248fc79266a3600604c216af9
size 940312928

3
Arch-Function-1.5B-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:76eae2db1db1642979328ba69401840183c1879a294b93f1da30feb27c969660
size 1181570912

3
Arch-Function-1.5B-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7a8ab009c645a2c6902ac3791f8ed3e7e0efa169e22d3511baa30a5863a7c90c
size 1125050720

3
Arch-Function-1.5B-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f6186e08b049f48d121496ea006b1f76279bffa4516043d46681554fb0f726d4
size 1098729824

3
Arch-Function-1.5B-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6edcc59ed491326423ce356d3160a25f86a061e83516ea24e9f6625cc2084038
size 1272740192

3
Arch-Function-1.5B-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09c785cf1eaf7a7f51c9021587931097ef7ea525c4f2136201e73df3e09ebf98
size 1329260384

3
Arch-Function-1.5B-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e09965eaa91e4900b76e1f0b3a11f26c578df3ab628bf34852b369662a909075
size 1646573408

3
Arch-Function-1.5B-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5461e083d5f2642516726f577ec653eef4ffdc5dfde2fdcb96878fefd142373e
size 3093669472

3
Arch-Function-1.5B.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9396b8c25e24da4fbcc8b8db302d8bdc1a6cb815fde90b3734b2abdf4ce11b9c
size 2042214

158
README.md
View File

@@ -1,47 +1,123 @@
---
license: Apache License 2.0
#model-type:
##如 gpt、phi、llama、chatglm、baichuan 等
#- gpt
#domain:
##如 nlp、cv、audio、multi-modal
#- nlp
#language:
##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
#- cn
#metrics:
##如 CIDEr、Blue、ROUGE 等
#- CIDEr
#tags:
##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
#- pretrained
#tools:
##如 vllm、fastchat、llamacpp、AdaSeq 等
#- vllm
base_model: katanemo/Arch-Function-1.5B
language:
- en
license: other
license_name: katanemo-research
license_link: https://huggingface.co/katanemolabs/Arch-Function-1.5B/blob/main/LICENSE
pipeline_tag: text-generation
quantized_by: bartowski
---
### 当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。
#### 您可以通过如下git clone命令，或者ModelScope SDK来下载模型
SDK下载
```bash
#安装ModelScope
pip install modelscope
## Llamacpp imatrix Quantizations of Arch-Function-1.5B
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3930">b3930</a> for quantization.
Original model: https://huggingface.co/katanemo/Arch-Function-1.5B
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
## Prompt format
```
```python
#SDK模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('bartowski/Arch-Function-1.5B-GGUF')
```
Git下载
```
#Git模型下载
git clone https://www.modelscope.cn/bartowski/Arch-Function-1.5B-GGUF.git
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
<p style="color: lightgrey;">如果您是本模型的贡献者，我们邀请您根据<a href="https://modelscope.cn/docs/ModelScope%E6%A8%A1%E5%9E%8B%E6%8E%A5%E5%85%A5%E6%B5%81%E7%A8%8B%E6%A6%82%E8%A7%88" style="color: lightgrey; text-decoration: underline;">模型贡献文档</a>，及时完善模型卡片内容。</p>
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Arch-Function-1.5B-f16.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-f16.gguf) | f16 | 3.09GB | false | Full F16 weights. |
| [Arch-Function-1.5B-Q8_0.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q8_0.gguf) | Q8_0 | 1.65GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Arch-Function-1.5B-Q6_K_L.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q6_K_L.gguf) | Q6_K_L | 1.33GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Arch-Function-1.5B-Q6_K.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q6_K.gguf) | Q6_K | 1.27GB | false | Very high quality, near perfect, *recommended*. |
| [Arch-Function-1.5B-Q5_K_L.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q5_K_L.gguf) | Q5_K_L | 1.18GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Arch-Function-1.5B-Q5_K_M.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q5_K_M.gguf) | Q5_K_M | 1.13GB | false | High quality, *recommended*. |
| [Arch-Function-1.5B-Q5_K_S.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q5_K_S.gguf) | Q5_K_S | 1.10GB | false | High quality, *recommended*. |
| [Arch-Function-1.5B-Q4_K_L.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_K_L.gguf) | Q4_K_L | 1.04GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Arch-Function-1.5B-Q4_K_M.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_K_M.gguf) | Q4_K_M | 0.99GB | false | Good quality, default size for must use cases, *recommended*. |
| [Arch-Function-1.5B-Q4_K_S.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_K_S.gguf) | Q4_K_S | 0.94GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Arch-Function-1.5B-Q4_0.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_0.gguf) | Q4_0 | 0.94GB | false | Legacy format, generally not worth using over similarly sized formats |
| [Arch-Function-1.5B-Q3_K_XL.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q3_K_XL.gguf) | Q3_K_XL | 0.94GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Arch-Function-1.5B-Q4_0_8_8.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_0_8_8.gguf) | Q4_0_8_8 | 0.93GB | false | Optimized for ARM inference. Requires 'sve' support (see link below). *Don't use on Mac or Windows*. |
| [Arch-Function-1.5B-Q4_0_4_8.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_0_4_8.gguf) | Q4_0_4_8 | 0.93GB | false | Optimized for ARM inference. Requires 'i8mm' support (see link below). *Don't use on Mac or Windows*. |
| [Arch-Function-1.5B-Q4_0_4_4.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q4_0_4_4.gguf) | Q4_0_4_4 | 0.93GB | false | Optimized for ARM inference. Should work well on all ARM chips, pick this if you're unsure. *Don't use on Mac or Windows*. |
| [Arch-Function-1.5B-IQ4_XS.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-IQ4_XS.gguf) | IQ4_XS | 0.90GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Arch-Function-1.5B-Q3_K_L.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-Q3_K_L.gguf) | Q3_K_L | 0.88GB | false | Lower quality but usable, good for low RAM availability. |
| [Arch-Function-1.5B-IQ3_M.gguf](https://huggingface.co/bartowski/Arch-Function-1.5B-GGUF/blob/main/Arch-Function-1.5B-IQ3_M.gguf) | IQ3_M | 0.78GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using.
Thanks!
## Downloading using huggingface-cli
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/Arch-Function-1.5B-GGUF --include "Arch-Function-1.5B-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/Arch-Function-1.5B-GGUF --include "Arch-Function-1.5B-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (Arch-Function-1.5B-Q8_0) or download them all in place (./)
## Q4_0_X_X
These are *NOT* for Metal (Apple) offloading, only ARM chips.
If you're using an ARM chip, the Q4_0_X_X quants will have a substantial speedup. Check out Q4_0_4_4 speed comparisons [on the original pull request](https://github.com/ggerganov/llama.cpp/pull/5780#pullrequestreview-21657544660)
To check which one would work best for your ARM chip, you can check [AArch64 SoC features](https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html) (thanks EloyOn!).
## Which file should I choose?
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
Thank you ZeroWw for the inspiration to experiment with embed/output
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}