commit 0354031f039786afb5499ed942fe78eab95413a1 Author: ModelHub XC Date: Wed Sep 23 22:07:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Halu-OAS-8B-Llama3-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..5eb0148 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,50 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.f16.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Halu-OAS-8B-Llama3.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Halu-OAS-8B-Llama3.IQ3_M.gguf b/Halu-OAS-8B-Llama3.IQ3_M.gguf new file mode 100644 index 0000000..eacc3c5 --- /dev/null +++ b/Halu-OAS-8B-Llama3.IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7f24a969ab42eb8e9445553d8f67e4e17d53d7b78110388183ee50edef14e606 +size 3784823552 diff --git a/Halu-OAS-8B-Llama3.IQ3_S.gguf b/Halu-OAS-8B-Llama3.IQ3_S.gguf new file mode 100644 index 0000000..1e505e7 --- /dev/null +++ b/Halu-OAS-8B-Llama3.IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5fc053c2b1d4ad6329e9709d0fb5145c6aaa41f51bbb354fc325769fe0dfc150 +size 3682325248 diff --git a/Halu-OAS-8B-Llama3.IQ3_XS.gguf b/Halu-OAS-8B-Llama3.IQ3_XS.gguf new file mode 100644 index 0000000..5d24cb2 --- /dev/null +++ b/Halu-OAS-8B-Llama3.IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86b8b0861deab706bdd616006569b9dd888c5f27c0e024e37e44d9b5d1b27192 +size 3518747392 diff --git a/Halu-OAS-8B-Llama3.IQ4_XS.gguf b/Halu-OAS-8B-Llama3.IQ4_XS.gguf new file mode 100644 index 0000000..e01e20e --- /dev/null +++ b/Halu-OAS-8B-Llama3.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6f1539be1a2c816a192e9225ef5eb75a631f704998bd436f6e64f6ba150ec1ab +size 4484363008 diff --git a/Halu-OAS-8B-Llama3.Q2_K.gguf b/Halu-OAS-8B-Llama3.Q2_K.gguf new file mode 100644 index 0000000..4a4adfa --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59389d7866d06f11a7857370346c0f6d01d16d5433675969b366e244d81bd7f0 +size 3179131648 diff --git a/Halu-OAS-8B-Llama3.Q3_K_L.gguf b/Halu-OAS-8B-Llama3.Q3_K_L.gguf new file mode 100644 index 0000000..ca81d14 --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e457753ddf8942d15e74f0361b22ebdb679e3d993e09635aab5e8b6955b8da18 +size 4321956608 diff --git a/Halu-OAS-8B-Llama3.Q3_K_M.gguf b/Halu-OAS-8B-Llama3.Q3_K_M.gguf new file mode 100644 index 0000000..22f340a --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:924ee5bd5401d00a1f355949c608cd7073ff6ad95544ce42190f70179ea1fb94 +size 4018918144 diff --git a/Halu-OAS-8B-Llama3.Q3_K_S.gguf b/Halu-OAS-8B-Llama3.Q3_K_S.gguf new file mode 100644 index 0000000..a57d07f --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fe6ce45a4346d5ca71e8215f91b5867331338b9c313427beac87bb6f1b9d84b +size 3664499456 diff --git a/Halu-OAS-8B-Llama3.Q4_K_M.gguf b/Halu-OAS-8B-Llama3.Q4_K_M.gguf new file mode 100644 index 0000000..06353e3 --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0713d4690f19d3c996811b5af541c13a7e6559e86ac639486d05d06c8ff0dd99 +size 4920734464 diff --git a/Halu-OAS-8B-Llama3.Q4_K_S.gguf b/Halu-OAS-8B-Llama3.Q4_K_S.gguf new file mode 100644 index 0000000..6cbb1d8 --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7bf365719230a66d4ebc4979f80e9de9ca39ae64c12cf9208c9184ca472cef93 +size 4692669184 diff --git a/Halu-OAS-8B-Llama3.Q5_K_M.gguf b/Halu-OAS-8B-Llama3.Q5_K_M.gguf new file mode 100644 index 0000000..297c09c --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1edcc54fd1548b33644797cf0f4c812c24624bec158ce11698e53521991dbca6 +size 5732987648 diff --git a/Halu-OAS-8B-Llama3.Q5_K_S.gguf b/Halu-OAS-8B-Llama3.Q5_K_S.gguf new file mode 100644 index 0000000..602283c --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83d2e8b21f4c48f7f5266d410ba21108bd57a907482a813d3effbd5c993eb6d7 +size 5599294208 diff --git a/Halu-OAS-8B-Llama3.Q6_K.gguf b/Halu-OAS-8B-Llama3.Q6_K.gguf new file mode 100644 index 0000000..cec254b --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e59ac4258e33b6a113e183e0a87a31cffb8a6500da286166db720937db6de901 +size 6596006656 diff --git a/Halu-OAS-8B-Llama3.Q8_0.gguf b/Halu-OAS-8B-Llama3.Q8_0.gguf new file mode 100644 index 0000000..450a8c8 --- /dev/null +++ b/Halu-OAS-8B-Llama3.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a049e1d6e24068497d43410fe7e11627ca0e1650416f2bf4ff3b51050ceb71bc +size 8540771072 diff --git a/Halu-OAS-8B-Llama3.f16.gguf b/Halu-OAS-8B-Llama3.f16.gguf new file mode 100644 index 0000000..052c4c9 --- /dev/null +++ b/Halu-OAS-8B-Llama3.f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ffb957cdbf70ffe063aa998a42de6be4101d2e1724640d3fb5be89358c09c9c3 +size 16068891392 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ab744fe --- /dev/null +++ b/README.md @@ -0,0 +1,69 @@ +--- +base_model: Hastagaras/Halu-OAS-8B-Llama3 +language: +- en +library_name: transformers +license: llama3 +quantized_by: mradermacher +tags: +- not-for-all-audiences +--- +## About + + + + + + +static quants of https://huggingface.co/Hastagaras/Halu-OAS-8B-Llama3 + + +weighted/imatrix quants are available at https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-i1-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q2_K.gguf) | Q2_K | 3.3 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.IQ3_XS.gguf) | IQ3_XS | 3.6 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q3_K_S.gguf) | Q3_K_S | 3.8 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.IQ3_S.gguf) | IQ3_S | 3.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.IQ3_M.gguf) | IQ3_M | 3.9 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q3_K_M.gguf) | Q3_K_M | 4.1 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q3_K_L.gguf) | Q3_K_L | 4.4 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.IQ4_XS.gguf) | IQ4_XS | 4.6 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q4_K_S.gguf) | Q4_K_S | 4.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q4_K_M.gguf) | Q4_K_M | 5.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q5_K_S.gguf) | Q5_K_S | 5.7 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q5_K_M.gguf) | Q5_K_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q6_K.gguf) | Q6_K | 6.7 | very good quality | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.Q8_0.gguf) | Q8_0 | 8.6 | fast, best quality | +| [GGUF](https://huggingface.co/mradermacher/Halu-OAS-8B-Llama3-GGUF/resolve/main/Halu-OAS-8B-Llama3.f16.gguf) | f16 | 16.2 | 16 bpw, overkill | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. + +