commit 3f5f696ab53bbafb655ff79a5c621b673d5ec42a Author: ModelHub XC Date: Tue Apr 21 22:01:33 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..0aa3451 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Ai-PL_Gemma_Legal-Areas.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ1_M.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ1_M.gguf new file mode 100644 index 0000000..012bc62 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a58554912589fa3340914d4f84ef5ec75934951de7f31b3ddef896197841a377 +size 813845152 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ1_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ1_S.gguf new file mode 100644 index 0000000..b7400ea --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a905b4f28ae02bf8bbcd2a572c3e604d769ca95754e4b05edcbaa530cd00c311 +size 770960032 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ2_M.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_M.gguf new file mode 100644 index 0000000..196a4f5 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7c5509a7536b4013754bf6619848bebfd8a9069fa76cbfb1faaf311766beb4d +size 1019472544 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ2_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_S.gguf new file mode 100644 index 0000000..f909761 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:edc4bc3fec88ff446435f86ea404fc2faa8bdcc3f543f04de04c77198bbbf68e +size 962292384 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XS.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XS.gguf new file mode 100644 index 0000000..448114e --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:adbd97b396cacce6e69edfc1456d999303b8f3d99cfdfdff26025f306eb6a258 +size 944859808 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XXS.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..f9f2792 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc9d9712d92d7efc1ce5faf8aaed55bdb67670bbebe53166bba6fac0ec0c6a8e +size 885320352 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ3_M.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_M.gguf new file mode 100644 index 0000000..9290693 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8aaded34fc2877b1f5d7fa6d3456ff00a4690595a60c88cd0d811e735c50805f +size 1308175008 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ3_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_S.gguf new file mode 100644 index 0000000..e792095 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac2176120e4b6b037d0eb4f11604589d801f47af81480d870e8b0a889907eb45 +size 1289235104 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XS.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XS.gguf new file mode 100644 index 0000000..78fe993 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5d3b296d4ba9c39d2ef97db12d408e4a4045faab238afb638b1c4aa62c407a0 +size 1244359328 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XXS.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..712752c --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:00021457a8d8fce6b3df9fd9f99af01fecc638314e3240e13bc2eb9eb09dc4b8 +size 1125378720 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ4_NL.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ4_NL.gguf new file mode 100644 index 0000000..6127e9c --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b2e936db41b4f605889b9f4b2fea97b87cf20936bff7b53e613cb4863e4d2eb +size 1552370336 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-IQ4_XS.gguf b/Ai-PL_Gemma_Legal-Areas.i1-IQ4_XS.gguf new file mode 100644 index 0000000..21ab6bd --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ceda80f42e664438054af4201797f3310216a28ceca3cca62723f50204129c6b +size 1490733728 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q2_K.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q2_K.gguf new file mode 100644 index 0000000..108cdb9 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0fbd1981e9a7856c6a61f8dfedcb7e905fb83a2aaa473428b89a323a1f4b444 +size 1157925536 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q2_K_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q2_K_S.gguf new file mode 100644 index 0000000..abf9f8c --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:343e480664e3e65863ca10b93ebdaa1a87511857f9dfeb126fa8addc63484c8b +size 1104644768 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_L.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_L.gguf new file mode 100644 index 0000000..6715a83 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1c7f6d89ea24c7b59c1a64d8819da1bcfe8e9ee530177f0b38a74bb791af9b3 +size 1465592480 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_M.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_M.gguf new file mode 100644 index 0000000..4422715 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2668cbadf10d7033a2f96fbb52dda9ab10bc129940479f5e74912013c79a3b19 +size 1383803552 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_S.gguf new file mode 100644 index 0000000..08290df --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6beba41e316ba7d40525e1e2e8936e04f454ef3ad4d03f4c3b5a5ad44cf28b72 +size 1287981728 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q4_0.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q4_0.gguf new file mode 100644 index 0000000..ff3282a --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88a771045af745291bb29f41743d5ce22496b381a5ebd84ad2c0854a96b58999 +size 1555384992 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q4_1.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q4_1.gguf new file mode 100644 index 0000000..2dcd0ea --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21836f9a5bf2223c7893791fe66b5151e8c4b37536b5bcba558bf946202bfe8a +size 1675053728 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_M.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_M.gguf new file mode 100644 index 0000000..4728ea2 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d09b63851375a34b19e2e34027434bdda3b901053456b8b5d1d01ad024aa7d97 +size 1630263968 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_S.gguf new file mode 100644 index 0000000..9237be0 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6484f5c37112b249399513f8d79fd7ed673c366893454f24d600651b8df492eb +size 1559841440 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_M.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_M.gguf new file mode 100644 index 0000000..38d2666 --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4049ce79e2f186cde4ac54807703e46e1955f47516e0c5af18bcbca2f90538dd +size 1839651488 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_S.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_S.gguf new file mode 100644 index 0000000..f7cb07e --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:790913b60130f710290131c49a914edfab0690fcd4e52b4f407093d2469024c4 +size 1798916768 diff --git a/Ai-PL_Gemma_Legal-Areas.i1-Q6_K.gguf b/Ai-PL_Gemma_Legal-Areas.i1-Q6_K.gguf new file mode 100644 index 0000000..93190cc --- /dev/null +++ b/Ai-PL_Gemma_Legal-Areas.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d37bc94abd97f92b74e03adf2bb7023c5b61abc45619b1d4ec6d5f50da3c53cb +size 2062125728 diff --git a/README.md b/README.md new file mode 100644 index 0000000..b90939b --- /dev/null +++ b/README.md @@ -0,0 +1,76 @@ +--- +base_model: Mohit-3430/Ai-PL_Gemma_Legal-Areas +language: +- en +library_name: transformers +license: mit +quantized_by: mradermacher +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/Mohit-3430/Ai-PL_Gemma_Legal-Areas + + +static quants are available at https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ1_S.gguf) | i1-IQ1_S | 0.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ1_M.gguf) | i1-IQ1_M | 0.9 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ2_S.gguf) | i1-IQ2_S | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ2_M.gguf) | i1-IQ2_M | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.2 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.2 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q2_K.gguf) | i1-Q2_K | 1.3 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.3 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.4 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ3_S.gguf) | i1-IQ3_S | 1.4 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ3_M.gguf) | i1-IQ3_M | 1.4 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.5 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.6 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-IQ4_NL.gguf) | i1-IQ4_NL | 1.7 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q4_0.gguf) | i1-Q4_0 | 1.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.7 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q4_K_M.gguf) | i1-Q4_K_M | 1.7 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q4_1.gguf) | i1-Q4_1 | 1.8 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.9 | | +| [GGUF](https://huggingface.co/mradermacher/Ai-PL_Gemma_Legal-Areas-i1-GGUF/resolve/main/Ai-PL_Gemma_Legal-Areas.i1-Q6_K.gguf) | i1-Q6_K | 2.2 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..beb333a --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13a0111b8453f6cab6cad5a7c19a622b7dce70ee713c99fc6e96cef770c86e66 +size 2068543