commit 0f0b5e1be2dd51c76f892137f9c353d02c89e7db Author: ModelHub XC Date: Mon May 18 06:56:24 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/KernelLLM-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..a14c65c --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +KernelLLM.imatrix.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +KernelLLM.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/KernelLLM.i1-IQ1_M.gguf b/KernelLLM.i1-IQ1_M.gguf new file mode 100644 index 0000000..36ce194 --- /dev/null +++ b/KernelLLM.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c0f479b05bbd1a2fddffe00690bb514022f8e0302de462c8e22c9cedacfacfd +size 2161973344 diff --git a/KernelLLM.i1-IQ1_S.gguf b/KernelLLM.i1-IQ1_S.gguf new file mode 100644 index 0000000..f0e4684 --- /dev/null +++ b/KernelLLM.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dfc6bfbb8119512e3cdfefde6bcc463e890fadec994f1bd759ea594b96b2e7e9 +size 2019629152 diff --git a/KernelLLM.i1-IQ2_M.gguf b/KernelLLM.i1-IQ2_M.gguf new file mode 100644 index 0000000..fc44214 --- /dev/null +++ b/KernelLLM.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c376934057ddf6439114b58b2ecf1e2d55aedf3bf33aea3d046b49f067fc9bba +size 2948282464 diff --git a/KernelLLM.i1-IQ2_S.gguf b/KernelLLM.i1-IQ2_S.gguf new file mode 100644 index 0000000..487a7b8 --- /dev/null +++ b/KernelLLM.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a7550e90378be279997b671cded38c21a6997abd797d018ecdbad8ad06ee44ce +size 2758490208 diff --git a/KernelLLM.i1-IQ2_XS.gguf b/KernelLLM.i1-IQ2_XS.gguf new file mode 100644 index 0000000..3614424 --- /dev/null +++ b/KernelLLM.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad9b54305da39610151a5ee51946ff9390026b842d60da0651631e26f1f0d813 +size 2605783136 diff --git a/KernelLLM.i1-IQ2_XXS.gguf b/KernelLLM.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..8cbac0f --- /dev/null +++ b/KernelLLM.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:030ac125e16c32a6f23e479a7d220dacb663f94184f7cfd92aed6fc6c2b919f7 +size 2399213664 diff --git a/KernelLLM.i1-IQ3_M.gguf b/KernelLLM.i1-IQ3_M.gguf new file mode 100644 index 0000000..6548c40 --- /dev/null +++ b/KernelLLM.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:be946e8e78227ab39f21f1ba0701345d6446659de64b8bad23a925b37bffaef3 +size 3784824928 diff --git a/KernelLLM.i1-IQ3_S.gguf b/KernelLLM.i1-IQ3_S.gguf new file mode 100644 index 0000000..7d8a358 --- /dev/null +++ b/KernelLLM.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:17e6d3f96afad9c09202895a86a9ee7d23a1d1e282c531ab9481e138c24e9f64 +size 3682326624 diff --git a/KernelLLM.i1-IQ3_XS.gguf b/KernelLLM.i1-IQ3_XS.gguf new file mode 100644 index 0000000..9742875 --- /dev/null +++ b/KernelLLM.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a4c7b952f1b90595a1818234386f190f012b6f5d4a26d9e76fbee2a0f804974 +size 3518748768 diff --git a/KernelLLM.i1-IQ3_XXS.gguf b/KernelLLM.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..9f06daf --- /dev/null +++ b/KernelLLM.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6801c641caa0bd58ec52fe5bb1ac5030a832fc8d4757cb86316f4888cf350abd +size 3274913888 diff --git a/KernelLLM.i1-IQ4_NL.gguf b/KernelLLM.i1-IQ4_NL.gguf new file mode 100644 index 0000000..5c4e522 --- /dev/null +++ b/KernelLLM.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:614d249e86d5fd06fd2bf04f414bc82a5c4976df678d867015c4640a2f664187 +size 4677990496 diff --git a/KernelLLM.i1-IQ4_XS.gguf b/KernelLLM.i1-IQ4_XS.gguf new file mode 100644 index 0000000..3e14ece --- /dev/null +++ b/KernelLLM.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f2d616673d6e1156cc62e76ad7d470f4c6c84ad2a4f0effc982f60e0278dcb85 +size 4447664224 diff --git a/KernelLLM.i1-Q2_K.gguf b/KernelLLM.i1-Q2_K.gguf new file mode 100644 index 0000000..e0a0002 --- /dev/null +++ b/KernelLLM.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3bdf29c26fd45aa796c7e73230e0e7ae367310996eb9cf4f084ff5bef3bf3971 +size 3179133024 diff --git a/KernelLLM.i1-Q2_K_S.gguf b/KernelLLM.i1-Q2_K_S.gguf new file mode 100644 index 0000000..c753e81 --- /dev/null +++ b/KernelLLM.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0804ca59978d5812dcce06325ff52b1f49b2b020c108c39d8f31ef3848c20973 +size 2988816480 diff --git a/KernelLLM.i1-Q3_K_L.gguf b/KernelLLM.i1-Q3_K_L.gguf new file mode 100644 index 0000000..71642bc --- /dev/null +++ b/KernelLLM.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c8200324b7abfdf4b12b6fb3634946fca5ff46f80e10d314f29de1782f518f0 +size 4321957984 diff --git a/KernelLLM.i1-Q3_K_M.gguf b/KernelLLM.i1-Q3_K_M.gguf new file mode 100644 index 0000000..7e4df71 --- /dev/null +++ b/KernelLLM.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:23db01ecbd727791e4a8a5fe91410ca30a23c29b9db3b045d9329c1a3bb94443 +size 4018919520 diff --git a/KernelLLM.i1-Q3_K_S.gguf b/KernelLLM.i1-Q3_K_S.gguf new file mode 100644 index 0000000..a16864c --- /dev/null +++ b/KernelLLM.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:99568c45c8210ea0b924a2f9ec9ab15a3c524fcb0bc5443902e49d799152e707 +size 3664500832 diff --git a/KernelLLM.i1-Q4_0.gguf b/KernelLLM.i1-Q4_0.gguf new file mode 100644 index 0000000..ade8a54 --- /dev/null +++ b/KernelLLM.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad56bf30d096d9826823f1a4ca2355ba3a6ffdf50de3aff5366223adfe879cc9 +size 4675893344 diff --git a/KernelLLM.i1-Q4_1.gguf b/KernelLLM.i1-Q4_1.gguf new file mode 100644 index 0000000..3e5ef84 --- /dev/null +++ b/KernelLLM.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:069f8cfb1608be75026b241aa8d81ab448058f934201ecf5d6b55861934fcbc7 +size 5130254432 diff --git a/KernelLLM.i1-Q4_K_M.gguf b/KernelLLM.i1-Q4_K_M.gguf new file mode 100644 index 0000000..af22b0d --- /dev/null +++ b/KernelLLM.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eadc82e7a9f77156cb7c91737dcaec4ee0c5b807869c3451d5e532fdd0ad143f +size 4920735840 diff --git a/KernelLLM.i1-Q4_K_S.gguf b/KernelLLM.i1-Q4_K_S.gguf new file mode 100644 index 0000000..3f629ae --- /dev/null +++ b/KernelLLM.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:35ac3acd42a0fedf97585f2ff44ede7d156e1468c19ef31f87ef927778220aff +size 4692670560 diff --git a/KernelLLM.i1-Q5_K_M.gguf b/KernelLLM.i1-Q5_K_M.gguf new file mode 100644 index 0000000..c3d7c31 --- /dev/null +++ b/KernelLLM.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:62d48ba3f60386b0857803998536cc1051c9540bcf4bee26f75be1a80ce8e7ba +size 5732989024 diff --git a/KernelLLM.i1-Q5_K_S.gguf b/KernelLLM.i1-Q5_K_S.gguf new file mode 100644 index 0000000..8c79c1d --- /dev/null +++ b/KernelLLM.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0409a3439ded937e0417aa3371e8ae5f7149f0ac516846bbaa2bd64478d35dd4 +size 5599295584 diff --git a/KernelLLM.i1-Q6_K.gguf b/KernelLLM.i1-Q6_K.gguf new file mode 100644 index 0000000..5d4f3ce --- /dev/null +++ b/KernelLLM.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d455d6a3b8322695f0f40ab008fb69f638fbbdff6f2b23f9b61c443e5c1b82ff +size 6596008032 diff --git a/KernelLLM.imatrix.gguf b/KernelLLM.imatrix.gguf new file mode 100644 index 0000000..2cff089 --- /dev/null +++ b/KernelLLM.imatrix.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:040093e7436177a714763a1d8113b6d2296b824cb02f5443db3fb82af91f201f +size 5015200 diff --git a/README.md b/README.md new file mode 100644 index 0000000..e82ab8d --- /dev/null +++ b/README.md @@ -0,0 +1,88 @@ +--- +base_model: facebook/KernelLLM +datasets: +- ScalingIntelligence/KernelBench +- GPUMODE/KernelBook +language: +- en +library_name: transformers +license: other +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + + + + +weighted/imatrix quants of https://huggingface.co/facebook/KernelLLM + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#KernelLLM-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/KernelLLM-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.8 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q4_1.gguf) | i1-Q4_1 | 5.2 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | | +| [GGUF](https://huggingface.co/mradermacher/KernelLLM-i1-GGUF/resolve/main/KernelLLM.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + +