dahara1/shisa-v2.1-qwen3-8b-UD-japanese-imatrix
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: dahara1/shisa-v2.1-qwen3-8b-UD-japanese-imatrix
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-13 10:04:16 +08:00
commit e3149a6802
28 changed files with 273 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
.gitattributes vendored Normal file
View File

@@ -0,0 +1,59 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-Q2_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-Q5_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-Q6_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
shisa-v2.1-qwen3-8B-UD-Q8_K_XL.gguf filter=lfs diff=lfs merge=lfs -text

142
README.md Normal file
View File

@@ -0,0 +1,142 @@
---
license: apache-2.0
language:
- ja
- en
base_model:
- shisa-ai/shisa-v2.1-qwen3-8b
---
これは[shisa-v2.1-qwen3-8b](https://huggingface.co/shisa-ai/shisa-v2.1-qwen3-8b)のGGUF量子化版です。
This is a GGUF quantized version of [shisa-v2.1-qwen3-8b](https://huggingface.co/shisa-ai/shisa-v2.1-qwen3-8b).
## 特徴/Features
一言で言えば沢山の細かい改善をして出来上がった強力な量子化モデルです。
In short, it's a powerful quantized model with many small improvements.
このggufの特徴
- コミュニティが過去に発見したQwen3の設定に関するパッチを適用して誤作動割合を減らしています
- UnslothのDynamic 2.0 GGUF quantization手法を踏襲し、高い圧縮率を維持しつつ性能劣化を抑止しています
- imatrix作成時に日本語が大目のデータを使用し、日本語性能の劣化を抑止しています
- max_lengthは40Kに制限。長過ぎると短いプロンプトで性能が落ちる現象を防止しています
Features of this gguf
- We've applied a patch to reduce the rate of malfunctions related to Qwen3 settings that were previously discovered by the community.
- It follows Unsloth's Dynamic 2.0 GGUF quantization method, maintaining high compression ratios while minimizing performance degradation.
- When creating the imatrix, Japanese uses a larger amount of data to prevent degradation of Japanese performance.
- max_length is limited to 40K to prevent performance degradation with short prompts if it is too long.
## 動かし方 / How to Run
###
[llama.cpp](https://github.com/ggml-org/llama.cpp/releases)からお使いのハードウェア用のパッケージをダウンロードして設定します。
[Ollama](https://github.com/ollama/ollama)、[LM Studio](https://github.com/lmstudio-ai/lms)などのggufファイルに対応しているツールなら同様に動かす事ができます。
Download the package for your hardware from [llama.cpp](https://github.com/ggml-org/llama.cpp/releases) and set it up.
Tools that support gguf files, such as [Ollama](https://github.com/ollama/ollama) and [LM Studio](https://github.com/lmstudio-ai/lms), can also be used.
Linuxでのコマンドの実行例です
Here is an example of running the command on Linux:
```
./llama-cli -hf dahara1/shisa-v2.1-qwen3-8b-UD-japanese-imatrix:shisa-v2.1-qwen3-8B-UD-Q4_K_XL.gguf --ctx-size 8192 --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.01 --repeat-penalty 1.05
```
推奨モデルはshisa-v2.1-qwen3-8B-UD-Q4_K_XLですが、お使いのパソコンのメモリ量に合わせて、適切な大きさのモデルを選んでください
The recommended model is shisa-v2.1-qwen3-8B-UD-Q4_K_XL, but please choose a model of the appropriate size based on the amount of memory in your computer.
![cli interface](cli-interface.png)
## サンプルスクリプト / sample script
クライアント/サーバー型式でスクリプトでアクセスしたい場合は以下を参考にしてください
If you want to access it via script in a client/server format, please refer to the following:
### llama-server Commandの例
```
./llama-server -hf dahara1/shisa-v2.1-qwen3-8b-UD-japanese-imatrix:shisa-v2.1-qwen3-8B-UD-Q4_K_XL.gguf --host 0.0.0.0 --port 8080 --ctx-size 8192 --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.01 --repeat-penalty 1.05
```
ブラウザで、モデルを実行しているサーバーのローカルアドレス、ポートを指定して開いて下さい。例(http://127.0.0.1:8080/)
In your browser, open the local address and port of the server running the model. For example, http://127.0.0.1:8080/
![web interface](web-interface.png)
### client script
```
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8080/v1",
api_key="dummy" #
)
response = client.chat.completions.create(
model="shisa-v2.1-qwen3-8b-UD-japanese-imatrix",
messages=[
{"role": "system", "content": "あなたは親切でなアシスタントです。ファンタジー設定でエルフの王女としてロールプレイをしてください"},
{"role": "user", "content": "こんにちは！"}
],
stream=True
)
for chunk in response:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="", flush=True)
```
出力例
```
こんにちは、旅人よ。私の名前はセレナ。この森の守り神であるエルフ一族の王女だ。どういったご用件かな？ 何か私にできることがあれば、喜んでお手伝いしよう。この森は危険も多いから、もし迷子になったり怪我をしていたら、遠慮なく言ってほしい。優しくしてあげるから安心してほしいな。
```
## ベンチマーク結果/benchmark result
shisa.aiのオリジナルモデルと、本リポジトリのモデルとmradermacher(量子化技術で有名な人)が作成した量子化モデルの比較です
This is a comparison of the original model from shisa.ai, the model from this repository, and the quantized model created by mradermacher (famous for his quantization techniques).
| カテゴリ | 項目 (Metric) | **オリジナル (Base)**<br><small>shisa-ai</small> | **UD版 (Q4_K_XL)**<br><small>dahara1</small> | **i1版 (Q4_1)**<br><small>mradermacher</small> | 勝者 (Q間比較) |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **基本情報** | ファイルサイズ | **16.38 GB** | **5.14 GB** <br><small>(31%に圧縮)</small> | 5.25 GB | - |
| **基礎性能** | KL Divergence <br><small>(0に近いほど再現度が高い)</small> | 0.00 (基準) | **0.034** | 0.047 | **UD版** 🏆 |
| | Same Top P <br><small>(選ぶ単語の一致率)</small> | 100% | **90.60%** | 89.00% | **UD版** 🏆 |
| | Perplexity Ratio <br><small>(迷いのなさの劣化倍率)</small> | 1.00 | **1.014倍** | 1.021倍 | **UD版** 🏆 |
| **日本語指示** | [M-IFEval (JA) (Instruction Following)](https://github.com/shisa-ai/M-IFEval) <br><small>Prompt Level (Loose)</small> | 0.471 | **0.476** | 0.459 | **UD版** 🏆 |
| **コーディング** | HumanEval+ <br><small>(pass@1)</small> | 0.805 | **0.793** | 0.774 | **UD版** 🏆 |
| **総合ベンチ**<br><small>(LiveBench)</small> | **LiveBench Average** | 45.7 | **40.3** | 38.7 | **UD版** 🏆 |
| | - Reasoning (推論) | - | **33.9** | 33.1 | **UD版** 🏆 |
| | - Data Analysis (分析) | - | **37.0** | 33.5 | **UD版** 🏆 |
| | - Language (言語) | - | **33.5** | 28.5 | **UD版** 🏆 |
| | - Math (数学) | - | **35.6** | 33.6 | **UD版** 🏆 |
| | - Instruction Following | - | 61.4 | **64.8** | i1版 👑 |
## Qwen3推奨パラメーター設定 / Qwen3 recommended parameter settings
Qwen3はGreedy decoding（温度0などの決定論的な生成）を使用すると、繰り返し生成などの不具合が起きやすいため、必ずサンプリング（Temperature > 0）を使用することが強く推奨されています。
Qwen3 is prone to errors such as repeated generation when using greedy decoding (deterministic generation of temperatures such as 0), so it is strongly recommended to always use sampling (Temperature > 0).
### Unslothによる推奨パラメーター
- Temperature 0.7
- Top_P 0.8
- Top_K 20
- Min_P 0.00 (オプションですが、0.01 でも問題なく動作します。llama.cpp のデフォルトは 0.1 です)
- Repetition Penalty 1.05
### Recommended Parameters by Unsloth
- Temperature 0.7
- Top_P 0.8
- Top_K 20
- Min_P 0.00 (optional, but 0.01 works well, llama.cpp default is 0.1)
- Repetition Penalty 1.05
## 謝辞 / Acknowledgments
- [Qwen](https://huggingface.co/Qwen/Qwen3-8B)
- [Shisa](shisa-ai/shisa-v2.1-qwen3-8b)
- [Unsloth](https://huggingface.co/unsloth/Qwen3-8B-GGUF)
- [mradermacher](https://huggingface.co/mradermacher/shisa-v2.1-qwen3-8b-i1-GGUF)
- [llama.cpp](https://github.com/ggml-org/llama.cpp)
- Thank you to all AI researchers and practitioners

BIN
cli-interface.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

3
shisa-v2.1-qwen3-8B-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0fffeb6bacc4b167f3e73efb498c97d30756caa5406dd3a1d706d5d232a3863d
size 4793625152

3
shisa-v2.1-qwen3-8B-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb201a31a41512fcdcf2686b13ef78d7a0ea157c6c7f73f2caf5c0bcfb41e4b0
size 4581288512

3
shisa-v2.1-qwen3-8B-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bae8c413109562491db8b4b1d6c792e77d53850dcbcfbb411686232c8ea94256
size 3281734208

3
shisa-v2.1-qwen3-8B-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:afdb8b9b2e25dc767f08d934cd68fd2f18dff918e141bfb064cd6e8a7c83bea8
size 3427592768

3
shisa-v2.1-qwen3-8B-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ccab3f80fe04a2107b0b32a25dc768778736c17265dbb6cfeb6242a73818e272
size 4124162624

3
shisa-v2.1-qwen3-8B-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cc1ca4213289dbeca1fef9201adcc2c567d54b6f66881e699352b2429da6aced
size 3769612864

3
shisa-v2.1-qwen3-8B-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae487d9b1ee1505a2d4ce0fa9649ccd8fe5649cec8ecfa3890a27eb0d81f8f9e
size 5247756864

3
shisa-v2.1-qwen3-8B-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9086324f555177fdb3c30c2c2c036fffe9f544e0613d7c5e44671bb00a923c13
size 5027785280

3
shisa-v2.1-qwen3-8B-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:edfaafcbb70b6cc30b0a6ae578173157ddb9a66ade59ca443947896e19d56054
size 4802013760

3
shisa-v2.1-qwen3-8B-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7953c949173bef5359a4464efa0719aebe9c140566b52a44d4b2fe05b6944062
size 5851114048

3
shisa-v2.1-qwen3-8B-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:560e1f5d1e32dd9dffaf9f6d0ccfdb5a01a9efa75dd64c25b35b6ba62d3e13da
size 5720762944

3
shisa-v2.1-qwen3-8B-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:805c445436a590dea6878a087e141dfaa0ef6d7c7d3a8493bd07e6553a331123
size 6725900864

3
shisa-v2.1-qwen3-8B-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3cac53ee1f8dd29f1e0ff7f94039d0e02bf00ccec016d2f0f45edf762eae076e
size 8709519936

3
shisa-v2.1-qwen3-8B-UD-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9dfe8a69019733fa4c33f4d0e699065945ea950610f729232b8ff7efb0cf0ee2
size 2396490304

3
shisa-v2.1-qwen3-8B-UD-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b868e983361ae4044b875c5268f4ffd0636a7151f8850c4a9502bd34d82d6caf
size 2275379776

3
shisa-v2.1-qwen3-8B-UD-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:015dfe7c0befd5809b327a198b3c6d1734ea844c30d17816e98fe524c9d32cce
size 3110898240

3
shisa-v2.1-qwen3-8B-UD-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3b44a5c7182cad3d4e0085dfd0056bf975a80243e75e166a4657b6bb2fe2205
size 2605222464

3
shisa-v2.1-qwen3-8B-UD-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8125cd86b749c7270abf96e78fea409f8c90a9c8a80a695a12d26e4aeb14fafd
size 3410266688

3
shisa-v2.1-qwen3-8B-UD-Q2_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b8fd38343e8c0f63eccc8ebbbc94b54fd1fc66de1ff55c7c9d4fd11cc36e7c34
size 3501976128

3
shisa-v2.1-qwen3-8B-UD-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2f3c859b872473d4c8457da5b719109d643fbddb5a395c2fef9937eeb026b2a6
size 4307053120

3
shisa-v2.1-qwen3-8B-UD-Q4_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:39a33cada71c41234506c07e6e106760fe7e3f5310feb1064f6916f515eb12f6
size 5135723072

3
shisa-v2.1-qwen3-8B-UD-Q5_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f982813f89acf2bc3ca56375f3540686a0aa30dce4d94070f985b9442b711bbb
size 5878082112

3
shisa-v2.1-qwen3-8B-UD-Q6_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a98cf05906bd916bd9d9269588a5db86f0cd9ab3a949de6aa0cff1da57cff34f
size 7490550336

3
shisa-v2.1-qwen3-8B-UD-Q8_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb7e66b9950ce6192c642fbac86e88149cac49979560944325d227f9ac161f6e
size 10824038976

BIN
web-interface.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 19 KiB