初始化项目，由ModelHub XC社区提供模型

Model: second-state/Seed-OSS-36B-Instruct-GGUF Source: Original Platform
2026-06-22 02:20:16 +08:00
commit 86f8196b59
18 changed files with 248 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,50 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-f16-00001-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-f16-00002-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
+Seed-OSS-36B-Instruct-f16-00003-of-00003.gguf filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,120 @@
+---
+base_model: ByteDance-Seed/Seed-OSS-36B-Instruct
+model_creator: ByteDance-Seed
+model_name: Seed-OSS-36B-Instruct
+quantized_by: Second State Inc.
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+<!-- header start -->
+<!-- 200823 -->
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
+<!-- header end -->
+
+# Seed-OSS-36B-Instruct-GGUF
+
+## Original Model
+
+[ByteDance-Seed/Seed-OSS-36B-Instruct](https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct)
+
+## Run with LlamaEdge
+
+- LlamaEdge version: coming soon
+
+<!-- - LlamaEdge version: [v0.25.1](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.25.1) and above -->
+
+- Prompt template
+
+  - Prompt type:
+ 
+    - `seed-oss-think` for think mode
+    - `seed-oss-no-think` for no think mode
+
+  - Prompt string
+    - `Thinking` mode
+
+      ```text
+      <seed:bos>system
+      You are Doubao, a helpful AI assistant.
+      <seed:eos>
+
+      <seed:bos>user
+      {user_message_1}
+      <seed:eos>
+
+      <seed:bos>assistant
+      <seed:think>{thinking_content}</seed:think>
+      {assistant_message_1}
+      <seed:eos>
+
+      <seed:bos>user
+      {user_message_2}
+      <seed:eos>
+
+      <seed:bos>assistant
+      ```
+
+    - `No-thinking` mode
+
+      ```text
+      <seed:bos>system
+      You are Doubao, a helpful AI assistant.
+      <seed:eos>
+
+      <seed:bos>system
+      You are an intelligent assistant that can answer questions in one step without the need for reasoning and thinking, that is, your thinking budget is 0. Next, please skip       the thinking process and directly start answering the user's questions.
+      <seed:eos>
+
+      <seed:bos>user
+      {user_message_1}
+      <seed:eos>
+
+      <seed:bos>assistant
+      {assistant_message_1}
+      <seed:eos>
+
+      <seed:bos>user
+      {user_message_2}
+      <seed:eos>
+
+      <seed:bos>assistant
+      ```
+
+- Context size: `512000`
+
+- Run as LlamaEdge service
+
+  ```bash
+  wasmedge --dir .:. \
+    --nn-preload default:GGML:AUTO:Seed-OSS-36B-Instruct-Q5_K_M.gguf \
+    llama-api-server.wasm \
+    --prompt-template seed-oss-no-think \
+    --ctx-size 512000 \
+    --model-name seed-oss
+  ```
+
+## Quantized GGUF Models
+
+| Name | Quant method | Bits | Size | Use case |
+| ---- | ---- | ---- | ---- | ----- |
+| [Seed-OSS-36B-Instruct-Q2_K.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q2_K.gguf)     | Q2_K   | 2 | 13.6 GB| smallest, significant quality loss - not recommended for most purposes |
+| [Seed-OSS-36B-Instruct-Q3_K_L.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q3_K_L.gguf) | Q3_K_L | 3 | 19.1 GB| small, substantial quality loss |
+| [Seed-OSS-36B-Instruct-Q3_K_M.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3 | 17.6 GB| very small, high quality loss |
+| [Seed-OSS-36B-Instruct-Q3_K_S.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3 | 15.9 GB| very small, high quality loss |
+| [Seed-OSS-36B-Instruct-Q4_0.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q4_0.gguf)     | Q4_0   | 4 | 20.6 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
+| [Seed-OSS-36B-Instruct-Q4_K_M.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 21.8 GB| medium, balanced quality - recommended |
+| [Seed-OSS-36B-Instruct-Q4_K_S.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4 | 20.7 GB| small, greater quality loss |
+| [Seed-OSS-36B-Instruct-Q5_0.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q5_0.gguf)     | Q5_0   | 5 | 25.0 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
+| [Seed-OSS-36B-Instruct-Q5_K_M.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5 | 25.6 GB| large, very low quality loss - recommended |
+| [Seed-OSS-36B-Instruct-Q5_K_S.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5 | 25.0 GB| large, low quality loss - recommended |
+| [Seed-OSS-36B-Instruct-Q6_K.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q6_K.gguf)     | Q6_K   | 6 | 29.7 GB| very large, extremely low quality loss |
+| [Seed-OSS-36B-Instruct-Q8_0.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-Q8_0.gguf)     | Q8_0   | 8 | 38.4 GB| very large, extremely low quality loss - not recommended |
+| [Seed-OSS-36B-Instruct-f16-00001-of-00003.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-f16-00001-of-00003.gguf)       | f16   | 16 | 30.0 GB|  |
+| [Seed-OSS-36B-Instruct-f16-00002-of-00003.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-f16-00002-of-00003.gguf)       | f16   | 16 | 30.0 GB|  |
+| [Seed-OSS-36B-Instruct-f16-00003-of-00003.gguf](https://huggingface.co/second-state/Seed-OSS-36B-Instruct-GGUF/blob/main/Seed-OSS-36B-Instruct-f16-00003-of-00003.gguf)       | f16   | 16 | 12.4 GB|  |
+
+*Quantized with llama.cpp b6301.*
--- a/Seed-OSS-36B-Instruct-Q2_K.gguf
+++ b/Seed-OSS-36B-Instruct-Q2_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba2bf073afd4d158ab84c460398b090aa2e95bf37cf67cd1be0ad884a577bba7
+size 13604186816
--- a/Seed-OSS-36B-Instruct-Q3_K_L.gguf
+++ b/Seed-OSS-36B-Instruct-Q3_K_L.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b128b7b0c0264c5a4ee3611eea4f903ae9655bfd1b050f45c20d09cfaf762fe0
+size 19142695616
--- a/Seed-OSS-36B-Instruct-Q3_K_M.gguf
+++ b/Seed-OSS-36B-Instruct-Q3_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:488598f54c410c62dff67b01845a7fdfd5ce15b5d9a87a2934217bfae336d8ff
+size 17620949696
--- a/Seed-OSS-36B-Instruct-Q3_K_S.gguf
+++ b/Seed-OSS-36B-Instruct-Q3_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1248298473519cdd51ff8e82fe6927b958e9967170bd9c9faf438a1e04d2361c
+size 15855409856
--- a/Seed-OSS-36B-Instruct-Q4_0.gguf
+++ b/Seed-OSS-36B-Instruct-Q4_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a59580e69f8bba87db27186d7bcb904ee2cf537aae0fed0c8404f49baac7d3d7
+size 20551043776
--- a/Seed-OSS-36B-Instruct-Q4_K_M.gguf
+++ b/Seed-OSS-36B-Instruct-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc43e66ffc70e2d5a76eaaa4541b84178fb635c4acaaf3ccb782e61887b1432a
+size 21762149056
--- a/Seed-OSS-36B-Instruct-Q4_K_S.gguf
+++ b/Seed-OSS-36B-Instruct-Q4_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c752b89826282e27c460b2fba07c7f6d1025b7ef0f65db3fb53f5a7d7919cbf8
+size 20695222976
--- a/Seed-OSS-36B-Instruct-Q5_0.gguf
+++ b/Seed-OSS-36B-Instruct-Q5_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ac9ea0aa5aa95ad230ef0a04c7312a078fd9f3bd1a555f7c9befb79254b5890
+size 24970463936
--- a/Seed-OSS-36B-Instruct-Q5_K_M.gguf
+++ b/Seed-OSS-36B-Instruct-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec85d2c4bceb6167372720dd73694a353a335254bfb3e3ab7f16fff94a640d44
+size 25594366656
--- a/Seed-OSS-36B-Instruct-Q5_K_S.gguf
+++ b/Seed-OSS-36B-Instruct-Q5_K_S.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd2a23f999709c85157f0c8efa46b325b6abf7e49bc04ee35367664e39cec29e
+size 24970463936
--- a/Seed-OSS-36B-Instruct-Q6_K.gguf
+++ b/Seed-OSS-36B-Instruct-Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d6d66d812b2bf735ef34133d4abd73484f18e8d24a177fcd850cd9d71c537bf
+size 29666097856
--- a/Seed-OSS-36B-Instruct-Q8_0.gguf
+++ b/Seed-OSS-36B-Instruct-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed32089f5bee55dd8287b9a9a6bbe5b9a1b92237d374a7f21efc7a77322dddc0
+size 38421093056
--- a/Seed-OSS-36B-Instruct-f16-00001-of-00003.gguf
+++ b/Seed-OSS-36B-Instruct-f16-00001-of-00003.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a808953ceb73ff9ec3c8e90185d9a94a46de7d83a55a4691107ad02056cb53d6
+size 29961204576
--- a/Seed-OSS-36B-Instruct-f16-00002-of-00003.gguf
+++ b/Seed-OSS-36B-Instruct-f16-00002-of-00003.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:867e04eb9dc7949d6e4b64879de1589c16767efe6c840eb58ca448161bc884bd
+size 29960359072
--- a/Seed-OSS-36B-Instruct-f16-00003-of-00003.gguf
+++ b/Seed-OSS-36B-Instruct-f16-00003-of-00003.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecd018e6e742b5fe7d647343ba919363a301f5e335386110a35a34c2c243017b
+size 12389833728
--- a/config.json
+++ b/config.json
@@ -0,0 +1,33 @@
+{
+  "architectures": [
+    "SeedOssForCausalLM"
+  ],
+  "attention_bias": true,
+  "attention_dropout": 0.1,
+  "attention_out_bias": false,
+  "bos_token_id": 0,
+  "pad_token_id": 1,
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 27648,
+  "max_position_embeddings": 524288,
+  "mlp_bias": false,
+  "model_type": "seed_oss",
+  "num_attention_heads": 80,
+  "num_hidden_layers": 64,
+  "num_key_value_heads": 8,
+  "residual_dropout": 0.1,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": {
+    "rope_type": "default"
+  },
+  "rope_theta": 10000000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.55.0",
+  "use_cache": true,
+  "vocab_size": 155136
+}