second-state/SOLAR-10.7B-Instruct-v1.0-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: second-state/SOLAR-10.7B-Instruct-v1.0-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-29 10:11:12 +08:00
commit d604c9fc7f
17 changed files with 188 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
*.gguf filter=lfs diff=lfs merge=lfs -text

84
README.md Normal file
View File

@@ -0,0 +1,84 @@
---
base_model: upstage/SOLAR-10.7B-Instruct-v1.0
inference: false
library_name: transformers
license: cc-by-nc-4.0
model_creator: upstage
model_name: Solar 10.7B Instruct v1.0
model_type: solar
datasets:
- c-s-ale/alpaca-gpt4-data
- Open-Orca/OpenOrca
- Intel/orca_dpo_pairs
- allenai/ultrafeedback_binarized_cleaned
language:
- en
pipeline_tag: text-generation
quantized_by: Second State Inc.
---
<!-- header start -->
<!-- 200823 -->
<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
</div>
<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
<!-- header end -->
# SOLAR-10.7B-Instruct-v1.0-GGUF
## Original Model
[upstage/SOLAR-10.7B-Instruct-v1.0](https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0)
## Run with LlamaEdge
- LlamaEdge version: [v0.2.8](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.2.8) and above
- Prompt template
- Prompt type: `solar-instruct`
- Prompt string
```text
<s> ### User:
{user_message}
\### Assistant:
{assistant_message}</s>
```
Note that the `\` character is used to escape the `###` in the prompt string. Remove it in the practical use.
- Context size: `4096`
- Run as LlamaEdge service
```bash
wasmedge --dir .:. --nn-preload default:GGML:AUTO:SOLAR-10.7B-Instruct-v1.0-Q5_K_M.gguf llama-api-server.wasm -p solar-instruct
```
- Run as LlamaEdge command app
```bash
wasmedge --dir .:. --nn-preload default:GGML:AUTO:SOLAR-10.7B-Instruct-v1.0-Q5_K_M.gguf llama-chat.wasm -p solar-instruct
```
## Quantized GGUF Models
| Name | Quant method | Bits | Size | Use case |
| ---- | ---- | ---- | ---- | ----- |
| [SOLAR-10.7B-Instruct-v1.0-Q2_K.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q2_K.gguf) | Q2_K | 2 | 4.55 GB| smallest, significant quality loss - not recommended for most purposes |
| [SOLAR-10.7B-Instruct-v1.0-Q3_K_L.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q3_K_L.gguf) | Q3_K_L | 3 | 5.65 GB| small, substantial quality loss |
| [SOLAR-10.7B-Instruct-v1.0-Q3_K_M.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q3_K_M.gguf) | Q3_K_M | 3 | 5.19 GB| very small, high quality loss |
| [SOLAR-10.7B-Instruct-v1.0-Q3_K_S.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q3_K_S.gguf) | Q3_K_S | 3 | 4.66 GB| very small, high quality loss |
| [SOLAR-10.7B-Instruct-v1.0-Q4_0.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q4_0.gguf) | Q4_0 | 4 | 6.07 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
| [SOLAR-10.7B-Instruct-v1.0-Q4_K_M.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q4_K_M.gguf) | Q4_K_M | 4 | 6.46 GB| medium, balanced quality - recommended |
| [SOLAR-10.7B-Instruct-v1.0-Q4_K_S.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q4_K_S.gguf) | Q4_K_S | 4 | 6.1 GB| small, greater quality loss |
| [SOLAR-10.7B-Instruct-v1.0-Q5_0.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q5_0.gguf) | Q5_0 | 5 | 7.4 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
| [SOLAR-10.7B-Instruct-v1.0-Q5_K_M.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q5_K_M.gguf) | Q5_K_M | 5 | 7.6 GB| large, very low quality loss - recommended |
| [SOLAR-10.7B-Instruct-v1.0-Q5_K_S.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q5_K_S.gguf) | Q5_K_S | 5 | 7.4 GB| large, low quality loss - recommended |
| [SOLAR-10.7B-Instruct-v1.0-Q6_K.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q6_K.gguf) | Q6_K | 6 | 8.81 GB| very large, extremely low quality loss |
| [SOLAR-10.7B-Instruct-v1.0-Q8_0.gguf](https://huggingface.co/second-state/SOLAR-10.7B-Instruct-v1.0-GGUF/blob/main/SOLAR-10.7B-Instruct-v1.0-Q8_0.gguf) | Q8_0 | 8 | 11.4 GB| very large, extremely low quality loss - not recommended |

3
SOLAR-10.7B-Instruct-v1.0-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ecf16aecc958143aa311a743ad2610ee85adbb291be1b631d13179eb41ba6e0e
size 4549016160

3
SOLAR-10.7B-Instruct-v1.0-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9635f0e6a0ef66837ef0afc98b138521c7f72f3ea0746c4fc7bc4e808c553d37
size 5650750048

3
SOLAR-10.7B-Instruct-v1.0-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fad968520411760198100fb42fe3b6f0060ec3ac8a1aff94ca9f099619b68e9d
size 5188328032

3
SOLAR-10.7B-Instruct-v1.0-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4a6f35f3712913aa789abc63afc7eea9abd96a746e558849f0eb69b783f34f2c
size 4664564320

3
SOLAR-10.7B-Instruct-v1.0-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a3829f7cf39847e478b0d5ffb562d8776a5897ef2f3cfe586e52df71c44d2f40
size 6072384096

3
SOLAR-10.7B-Instruct-v1.0-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7839e428dfc2e7f6a9489b2206d607626bd71302ee01059577703aedfface34f
size 6461667936

3
SOLAR-10.7B-Instruct-v1.0-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aed183d3baaa6974d1988e6cdebe1c8ab8be52a2a4fe274b35139eede9633c76
size 6103841376

3
SOLAR-10.7B-Instruct-v1.0-Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c819a493ec2dbaa7222545b358c26cbc8998c03442236a430b1c863d80fa8083
size 7397390944

3
SOLAR-10.7B-Instruct-v1.0-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:715704d0c565664cf49dc6b4e0e087871724b7cb00ecf36a126df1d3de26b843
size 7597931104

3
SOLAR-10.7B-Instruct-v1.0-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:69af001a9c39a5cb64a9676111cb5b0444affd0122930e8fd53f32b5dd94c2d4
size 7397390944

3
SOLAR-10.7B-Instruct-v1.0-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0690fe4f91f47317b6c52e5c03d59b753ca03acefbd6d558eaf46751f5f14f52
size 8805210720

3
SOLAR-10.7B-Instruct-v1.0-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f8e3bb1b09ff245aaa11605bde7da976c1dd5847303a93c3993fa7882fe84391
size 11404155488

28
config.json Normal file
View File

@@ -0,0 +1,28 @@
{
"_name_or_path": "upstage/SOLAR-10.7B-Instruct-v1.0",
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"bos_token_id": 1,
"eos_token_id": 2,
"pad_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 4096,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 48,
"num_key_value_heads": 8,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"rope_theta": 10000.0,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.35.0",
"use_cache": true,
"vocab_size": 32000
}

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
solar-10.7b-instruct-v1.0.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4ade240f5dcc253272158f3659a56f5b1da8405510707476d23a7df943aa35f7
size 7597931104