初始化项目，由ModelHub XC社区提供模型

Model: second-state/Falcon3-10B-Instruct-GGUF
Source: Original Platform

.gitattributes

@@ -0,0 +1,48 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-10B-Instruct-f16.gguf filter=lfs diff=lfs merge=lfs -text

Falcon3-10B-Instruct-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4067977ccbfb6c0def8786a693970722d370ed2971d7bd44deb6be4ed80418a0
+size 3924047264

Falcon3-10B-Instruct-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:391c0b1538f430bd434c690642e464ca8f8173790899defa078d0464736639e8
+size 5450806688

Falcon3-10B-Instruct-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b28fc6a168d8032c77adb31d90dbd49aa1215a1e65c7bf73fc2961414398bd9
+size 5052478880

Falcon3-10B-Instruct-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b191ee8bcd6b8cabea7c7d38d8e2828f3e184db2d878a9527a4ee706989d116
+size 4591138208

Falcon3-10B-Instruct-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e23b7961ee78aba31aaf61ef8ce8cafa86e9c939667366d3dd8232a4edf43f3f
+size 5906347424

Falcon3-10B-Instruct-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0710b0eee168b52279db4507ced89cd2ad7400605bc94781b506c0df0149485a
+size 6287521184

Falcon3-10B-Instruct-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7677005715bba79fcb97e6a5752562dc43bd13319e0047a8b7a11a87666af1d9
+size 5952157088

Falcon3-10B-Instruct-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8532cb48319b0c0618068010d4c858035cd2ca53529e84547f7d960a0bcf1cf4
+size 7144191392

Falcon3-10B-Instruct-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21aa4e0ffd723e44050fb606fb06c6f751b61272d0a3eb230b7d0b0c0096e7ec
+size 7340553632

Falcon3-10B-Instruct-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdfd0a81f84c5eb0c2cb2ac110dffb8e003572dc62c88271b606f3f9e362bf77
+size 7144191392

Falcon3-10B-Instruct-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac26bf63c98a398918333fa594b21d263f745c61575d088c630a1893068ac351
+size 8459400608

Falcon3-10B-Instruct-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebb06e4b7b348a7e5f3d213af89f8b46326b98991395cb1c631154c0dfedb05c
+size 10955240864

Falcon3-10B-Instruct-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:68f5dfc8589711205d954f2d281324c793fb6c8bd4ecf37711b89be1cdc4377d
+size 20616557984

README.md

@@ -0,0 +1,90 @@
+---
+base_model: tiiuae/Falcon3-10B-Instruct
+license: other
+license_name: falcon-llm-license
+license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
+model_creator: tiiuae
+model_name: Falcon3-10B-Instruct
+quantized_by: Second State Inc.
+language:
+- en
+- fr
+- es
+- pt
+library_name: transformers
+tags:
+- falcon3
+---
+
+<!-- header start -->
+<!-- 200823 -->
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
+<!-- header end -->
+
+# Falcon3-10B-Instruct-GGUF
+
+## Original Model
+
+[tiiuae/Falcon3-10B-Instruct](https://huggingface.co/tiiuae/Falcon3-10B-Instruct)
+
+## Run with LlamaEdge
+
+- LlamaEdge version: [v0.16.0](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.16.0) and above
+
+- Prompt template
+
+  - Prompt type: `falcon3`
+
+  - Prompt string
+
+    ```text
+    <|system|>
+    You are a helpful friendly assistant Falcon3 from TII, try to follow instructions as much as possible.
+    <|user|>
+    {user_message}
+    <|assistant|>
+    ```
+
+- Context size: `32000`
+
+- Run as LlamaEdge service
+
+  ```bash
+  wasmedge --dir .:. --nn-preload default:GGML:AUTO:Falcon3-10B-Instruct-Q5_K_M.gguf \
+    llama-api-server.wasm \
+    --model-name Falcon3-10B-Instruct \
+    --prompt-template falcon3 \
+    --ctx-size 32000
+  ```
+
+- Run as LlamaEdge command app
+
+  ```bash
+  wasmedge --dir .:. --nn-preload default:GGML:AUTO:Falcon3-10B-Instruct-Q5_K_M.gguf \
+    llama-chat.wasm \
+    --prompt-template falcon3 \
+    --ctx-size 32000
+  ```
+
+## Quantized GGUF Models
+
+| Name | Quant method | Bits | Size | Use case |
+| ---- | ---- | ---- | ---- | ----- |
+| [Falcon3-10B-Instruct-Q2_K.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q2_K.gguf)     | Q2_K   | 2 | 3.92 GB| smallest, significant quality loss - not recommended for most purposes |
+| [Falcon3-10B-Instruct-Q3_K_L.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q3_K_L.gguf) | Q3_K_L | 3 | 5.45 GB| small, substantial quality loss |
+| [Falcon3-10B-Instruct-Q3_K_M.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3 | 5.05 GB| very small, high quality loss |
+| [Falcon3-10B-Instruct-Q3_K_S.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3 | 4.59 GB| very small, high quality loss |
+| [Falcon3-10B-Instruct-Q4_0.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q4_0.gguf)     | Q4_0   | 4 | 5.91 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
+| [Falcon3-10B-Instruct-Q4_K_M.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 6.29 GB| medium, balanced quality - recommended |
+| [Falcon3-10B-Instruct-Q4_K_S.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4 | 5.95 GB| small, greater quality loss |
+| [Falcon3-10B-Instruct-Q5_0.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q5_0.gguf)     | Q5_0   | 5 | 7.14 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
+| [Falcon3-10B-Instruct-Q5_K_M.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5 | 7.34 GB| large, very low quality loss - recommended |
+| [Falcon3-10B-Instruct-Q5_K_S.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5 | 7.14 GB| large, low quality loss - recommended |
+| [Falcon3-10B-Instruct-Q6_K.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q6_K.gguf)     | Q6_K   | 6 | 8.46 GB| very large, extremely low quality loss |
+| [Falcon3-10B-Instruct-Q8_0.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-Q8_0.gguf)     | Q8_0   | 8 | 11.0 GB| very large, extremely low quality loss - not recommended |
+| [Falcon3-10B-Instruct-f16.gguf](https://huggingface.co/second-state/Falcon3-10B-Instruct-GGUF/blob/main/Falcon3-10B-Instruct-f16.gguf)       | f16    | 16 | 20.6 GB| |
+
+*Quantized with llama.cpp b4381*

config.json

@@ -0,0 +1,28 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 11,
+  "eos_token_id": 11,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "intermediate_size": 23040,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 4,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000042,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.46.1",
+  "use_cache": true,
+  "vocab_size": 131072
+}