初始化项目，由ModelHub XC社区提供模型

Model: second-state/Falcon3-7B-Instruct-GGUF
Source: Original Platform

.gitattributes

@@ -0,0 +1,48 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Falcon3-7B-Instruct-f16.gguf filter=lfs diff=lfs merge=lfs -text

Falcon3-7B-Instruct-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eb1d9cfbbbbf52fcbac1fd7a622827f35b59c45b5759f58ce30ef920a7955b5
+size 2886982848

Falcon3-7B-Instruct-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f8f7f347bd7e191f0081f8a52729cae4a78c4f8cf5e7b30ed1a9ff5ea40c97f
+size 3967982784

Falcon3-7B-Instruct-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef1735498d3b4b73c01417c3b1b2822ea6513ccdabee784ba093cf9551c32e28
+size 3685850304

Falcon3-7B-Instruct-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5a02d555eecc7fb80dcd80b4334aaa0ab13aebef9af6e27084657bcc7e1c475
+size 3366214848

Falcon3-7B-Instruct-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:573d62443f3b7f6204130387e0836528238bde348714f8f15a4e0992c0a3c9ff
+size 4302904512

Falcon3-7B-Instruct-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83a4ed64bd40e7d6a85b906bd8c50d43654c6b1bafe60e6051c07c2897cde1f2
+size 4569726144

Falcon3-7B-Instruct-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bf573b3cb7cbad8a92a55b4c9983ecdc32539609ea5e0b1cc9240a1a17a7944
+size 4331019456

Falcon3-7B-Instruct-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a46076284c99d70ce4cf4bfe4106654f8d6d9e013c6a821a683106eb8cd938b
+size 5184494784

Falcon3-7B-Instruct-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d228f8f23f0bef60e188c30f48f5ae37d83ade014c2583cdfe77d910c516e0c
+size 5321948352

Falcon3-7B-Instruct-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d1406dc2654ad8e90f5d0230e2cbf36bbbe3f71bd400131f6aca73582981f89
+size 5184494784

Falcon3-7B-Instruct-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bde13a9eb589ddee9773e2ebaa8616dd1a06a3f9bc603357f2092b840e2822b9
+size 6121184448

Falcon3-7B-Instruct-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:385ccaf41f35ea1556724ff0f720ad76d33b4fac302496a5384b22bac8ce48fb
+size 7926783168

Falcon3-7B-Instruct-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b046b87b4e25c49b8a4b926854b3cb28a4f7b7117c76ae58518a24d8d998bc0
+size 14916197568

README.md

@@ -0,0 +1,90 @@
+---
+base_model: tiiuae/Falcon3-7B-Instruct
+license: other
+license_name: falcon-llm-license
+license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
+model_creator: tiiuae
+model_name: Falcon3-7B-Instruct
+quantized_by: Second State Inc.
+language:
+- en
+- fr
+- es
+- pt
+library_name: transformers
+tags:
+- falcon3
+---
+
+<!-- header start -->
+<!-- 200823 -->
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
+<!-- header end -->
+
+# Falcon3-7B-Instruct-GGUF
+
+## Original Model
+
+[tiiuae/Falcon3-7B-Instruct](https://huggingface.co/tiiuae/Falcon3-7B-Instruct)
+
+## Run with LlamaEdge
+
+- LlamaEdge version: [v0.16.0](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.16.0) and above
+
+- Prompt template
+
+  - Prompt type: `falcon3`
+
+  - Prompt string
+
+    ```text
+    <|system|>
+    You are a helpful friendly assistant Falcon3 from TII, try to follow instructions as much as possible.
+    <|user|>
+    {user_message}
+    <|assistant|>
+    ```
+
+- Context size: `32000`
+
+- Run as LlamaEdge service
+
+  ```bash
+  wasmedge --dir .:. --nn-preload default:GGML:AUTO:Falcon3-7B-Instruct-Q5_K_M.gguf \
+    llama-api-server.wasm \
+    --model-name Falcon3-7B-Instruct \
+    --prompt-template falcon3 \
+    --ctx-size 32000
+  ```
+
+- Run as LlamaEdge command app
+
+  ```bash
+  wasmedge --dir .:. --nn-preload default:GGML:AUTO:Falcon3-7B-Instruct-Q5_K_M.gguf \
+    llama-chat.wasm \
+    --prompt-template falcon3 \
+    --ctx-size 32000
+  ```
+
+## Quantized GGUF Models
+
+| Name | Quant method | Bits | Size | Use case |
+| ---- | ---- | ---- | ---- | ----- |
+| [Falcon3-7B-Instruct-Q2_K.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q2_K.gguf)     | Q2_K   | 2 | 2.89 GB| smallest, significant quality loss - not recommended for most purposes |
+| [Falcon3-7B-Instruct-Q3_K_L.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q3_K_L.gguf) | Q3_K_L | 3 | 3.97 GB| small, substantial quality loss |
+| [Falcon3-7B-Instruct-Q3_K_M.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3 | 3.69 GB| very small, high quality loss |
+| [Falcon3-7B-Instruct-Q3_K_S.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3 | 3.37 GB| very small, high quality loss |
+| [Falcon3-7B-Instruct-Q4_0.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q4_0.gguf)     | Q4_0   | 4 | 4.30 GB| legacy; small, very high quality loss - prefer using Q3_K_M |
+| [Falcon3-7B-Instruct-Q4_K_M.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 4.57 GB| medium, balanced quality - recommended |
+| [Falcon3-7B-Instruct-Q4_K_S.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4 | 4.33 GB| small, greater quality loss |
+| [Falcon3-7B-Instruct-Q5_0.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q5_0.gguf)     | Q5_0   | 5 | 5.18 GB| legacy; medium, balanced quality - prefer using Q4_K_M |
+| [Falcon3-7B-Instruct-Q5_K_M.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5 | 5.32 GB| large, very low quality loss - recommended |
+| [Falcon3-7B-Instruct-Q5_K_S.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5 | 5.18 GB| large, low quality loss - recommended |
+| [Falcon3-7B-Instruct-Q6_K.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q6_K.gguf)     | Q6_K   | 6 | 6.12 GB| very large, extremely low quality loss |
+| [Falcon3-7B-Instruct-Q8_0.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-Q8_0.gguf)     | Q8_0   | 8 | 7.93 GB| very large, extremely low quality loss - not recommended |
+| [Falcon3-7B-Instruct-f16.gguf](https://huggingface.co/second-state/Falcon3-7B-Instruct-GGUF/blob/main/Falcon3-7B-Instruct-f16.gguf)       | f16    | 16 | 14.9 GB| |
+
+*Quantized with llama.cpp b4381*

config.json

@@ -0,0 +1,28 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 11,
+  "eos_token_id": 11,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "intermediate_size": 23040,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 4,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000042,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.46.1",
+  "use_cache": true,
+  "vocab_size": 131072
+}