初始化项目，由ModelHub XC社区提供模型
Model: EasierAI/Falcon-3-1B Source: Original Platform
This commit is contained in:
ModelHub XC
52
.gitattributes
vendored
Normal file
52
.gitattributes
vendored
Normal file
@@ -0,0 +1,52 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q5_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon3-1B-Instruct-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Falcon-3-1B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Falcon-3-1B-Instruct-Q2_K.gguf
Normal file
3
Falcon-3-1B-Instruct-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a2c2645263d17b1277627eb8cb16296138bdadcf47a0a34ee57e349bf7e6c7f2
|
||||
size 224950400
|
||||
3
Falcon-3-1B-Instruct-Q3_K_M.gguf
Normal file
3
Falcon-3-1B-Instruct-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:60048f391f09c8ff354f207b12894e63e640b6dca9ba3bdec379204f74b8de65
|
||||
size 884963456
|
||||
3
Falcon-3-1B-Instruct-Q4_0.gguf
Normal file
3
Falcon-3-1B-Instruct-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3b46b6e6e7c98208328c41522b9508a475aa667769b8ce0014eebe45a94a1899
|
||||
size 1013250176
|
||||
3
Falcon-3-1B-Instruct-Q4_K_M.gguf
Normal file
3
Falcon-3-1B-Instruct-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:54a4d303f5fb238db640433a61a600e308bfbb40cca2d7b451092e247df46616
|
||||
size 1057044608
|
||||
3
Falcon-3-1B-Instruct-Q5_0.gguf
Normal file
3
Falcon-3-1B-Instruct-Q5_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:041c14673e373d2714da69ae10aa239db390c65ae4327d7420e1027eb5c6a9de
|
||||
size 1188362368
|
||||
3
Falcon-3-1B-Instruct-Q5_K_M.gguf
Normal file
3
Falcon-3-1B-Instruct-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3cc311d49fef58586194b61316c907b3da7f83c94ccf7c2b158e1b1710e7df86
|
||||
size 1210923136
|
||||
3
Falcon-3-1B-Instruct-Q6_K.gguf
Normal file
3
Falcon-3-1B-Instruct-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:959bd1814873fd80b682a9094416c2b809fe0184917d408f651377cf07684570
|
||||
size 1374419072
|
||||
3
Falcon-3-1B-Instruct-Q8_0.gguf
Normal file
3
Falcon-3-1B-Instruct-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:dda44709157103fc272ea3c44ae0deb7fbec5b2d82b313362680dec08e66f57a
|
||||
size 1778710656
|
||||
126
README.md
Normal file
126
README.md
Normal file
@@ -0,0 +1,126 @@
|
||||
|
||||
---
|
||||
language:
|
||||
- en
|
||||
- fr
|
||||
- es
|
||||
- pt
|
||||
base_model:
|
||||
- tiiuae/Falcon3-1B-Instruct
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
tags:
|
||||
- falcon3
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
<img src="https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/general/falco3-logo.png" alt="drawing" width="500"/>
|
||||
</div>
|
||||
|
||||
# Falcon3-1B-Instruct-GGUF
|
||||
|
||||
|
||||
**Falcon3** family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B parameters.
|
||||
|
||||
**Falcon3-1B-Instruct** achieves strong results on reasoning, language understanding, instruction following, code and mathematics tasks.
|
||||
Falcon3-1B-Instruct supports 4 languages (English, French, Spanish, Portuguese) and a context length of up to 8K.
|
||||
|
||||
This repository contains the GGUFs instruction-tuned 1B Falcon3 model.
|
||||
|
||||
## Model Details
|
||||
- Architecture
|
||||
- Transformer-based causal decoder-only architecture
|
||||
- 18 decoder blocks
|
||||
- Grouped Query Attention (GQA) for faster inference: 8 query heads and 4 key-value heads
|
||||
- Wider head dimension: 256
|
||||
- High RoPE value to support long context understanding: 1000042
|
||||
- Uses SwiGLU and RMSNorm
|
||||
- 8K context length
|
||||
- 131K vocab size
|
||||
- Pruned and healed using larger Falcon models (3B and 7B respectively) on only 80 Gigatokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 256 H100 GPU chips
|
||||
- Posttrained on 1.2 million samples of STEM, conversational, code, safety and function call data
|
||||
- Supports EN, FR, ES, PT
|
||||
- Developed by [Technology Innovation Institute](https://www.tii.ae)
|
||||
- License: TII Falcon-LLM License 2.0
|
||||
- Model Release Date: December 2024
|
||||
- Quantization: q2_K, q3_K_M, q4_0, q4_K_M, q5_0, q5_K_M, q6_K, q8_0
|
||||
|
||||
|
||||
## Getting started
|
||||
|
||||
### 1. Download GGUF models from hugging face
|
||||
|
||||
First, download the model from Hugging Face. You can use the `huggingface_hub` library or download it manually:
|
||||
|
||||
|
||||
|
||||
```bash
|
||||
pip install huggingface_hub
|
||||
huggingface-cli download {model_name}
|
||||
```
|
||||
|
||||
|
||||
|
||||
This will download the model to your current directory. Make sure to replace {model_name} with the actual username and model name from your Hugging Face repository.
|
||||
|
||||
|
||||
|
||||
## 2. Install llama.cpp
|
||||
|
||||
You have several options for installing llama.cpp:
|
||||
|
||||
**1. Build from source:**
|
||||
|
||||
This gives you the most flexibility and control. Follow the instructions in the llama.cpp repository to build from source:
|
||||
|
||||
```bash
|
||||
|
||||
git clone https://github.com/ggerganov/llama.cpp
|
||||
cd llama.cpp
|
||||
cmake -B build
|
||||
cmake --build build --config Release
|
||||
```
|
||||
|
||||
For more information about how to build llama.cpp from source please refere to llama.cpp documentation on how to build from source: **[llama.cpp build from source](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md)**.
|
||||
|
||||
**2. Download pre-built binaries:**
|
||||
|
||||
If you prefer a quicker setup, you can download pre-built binaries for your operating system. Check the llama.cpp repository for available binaries.
|
||||
|
||||
**3. Use Docker:**
|
||||
|
||||
For a more contained environment, you can use the official llama.cpp Docker image. Refer to the llama.cpp documentation for instructions on how to use the Docker image.
|
||||
|
||||
For detailed instructions and more information, please check the llama.cpp documentation on docker: **[llama.cpp docker](https://github.com/ggerganov/llama.cpp/blob/master/docs/docker.mdg)**.
|
||||
|
||||
### 3. Start playing with your model
|
||||
|
||||
Run simple text completion
|
||||
```bash
|
||||
llama-cli -m {path-to-gguf-model} -p "I believe the meaning of life is" -n 128
|
||||
```
|
||||
|
||||
Run in conversation mode
|
||||
```bash
|
||||
llama-cli -m {path-to-gguf-model} -p "You are a helpful assistant" -cnv -co
|
||||
```
|
||||
## Useful links
|
||||
- View our [release blogpost](https://huggingface.co/blog/falcon3).
|
||||
- Feel free to join [our discord server](https://discord.gg/fwXpMyGc) if you have any questions or to interact with our researchers and developers.
|
||||
|
||||
## Technical Report
|
||||
Coming soon....
|
||||
|
||||
## Citation
|
||||
If the Falcon3 family of models were helpful to your work, feel free to give us a cite.
|
||||
|
||||
```
|
||||
@misc{Falcon3,
|
||||
title = {The Falcon 3 Family of Open Models},
|
||||
url = {https://huggingface.co/blog/falcon3},
|
||||
author = {Falcon-LLM Team},
|
||||
month = {December},
|
||||
year = {2024}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user