初始化项目,由ModelHub XC社区提供模型
Model: tiiuae/Falcon-H1-Tiny-Multilingual-100M-Base Source: Original Platform
This commit is contained in:
129
README.md
Normal file
129
README.md
Normal file
@@ -0,0 +1,129 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags:
|
||||
- falcon-h1
|
||||
- edge
|
||||
license: other
|
||||
license_name: falcon-llm-license
|
||||
license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
|
||||
---
|
||||
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/62441d1d9fdefb55a0b7d12c/l1du02RjuAZJcksI5tQ-F.png" alt="drawing" width="800"/>
|
||||
|
||||
# Table of Contents
|
||||
|
||||
0. [TL;DR](#TL;DR)
|
||||
1. [Model Details](#model-details)
|
||||
2. [Training Details](#training-details)
|
||||
3. [Usage](#usage)
|
||||
4. [Evaluation](#evaluation)
|
||||
5. [Citation](#citation)
|
||||
|
||||
# TL;DR
|
||||
|
||||
# Model Details
|
||||
|
||||
## Model Description
|
||||
|
||||
- **Developed by:** [https://www.tii.ae](https://www.tii.ae)
|
||||
- **Model type:** Causal decoder-only
|
||||
- **Architecture:** Hybrid Transformers + Mamba architecture
|
||||
- **Language(s) (NLP):** English
|
||||
- **Number of Parameters:** 90M
|
||||
- **License:** Falcon-LLM License
|
||||
|
||||
# Training details
|
||||
|
||||
For more details about the training protocol of this model, please refer to the [Falcon-H1-Tiny technical blogpost](https://huggingface.co/spaces/tiiuae/tiny-h1-blogpost).
|
||||
|
||||
# Usage
|
||||
|
||||
Currently to use this model you can either rely on Hugging Face `transformers`, `vLLM`, `sglang`, `llama.cpp`, `ollama` or `mlx` library.
|
||||
|
||||
## Inference
|
||||
|
||||
### 🤗 transformers
|
||||
|
||||
Refer to the snippet below to run H1 models using 🤗 transformers:
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "tiiuae/Falcon-H1-Tiny-100M-Multilingual-Base"
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
torch_dtype=torch.bfloat16,
|
||||
device_map="auto"
|
||||
)
|
||||
|
||||
# Perform text generation
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
transformers serve tiiuae/Falcon-H1-Tiny-100M-Multilingual-Base
|
||||
```
|
||||
|
||||
### `llama.cpp`
|
||||
|
||||
You can find all GGUF files compatible with `llama.cpp` under [our official collection]() - an example setup could be:
|
||||
|
||||
```bash
|
||||
brew install llama.cpp
|
||||
pip install huggingface_hub
|
||||
hf download tiiuae/Falcon-H1-Tiny-100M-Multilingual-Base Falcon-H1-Tiny-100M-Multilingual-Base-Q8_0.gguf --local-dir ./
|
||||
llama-cli ./ Falcon-H1-Tiny-100M-Multilingual-Base-Q8_0.gguf -cnv
|
||||
```
|
||||
|
||||
### `ollama`
|
||||
|
||||
```bash
|
||||
ollama run hf.co/tiiuae/Falcon-H1-Tiny-100M-Multilingual-Base:Q8_0
|
||||
```
|
||||
|
||||
### Apple `mlx`
|
||||
|
||||
```bash
|
||||
mlx_lm.chat --model tiiuae/Falcon-H1-Tiny-100M-Multilingual-Base
|
||||
```
|
||||
|
||||
### vLLM
|
||||
|
||||
For vLLM, simply start a server by executing the command below:
|
||||
|
||||
```bash
|
||||
# pip install vllm>=0.9.0
|
||||
vllm serve tiiuae/Falcon-H1-Tiny-100M-Multilingual-Base --tensor-parallel-size 2 --data-parallel-size 1
|
||||
```
|
||||
|
||||
### sglang
|
||||
|
||||
```bash
|
||||
python -m sglang.launch_server \
|
||||
--model ttiiuae/Falcon-H1-Tiny-100M-Multilingual-Base \
|
||||
--tensor-parallel-size 1
|
||||
```
|
||||
|
||||
# Evaluation
|
||||
|
||||
For detailed evaluation of Falcon-H1-Tiny series, please refer to our [technical blogpost](https://huggingface.co/spaces/tiiuae/tiny-h1-blogpost)
|
||||
|
||||
# Useful links
|
||||
|
||||
- View [our release blogpost](https://huggingface.co/spaces/tiiuae/tiny-h1-blogpost).
|
||||
- Feel free to join [our discord server](https://discord.gg/trwMYP9PYm) if you have any questions or to interact with our researchers and developers.
|
||||
|
||||
# Citation
|
||||
|
||||
If the Falcon-H1-Tiny family of models were helpful to your work, feel free to give us a cite.
|
||||
|
||||
```
|
||||
@misc{falcon_h1_tiny,
|
||||
title={Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale},
|
||||
author={Falcon-LLM Team},
|
||||
year={2026},
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user