初始化项目,由ModelHub XC社区提供模型
Model: unsloth/LFM2-2.6B-Exp-GGUF Source: Original Platform
This commit is contained in:
67
.gitattributes
vendored
Normal file
67
.gitattributes
vendored
Normal file
@@ -0,0 +1,67 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
LFM2-2.6B-Exp-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-UD-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-UD-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-UD-Q2_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-UD-Q6_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-BF16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-UD-Q8_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
LFM2-2.6B-Exp-UD-Q5_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
LFM2-2.6B-Exp-BF16.gguf
Normal file
3
LFM2-2.6B-Exp-BF16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4245df9558d58f8f0b9bf35fed571a5ceb28cdb18bb34ea936634e034620a8b0
|
||||
size 5141459584
|
||||
3
LFM2-2.6B-Exp-Q2_K.gguf
Normal file
3
LFM2-2.6B-Exp-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8150e847557907c3608d05861cd8c51aa5839c281ed71d66d5ddd5a2af8d727d
|
||||
size 983806592
|
||||
3
LFM2-2.6B-Exp-Q2_K_L.gguf
Normal file
3
LFM2-2.6B-Exp-Q2_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8150e847557907c3608d05861cd8c51aa5839c281ed71d66d5ddd5a2af8d727d
|
||||
size 983806592
|
||||
3
LFM2-2.6B-Exp-Q3_K_M.gguf
Normal file
3
LFM2-2.6B-Exp-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:79aa88595c3c213406016da34d0fd349b84c1b4bf2157ba3d915f212ba175cff
|
||||
size 1256059520
|
||||
3
LFM2-2.6B-Exp-Q3_K_S.gguf
Normal file
3
LFM2-2.6B-Exp-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:83e81a69b2bed9ce0b5860ad7a5eb04feabd285afe631664ea5197524b68ee36
|
||||
size 1159737984
|
||||
3
LFM2-2.6B-Exp-Q4_0.gguf
Normal file
3
LFM2-2.6B-Exp-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:52a4385c1b26d67d59370e7f2b799b835a37f1d41bdb76dad1da975959819999
|
||||
size 1483108992
|
||||
3
LFM2-2.6B-Exp-Q4_1.gguf
Normal file
3
LFM2-2.6B-Exp-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f08a724008ddca6b6a3df4fb041119f4ee4295c1e08970ec798efd228bb52e3c
|
||||
size 1635283584
|
||||
3
LFM2-2.6B-Exp-Q4_K_M.gguf
Normal file
3
LFM2-2.6B-Exp-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6ce3c8dd3271034833fa362d14de477f9ffc19d5549ae010bf9c8250471aa9c8
|
||||
size 1563669120
|
||||
3
LFM2-2.6B-Exp-Q4_K_S.gguf
Normal file
3
LFM2-2.6B-Exp-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:11874d6763b5fa8691e00ad2d316047b15d1e9266c8b4fc45d4976eb18e3d336
|
||||
size 1491890816
|
||||
3
LFM2-2.6B-Exp-Q5_K_M.gguf
Normal file
3
LFM2-2.6B-Exp-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5f5918473920c33c084de2ce09d86d5ce1a693853aa0345a61e423e76c27dda8
|
||||
size 1828958848
|
||||
3
LFM2-2.6B-Exp-Q5_K_S.gguf
Normal file
3
LFM2-2.6B-Exp-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:797498cf86c22a284f7f440b11ed6ad2e3f960feef5f4da9e7dcf2fd383da652
|
||||
size 1787458176
|
||||
3
LFM2-2.6B-Exp-Q6_K.gguf
Normal file
3
LFM2-2.6B-Exp-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0ba521d5d0327a550ae82a4bc1bc0eb38de694e3463e1a76bf1066f70245bdff
|
||||
size 2110829184
|
||||
3
LFM2-2.6B-Exp-Q8_0.gguf
Normal file
3
LFM2-2.6B-Exp-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:71307ae4c3f01259a7ba73ced04dcca4932d9aa8c5e7ab479085c40f4eaff0b8
|
||||
size 2733011584
|
||||
3
LFM2-2.6B-Exp-UD-Q2_K_XL.gguf
Normal file
3
LFM2-2.6B-Exp-UD-Q2_K_XL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8150e847557907c3608d05861cd8c51aa5839c281ed71d66d5ddd5a2af8d727d
|
||||
size 983806592
|
||||
3
LFM2-2.6B-Exp-UD-Q3_K_XL.gguf
Normal file
3
LFM2-2.6B-Exp-UD-Q3_K_XL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:79aa88595c3c213406016da34d0fd349b84c1b4bf2157ba3d915f212ba175cff
|
||||
size 1256059520
|
||||
3
LFM2-2.6B-Exp-UD-Q4_K_XL.gguf
Normal file
3
LFM2-2.6B-Exp-UD-Q4_K_XL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6ce3c8dd3271034833fa362d14de477f9ffc19d5549ae010bf9c8250471aa9c8
|
||||
size 1563669120
|
||||
3
LFM2-2.6B-Exp-UD-Q5_K_XL.gguf
Normal file
3
LFM2-2.6B-Exp-UD-Q5_K_XL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5f5918473920c33c084de2ce09d86d5ce1a693853aa0345a61e423e76c27dda8
|
||||
size 1828958848
|
||||
3
LFM2-2.6B-Exp-UD-Q6_K_XL.gguf
Normal file
3
LFM2-2.6B-Exp-UD-Q6_K_XL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a415393aa0a13994996d69f0315f43d627eb19e7a6e60444b42ebb5227617205
|
||||
size 2143335040
|
||||
3
LFM2-2.6B-Exp-UD-Q8_K_XL.gguf
Normal file
3
LFM2-2.6B-Exp-UD-Q8_K_XL.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ed15270c75bc08de9ba4c909021a42b9f6ca863608e7f1c9385b8cb4e6883dcd
|
||||
size 2858840704
|
||||
251
README.md
Normal file
251
README.md
Normal file
@@ -0,0 +1,251 @@
|
||||
---
|
||||
base_model:
|
||||
- LiquidAI/LFM2-2.6B-Exp
|
||||
library_name: transformers
|
||||
license: other
|
||||
license_name: lfm1.0
|
||||
license_link: https://huggingface.co/LiquidAI/LFM2-2.6B-Exp/blob/main/LICENSE
|
||||
language:
|
||||
- en
|
||||
- ar
|
||||
- zh
|
||||
- fr
|
||||
- de
|
||||
- ja
|
||||
- ko
|
||||
- es
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- liquid
|
||||
- unsloth
|
||||
- lfm2
|
||||
- edge
|
||||
---
|
||||
> [!NOTE]
|
||||
> Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
|
||||
>
|
||||
|
||||
<div>
|
||||
<p style="margin-top: 0;margin-bottom: 0;">
|
||||
<em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
|
||||
</p>
|
||||
<div style="display: flex; gap: 5px; align-items: center; ">
|
||||
<a href="https://github.com/unslothai/unsloth/">
|
||||
<img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
|
||||
</a>
|
||||
<a href="https://discord.gg/unsloth">
|
||||
<img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
|
||||
</a>
|
||||
<a href="https://docs.unsloth.ai/">
|
||||
<img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<center>
|
||||
<div style="text-align: center;">
|
||||
<img
|
||||
src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png"
|
||||
alt="Liquid AI"
|
||||
style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
|
||||
/>
|
||||
</div>
|
||||
<div style="display: flex; justify-content: center; gap: 0.5em;">
|
||||
<a href="https://playground.liquid.ai/chat">
|
||||
<a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> • <a href="https://docs.liquid.ai/lfm"><strong>Documentation</strong></a> • <a href="https://leap.liquid.ai/"><strong>LEAP</strong></a></a>
|
||||
</div>
|
||||
</center>
|
||||
|
||||
# LFM2-2.6B-Exp
|
||||
|
||||
LFM2-2.6B-Exp is an experimental checkpoint built on [LFM2-2.6B](https://huggingface.co/LiquidAI/LFM2-2.6B) using pure reinforcement learning.
|
||||
|
||||
Specifically trained on instruction following, knowledge, and math, it delivers particularly strong performance compared to other 3B models.
|
||||
In particular, its IFBench score surpasses DeepSeek R1-0528, a model 263 times larger.
|
||||
|
||||

|
||||
|
||||
Find more information about LFM2 in our [blog post](https://www.liquid.ai/blog/liquid-foundation-models-v2-our-second-series-of-generative-ai-models).
|
||||
|
||||
## 📄 Model details
|
||||
|
||||
Due to their small size, **we recommend fine-tuning LFM2 models on narrow use cases** to maximize performance.
|
||||
They are particularly suited for agentic tasks, data extraction, RAG, creative writing, and multi-turn conversations.
|
||||
However, we do not recommend using them for tasks that are knowledge-intensive or require programming skills.
|
||||
|
||||
| Property | [**LFM2-350M**](https://huggingface.co/LiquidAI/LFM2-350M) | [**LFM2-700M**](https://huggingface.co/LiquidAI/LFM2-700M) | [**LFM2-1.2B**](https://huggingface.co/LiquidAI/LFM2-1.2B) | [**LFM2-2.6B**](https://huggingface.co/LiquidAI/LFM2-2.6B) |
|
||||
| ------------------- | ----------------------------- | ----------------------------- | ----------------------------- | ----------------------------- |
|
||||
| **Parameters** | 354,483,968 | 742,489,344 | 1,170,340,608 | 2,569,272,320 |
|
||||
| **Layers** | 16 (10 conv + 6 attn) | 16 (10 conv + 6 attn) | 16 (10 conv + 6 attn) | 30 (22 conv + 8 attn) |
|
||||
| **Context length** | 32,768 tokens | 32,768 tokens | 32,768 tokens | 32,768 tokens |
|
||||
| **Vocabulary size** | 65,536 | 65,536 | 65,536 | 65,536 |
|
||||
| **Precision** | bfloat16 | bfloat16 | bfloat16 | bfloat16 |
|
||||
| **Training budget** | 10 trillion tokens | 10 trillion tokens | 10 trillion tokens | 10 trillion tokens |
|
||||
| **License** | LFM Open License v1.0 | LFM Open License v1.0 | LFM Open License v1.0 | LFM Open License v1.0
|
||||
|
||||
**Supported languages**: English, Arabic, Chinese, French, German, Japanese, Korean, and Spanish.
|
||||
|
||||
**Generation parameters**: We recommend the following parameters:
|
||||
* `temperature=0.3`
|
||||
* `min_p=0.15`
|
||||
* `repetition_penalty=1.05`
|
||||
|
||||
**Chat template**: LFM2 uses a ChatML-like chat template as follows:
|
||||
|
||||
```
|
||||
<|startoftext|><|im_start|>system
|
||||
You are a helpful assistant trained by Liquid AI.<|im_end|>
|
||||
<|im_start|>user
|
||||
What is C. elegans?<|im_end|>
|
||||
<|im_start|>assistant
|
||||
It's a tiny nematode that lives in temperate soil environments.<|im_end|>
|
||||
```
|
||||
|
||||
You can automatically apply it using the dedicated [`.apply_chat_template()`](https://huggingface.co/docs/transformers/en/chat_templating#applychattemplate) function from Hugging Face transformers.
|
||||
|
||||
**Tool use**: It consists of four main steps:
|
||||
1. **Function definition**: LFM2 takes JSON function definitions as input (JSON objects between `<|tool_list_start|>` and `<|tool_list_end|>` special tokens), usually in the system prompt
|
||||
2. **Function call**: LFM2 writes Pythonic function calls (a Python list between `<|tool_call_start|>` and `<|tool_call_end|>` special tokens), as the assistant answer.
|
||||
3. **Function execution**: The function call is executed and the result is returned (string between `<|tool_response_start|>` and `<|tool_response_end|>` special tokens), as a "tool" role.
|
||||
4. **Final answer**: LFM2 interprets the outcome of the function call to address the original user prompt in plain text.
|
||||
|
||||
Here is a simple example of a conversation using tool use:
|
||||
|
||||
```
|
||||
<|startoftext|><|im_start|>system
|
||||
List of tools: <|tool_list_start|>[{"name": "get_candidate_status", "description": "Retrieves the current status of a candidate in the recruitment process", "parameters": {"type": "object", "properties": {"candidate_id": {"type": "string", "description": "Unique identifier for the candidate"}}, "required": ["candidate_id"]}}]<|tool_list_end|><|im_end|>
|
||||
<|im_start|>user
|
||||
What is the current status of candidate ID 12345?<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<|tool_call_start|>[get_candidate_status(candidate_id="12345")]<|tool_call_end|>Checking the current status of candidate ID 12345.<|im_end|>
|
||||
<|im_start|>tool
|
||||
<|tool_response_start|>[{"candidate_id": "12345", "status": "Interview Scheduled", "position": "Clinical Research Associate", "date": "2023-11-20"}]<|tool_response_end|><|im_end|>
|
||||
<|im_start|>assistant
|
||||
The candidate with ID 12345 is currently in the "Interview Scheduled" stage for the position of Clinical Research Associate, with an interview date set for 2023-11-20.<|im_end|>
|
||||
```
|
||||
|
||||
You can directly pass tools as JSON schema or Python functions with `.apply_chat_template()` as shown in [this page](https://huggingface.co/docs/transformers/en/chat_extras) to automatically format the system prompt.
|
||||
|
||||
**Architecture**: Hybrid model with multiplicative gates and short convolutions: 10 double-gated short-range LIV convolution blocks and 6 grouped query attention (GQA) blocks.
|
||||
|
||||
**Pre-training mixture**: Approximately 75% English, 20% multilingual, and 5% code data sourced from the web and licensed materials.
|
||||
|
||||
**Training approach**:
|
||||
* Very large-scale SFT on 50% downstream tasks, 50% general domains
|
||||
* Custom DPO with length normalization and semi-online datasets
|
||||
* Iterative model merging
|
||||
* Reinforcement learning with verifiable rewards
|
||||
|
||||
## 🏃 How to run LFM2
|
||||
|
||||
### 1. Transformers
|
||||
|
||||
To run LFM2, you need to install Hugging Face [`transformers`](https://github.com/huggingface/transformers) v4.55 or a more recent version as follows:
|
||||
|
||||
```bash
|
||||
pip install -U transformers
|
||||
```
|
||||
|
||||
Here is an example of how to generate an answer with transformers in Python:
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
# Load model and tokenizer
|
||||
model_id = "LiquidAI/LFM2-2.6B-Exp"
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
device_map="auto",
|
||||
torch_dtype="bfloat16",
|
||||
# attn_implementation="flash_attention_2" <- uncomment on compatible GPU
|
||||
)
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
|
||||
# Generate answer
|
||||
prompt = "What is C. elegans?"
|
||||
input_ids = tokenizer.apply_chat_template(
|
||||
[{"role": "user", "content": prompt}],
|
||||
add_generation_prompt=True,
|
||||
return_tensors="pt",
|
||||
tokenize=True,
|
||||
).to(model.device)
|
||||
|
||||
output = model.generate(
|
||||
input_ids,
|
||||
do_sample=True,
|
||||
temperature=0.3,
|
||||
min_p=0.15,
|
||||
repetition_penalty=1.05,
|
||||
max_new_tokens=512,
|
||||
)
|
||||
|
||||
print(tokenizer.decode(output[0], skip_special_tokens=False))
|
||||
|
||||
# <|startoftext|><|im_start|>user
|
||||
# What is C. elegans?<|im_end|>
|
||||
# <|im_start|>assistant
|
||||
# C. elegans, also known as Caenorhabditis elegans, is a small, free-living
|
||||
# nematode worm (roundworm) that belongs to the phylum Nematoda.
|
||||
```
|
||||
|
||||
You can directly run and test the model with this [Colab notebook](https://colab.research.google.com/drive/1_q3jQ6LtyiuPzFZv7Vw8xSfPU5FwkKZY?usp=sharing).
|
||||
|
||||
### 2. vLLM
|
||||
|
||||
You need to install [`vLLM`](https://github.com/vllm-project/vllm) v0.10.2 or a more recent version as follows:
|
||||
|
||||
```bash
|
||||
uv pip install vllm==0.10.2 --extra-index-url https://wheels.vllm.ai/0.10.2/ --torch-backend=auto
|
||||
```
|
||||
|
||||
Here is an example of how to use it for inference:
|
||||
|
||||
```python
|
||||
from vllm import LLM, SamplingParams
|
||||
|
||||
prompts = [
|
||||
"What is C. elegans?",
|
||||
"Say hi in JSON format",
|
||||
"Define AI in Spanish"
|
||||
]
|
||||
sampling_params = SamplingParams(temperature=0.3, min_p=0.15, repetition_penalty=1.05)
|
||||
|
||||
llm = LLM(model="LiquidAI/LFM2-2.6B-Exp")
|
||||
|
||||
outputs = llm.generate(prompts, sampling_params)
|
||||
|
||||
for output in outputs:
|
||||
prompt = output.prompt
|
||||
generated_text = output.outputs[0].text
|
||||
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
|
||||
```
|
||||
|
||||
### 3. llama.cpp
|
||||
|
||||
You can run LFM2 with llama.cpp using its [GGUF checkpoint](https://huggingface.co/LiquidAI/LFM2-2.6B-Exp-GGUF). Find more information in the model card.
|
||||
|
||||
## 🔧 How to fine-tune LFM2
|
||||
|
||||
We recommend fine-tuning LFM2 models on your use cases to maximize performance.
|
||||
|
||||
| Notebook | Description | Link |
|
||||
|-------|------|------|
|
||||
| SFT (Unsloth) | Supervised Fine-Tuning (SFT) notebook with a LoRA adapter using Unsloth. | <a href="https://colab.research.google.com/drive/1HROdGaPFt1tATniBcos11-doVaH7kOI3?usp=sharing"><img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png" width="110" alt="Colab link"></a> |
|
||||
| SFT (TRL) | Supervised Fine-Tuning (SFT) notebook with a LoRA adapter using TRL. | <a href="https://colab.research.google.com/drive/1j5Hk_SyBb2soUsuhU0eIEA9GwLNRnElF?usp=sharing"><img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png" width="110" alt="Colab link"></a> |
|
||||
| DPO (TRL) | Preference alignment with Direct Preference Optimization (DPO) using TRL. | <a href="https://colab.research.google.com/drive/1MQdsPxFHeZweGsNx4RH7Ia8lG8PiGE1t?usp=sharing"><img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/vlOyMEjwHa_b_LXysEu2E.png" width="110" alt="Colab link"></a> |
|
||||
|
||||
## 📬 Contact
|
||||
|
||||
If you are interested in custom solutions with edge deployment, please contact [our sales team](https://www.liquid.ai/contact).
|
||||
|
||||
## Citation
|
||||
|
||||
```
|
||||
@article{liquidai2025lfm2,
|
||||
title={LFM2 Technical Report},
|
||||
author={Liquid AI},
|
||||
journal={arXiv preprint arXiv:2511.23404},
|
||||
year={2025}
|
||||
}
|
||||
```
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||
Reference in New Issue
Block a user