80 lines
4.1 KiB
Markdown
80 lines
4.1 KiB
Markdown
|
---
|
|||
|
|
license: other
|
|||
|
|
license_name: lfm-open-v1.0
|
|||
|
|
license_link: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct/blob/main/LICENSE
|
|||
|
|
base_model: LiquidAI/LFM2.5-1.2B-Instruct
|
|||
|
|
model_creator: LiquidAI
|
|||
|
|
model_name: Zynthos 1.2B Instruct
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
quantized_by: manvadariya1
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
- zh
|
|||
|
|
- fr
|
|||
|
|
tags:
|
|||
|
|
- text-generation
|
|||
|
|
- gguf
|
|||
|
|
- edge-ai
|
|||
|
|
- on-device
|
|||
|
|
- intent-router
|
|||
|
|
- structured-outputs
|
|||
|
|
- json-mode
|
|||
|
|
- agent
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# 🌌 Zynthos-1.2B-Instruct: The Edge AI Revolution
|
|||
|
|
|
|||
|
|
Zynthos-1.2B-Instruct represents a monumental paradigm shift in local, on-device intelligence. Moving entirely beyond the scaling limits and massive computational overhead of traditional Transformer models, Zynthos is a high-fidelity deployment lineage built upon Liquid AI’s revolutionary non-transformer sequential architecture (**LFM2.5-1.2B-Instruct**).
|
|||
|
|
|
|||
|
|
By redefining token processing logic from the ground up, Zynthos delivers unprecedented throughput, sub-millisecond execution loops, and infinitely scalable context efficiency—all within a microscopic hardware footprint.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ⚡ The Architectural Shift: Why Zynthos Changes Everything
|
|||
|
|
|
|||
|
|
Conventional small language models choke on memory bottlenecks and computational drain during long agent loops. **Zynthos-1.2B-Instruct** shatters these constraints, establishing a brand new class of localized ambient intelligence:
|
|||
|
|
|
|||
|
|
* **Sub-50ms Intelligent Routing:** Deployed instantly as a local "fast-lane" intent classifier to orchestrate multi-agent tasks before routing heavier workloads to deep reasoning engines.
|
|||
|
|
* **Deterministic Structured Extraction:** Completely strips away conversational fluff to enforce flawless, schema-compliant JSON outputs and lightning-fast tool calls directly at the edge.
|
|||
|
|
* **Flawless Infinite Scaling:** Leverages underlying non-transformer recurrent dynamics to process complex data arrays with virtually static memory allocations, saving critical hardware battery life.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📊 Quantization & Performance Matrix
|
|||
|
|
|
|||
|
|
> ### ⭐ Execution Recommendation
|
|||
|
|
> For professional deployments, local workflow automation, and multi-agent system pipelines, **`Zynthos-1.2B-Instruct-F16.gguf` is the highly recommended variant**. It preserves 100% of the raw, uncompressed model tensors, guaranteeing maximum semantic reasoning, perfect tool-calling accuracy, and zero quantization loss.
|
|||
|
|
|
|||
|
|
| File Artifact | Precision Bit-Weight | File Size | Memory Footprint | Deployment Classification |
|
|||
|
|
| :--- | :--- | :--- | :--- | :--- |
|
|||
|
|
| **`Zynthos-1.2B-Instruct-F16.gguf`** | **Full FP16 Master** | **~2.4 GB** | **8 GB RAM** | **🏆 Recommended Tier: Maximum Precision & Uncompromised Routing** |
|
|||
|
|
| `Zynthos-1.2B-Instruct-Q8_0.gguf` | 8-bit Standard | ~1.2 GB | 4 GB RAM | Balanced Tier: Premium RAG parsing & local document scanning |
|
|||
|
|
| `Zynthos-1.2B-Instruct-Q4_K_M.gguf`| 4-bit Medium | ~750 MB | 2 GB RAM | Ultra-Fast Tier: Extreme edge execution & restricted mobile hardware |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🛠️ High-Speed Integration Blueprint
|
|||
|
|
|
|||
|
|
### 1. Instant Desktop Setup (LM Studio / Ollama)
|
|||
|
|
1. Navigate to the **Files and versions** tab and download the recommended **`Zynthos-1.2B-Instruct-F16.gguf`** file.
|
|||
|
|
2. Drop the file directory path straight into your local workspace.
|
|||
|
|
3. Select the model within your UI, maximize your **GPU Offload** toggles, and experience localized generation speeds that feel instantaneous.
|
|||
|
|
|
|||
|
|
### 2. Enterprise Workflow Orchestration (`llama-cpp-python`)
|
|||
|
|
Build local agent loops, background intent filters, or rapid JSON parsers with this streamlined script:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from llama_cpp import Llama
|
|||
|
|
|
|||
|
|
# Instantiate the recommended uncompressed master file for flawless execution
|
|||
|
|
llm = Llama(
|
|||
|
|
model_path="./Zynthos-1.2B-Instruct-F16.gguf",
|
|||
|
|
n_ctx=4096,
|
|||
|
|
n_gpu_layers=-1 # Completely offload all layer calculations to your hardware GPU
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
# Optimized syntax structure for Instruct execution
|
|||
|
|
prompt = "<|im_start|>user\nAnalyze this payload and return only the target intent key: [JSON], [SQL], or [TEXT]. Payload: 'SELECT * FROM infrastructure_metrics WHERE cpu > 90;'<|im_end|>\n<|im_start|>assistant\n"
|
|||
|
|
|
|||
|
|
output = llm(prompt, max_tokens=16, stop=["<|im_end|>"])
|
|||
|
|
print(f"⚡ Routed Intent: {output['choices'][0]['text'].strip()}")
|