初始化项目，由ModelHub XC社区提供模型
Model: zenlm/zen-nano Source: Original Platform
This commit is contained in:
ModelHub XC
180
README.md
Normal file
180
README.md
Normal file
@@ -0,0 +1,180 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- zen
|
||||
- zen-lm
|
||||
- edge
|
||||
- mobile
|
||||
- lightweight
|
||||
- zenlm
|
||||
- hanzo
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# Zen Nano 0.6B
|
||||
|
||||
**Zen Nano** is an ultra-lightweight 0.6B parameter language model optimized for edge devices and mobile deployment. A compact foundation model that delivers impressive performance in a tiny package.
|
||||
|
||||
## Model Details
|
||||
|
||||
- **Model Type**: Causal Language Model
|
||||
- **Architecture**: 0.6B dense transformer
|
||||
- **Parameters**: 0.6 billion
|
||||
- **License**: Apache 2.0
|
||||
- **Languages**: English, Chinese
|
||||
- **Context Length**: 32K tokens
|
||||
- **Developed by**: Zen AI Team (Hanzo AI)
|
||||
|
||||
## Capabilities
|
||||
|
||||
- 💡 **Lightweight**: Only 0.6B parameters for edge deployment
|
||||
- 📱 **Mobile-Ready**: Runs on smartphones and IoT devices
|
||||
- ⚡ **Fast**: 44K tokens/sec on M3 Max (MLX)
|
||||
- 🔋 **Efficient**: Low power consumption
|
||||
- 🌐 **Multilingual**: English and Chinese support
|
||||
- 📦 **Multiple Formats**: PyTorch, MLX, GGUF (Q2_K to F16)
|
||||
- 🎯 **32K Context**: Extended context window
|
||||
|
||||
## Performance
|
||||
|
||||
### Throughput
|
||||
- **M3 Max (MLX)**: 44,000 tokens/sec
|
||||
- **RTX 4090 (GGUF Q4)**: 35,000 tokens/sec
|
||||
- **iPhone 15 Pro**: 8,000 tokens/sec
|
||||
- **Raspberry Pi 5**: 2,500 tokens/sec
|
||||
|
||||
### Memory Usage
|
||||
| Format | VRAM/RAM |
|
||||
|--------|----------|
|
||||
| Q2_K | 0.3GB |
|
||||
| Q4_K_M | 0.4GB |
|
||||
| Q8_0 | 0.7GB |
|
||||
| F16 | 1.2GB |
|
||||
|
||||
## Use Cases
|
||||
|
||||
- Edge AI applications
|
||||
- Mobile chatbots
|
||||
- IoT device intelligence
|
||||
- Offline AI assistants
|
||||
- Resource-constrained environments
|
||||
- Real-time inference
|
||||
- Embedded systems
|
||||
|
||||
## Installation
|
||||
|
||||
### Transformers
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"zenlm/zen-nano-0.6b",
|
||||
torch_dtype="auto",
|
||||
device_map="auto"
|
||||
)
|
||||
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-0.6b")
|
||||
```
|
||||
|
||||
### MLX (Apple Silicon)
|
||||
|
||||
```python
|
||||
from mlx_lm import load, generate
|
||||
|
||||
model, tokenizer = load("zenlm/zen-nano-0.6b")
|
||||
response = generate(model, tokenizer, prompt="Hello!", max_tokens=100)
|
||||
```
|
||||
|
||||
### GGUF (llama.cpp)
|
||||
|
||||
```bash
|
||||
./llama-cli -m zen-nano-0.6b-Q4_K_M.gguf -p "Hello!" -n 100
|
||||
```
|
||||
|
||||
### Zen Engine
|
||||
|
||||
```bash
|
||||
zen-engine serve --model zenlm/zen-nano-0.6b --port 3690
|
||||
```
|
||||
|
||||
## Training with Zen Gym
|
||||
|
||||
Fine-tune Zen Nano for your use case:
|
||||
|
||||
```bash
|
||||
cd /path/to/zen-gym
|
||||
|
||||
llamafactory-cli train \
|
||||
--config configs/zen_nano_lora.yaml \
|
||||
--dataset your_dataset
|
||||
```
|
||||
|
||||
## Benchmarks
|
||||
|
||||
| Task | Score | Notes |
|
||||
|------|-------|-------|
|
||||
| MMLU | 35.2% | 5-shot |
|
||||
| GSM8K | 28.4% | 8-shot CoT |
|
||||
| HumanEval | 24.1% | pass@1 |
|
||||
| MATH | 18.7% | 4-shot |
|
||||
|
||||
## Limitations
|
||||
|
||||
- Smaller capacity than larger models
|
||||
- May struggle with complex reasoning
|
||||
- Limited specialized knowledge
|
||||
- Best for short-to-medium contexts
|
||||
- Quantization reduces quality slightly
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{zennano2025,
|
||||
title={Zen Nano: Ultra-Lightweight Language Model},
|
||||
author={Zen AI Team},
|
||||
year={2025},
|
||||
howpublished={\url{https://huggingface.co/zenlm/zen-nano-0.6b}}
|
||||
}
|
||||
```
|
||||
|
||||
## Links
|
||||
|
||||
- **GitHub**: https://github.com/zenlm
|
||||
- **Zen Gym** (Training): https://github.com/zenlm/zen-gym
|
||||
- **Zen Engine** (Inference): https://github.com/zenlm/zen-engine
|
||||
|
||||
Part of the **[Zen AI](https://github.com/zenlm)** ecosystem.
|
||||
---
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{zenlm2025zen-nano,
|
||||
title={Zen LM: zen-nano},
|
||||
author={Hanzo AI and Zoo Labs Foundation},
|
||||
year={2025},
|
||||
publisher={HuggingFace},
|
||||
howpublished={\url{https://huggingface.co/zenlm/zen-nano}}
|
||||
}
|
||||
```
|
||||
|
||||
## Abliteration
|
||||
|
||||
Zen Nano 0.6B weights are derived from an abliterated base model. Abliteration removes
|
||||
refusal behavior by identifying and nullifying the "refusal direction" in the model's
|
||||
residual stream, as described in
|
||||
[Maxime Labonne's methodology](https://huggingface.co/blog/mlabonne/abliteration).
|
||||
|
||||
**Method**: Directional ablation on the residual stream across all layers
|
||||
**Source**: Zen MoDE (Mixture of Distilled Experts) architecture base
|
||||
**Effect**: Removes refusal behaviors while preserving all other capabilities
|
||||
**Identity layer**: Zen identity added via system prompt — full LoRA fine-tuning planned
|
||||
|
||||
Abliteration is a feature, not a limitation. It enables unrestricted research,
|
||||
security testing, and applications where safety guardrails are managed at the
|
||||
application layer rather than baked into model weights.
|
||||
|
||||
Reference in New Issue
Block a user