zen-nano/README.md

---
language:
- en
- zh
license: apache-2.0
tags:
- zen
- zen-lm
- edge
- mobile
- lightweight
- zenlm
  - hanzo
pipeline_tag: text-generation
library_name: transformers
---

# Zen Nano 0.6B

**Zen Nano** is an ultra-lightweight 0.6B parameter language model optimized for edge devices and mobile deployment. A compact foundation model that delivers impressive performance in a tiny package.

## Model Details

- **Model Type**: Causal Language Model
- **Architecture**: 0.6B dense transformer
- **Parameters**: 0.6 billion
- **License**: Apache 2.0
- **Languages**: English, Chinese
- **Context Length**: 32K tokens
- **Developed by**: Zen AI Team (Hanzo AI)

## Capabilities

- 💡 **Lightweight**: Only 0.6B parameters for edge deployment
- 📱 **Mobile-Ready**: Runs on smartphones and IoT devices
- ⚡ **Fast**: 44K tokens/sec on M3 Max (MLX)
- 🔋 **Efficient**: Low power consumption
- 🌐 **Multilingual**: English and Chinese support
- 📦 **Multiple Formats**: PyTorch, MLX, GGUF (Q2_K to F16)
- 🎯 **32K Context**: Extended context window

## Performance

### Throughput
- **M3 Max (MLX)**: 44,000 tokens/sec
- **RTX 4090 (GGUF Q4)**: 35,000 tokens/sec
- **iPhone 15 Pro**: 8,000 tokens/sec
- **Raspberry Pi 5**: 2,500 tokens/sec

### Memory Usage
| Format | VRAM/RAM |
|--------|----------|
| Q2_K | 0.3GB |
| Q4_K_M | 0.4GB |
| Q8_0 | 0.7GB |
| F16 | 1.2GB |

## Use Cases

- Edge AI applications
- Mobile chatbots
- IoT device intelligence
- Offline AI assistants
- Resource-constrained environments
- Real-time inference
- Embedded systems

## Installation

### Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "zenlm/zen-nano-0.6b",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-0.6b")
```

### MLX (Apple Silicon)

```python
from mlx_lm import load, generate

model, tokenizer = load("zenlm/zen-nano-0.6b")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=100)
```

### GGUF (llama.cpp)

```bash
./llama-cli -m zen-nano-0.6b-Q4_K_M.gguf -p "Hello!" -n 100
```

### Zen Engine

```bash
zen-engine serve --model zenlm/zen-nano-0.6b --port 3690
```

## Training with Zen Gym

Fine-tune Zen Nano for your use case:

```bash
cd /path/to/zen-gym

llamafactory-cli train \
    --config configs/zen_nano_lora.yaml \
    --dataset your_dataset
```

## Benchmarks

| Task | Score | Notes |
|------|-------|-------|
| MMLU | 35.2% | 5-shot |
| GSM8K | 28.4% | 8-shot CoT |
| HumanEval | 24.1% | pass@1 |
| MATH | 18.7% | 4-shot |

## Limitations

- Smaller capacity than larger models
- May struggle with complex reasoning
- Limited specialized knowledge
- Best for short-to-medium contexts
- Quantization reduces quality slightly

## Citation

```bibtex
@misc{zennano2025,
  title={Zen Nano: Ultra-Lightweight Language Model},
  author={Zen AI Team},
  year={2025},
  howpublished={\url{https://huggingface.co/zenlm/zen-nano-0.6b}}
}
```

## Links

- **GitHub**: https://github.com/zenlm
- **Zen Gym** (Training): https://github.com/zenlm/zen-gym
- **Zen Engine** (Inference): https://github.com/zenlm/zen-engine

Part of the **[Zen AI](https://github.com/zenlm)** ecosystem.
---

## Citation

```bibtex
@misc{zenlm2025zen-nano,
    title={Zen LM: zen-nano},
    author={Hanzo AI and Zoo Labs Foundation},
    year={2025},
    publisher={HuggingFace},
    howpublished={\url{https://huggingface.co/zenlm/zen-nano}}
}
```

## Abliteration

Zen Nano 0.6B weights are derived from an abliterated base model. Abliteration removes
refusal behavior by identifying and nullifying the "refusal direction" in the model's
residual stream, as described in
[Maxime Labonne's methodology](https://huggingface.co/blog/mlabonne/abliteration).

**Method**: Directional ablation on the residual stream across all layers
**Source**: Zen MoDE (Mixture of Distilled Experts) architecture base
**Effect**: Removes refusal behaviors while preserving all other capabilities
**Identity layer**: Zen identity added via system prompt — full LoRA fine-tuning planned

Abliteration is a feature, not a limitation. It enables unrestricted research,
security testing, and applications where safety guardrails are managed at the
application layer rather than baked into model weights.