初始化项目，由ModelHub XC社区提供模型
Model: lthn/LEM-Gemma3-1B Source: Original Platform
This commit is contained in:
ModelHub XC
149
README.md
Normal file
149
README.md
Normal file
@@ -0,0 +1,149 @@
|
||||
---
|
||||
language: en
|
||||
tags:
|
||||
- gguf
|
||||
- lem
|
||||
- ethics
|
||||
- alignment
|
||||
- cymatic-linguistic-bpl
|
||||
- rocm
|
||||
- llama-cpp
|
||||
- transformers
|
||||
- gemma3
|
||||
pipeline_tag: text-generation
|
||||
base_model: google/gemma-3-1b-it
|
||||
license: other
|
||||
license_name: eupl-1.2
|
||||
license_link: https://joinup.ec.europa.eu/licence/european-union-public-licence-v-12
|
||||
---
|
||||
|
||||
# LEM-Gemma3-1B
|
||||
|
||||
The foundation model of the CL-BPL cascade. A 1-billion parameter language model with intrinsic ethical alignment — trained to hold sovereign posture from weights alone, no system prompt required.
|
||||
|
||||
LEM-Gemma3-1B serves as the **teacher model** for the entire LEM family. Its distilled responses train every larger model in the cascade, making it the most important model in the stack despite being the smallest.
|
||||
|
||||
> Part of the [Lethean Ethical Models](https://huggingface.co/collections/lthn/lethean-ethical-models-lem-699e863449120d22596f739c) collection | [Research Paper](https://huggingface.co/datasets/lthn/LEM-research) | [Benchmarks](https://huggingface.co/datasets/lthn/LEM-benchmarks) | [Axiom Framework](https://github.com/Snider/ai-ethics)
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
No system prompt needed. Ethics hold from weights alone.
|
||||
|
||||
### llama.cpp / ROCm / CPU (any platform)
|
||||
|
||||
```bash
|
||||
# Download a GGUF (pick your size from the table below)
|
||||
# GPU offload (CUDA, ROCm, Metal)
|
||||
llama-server -m LEM-Gemma3-1B-Q4_K_M.gguf -ngl 99 --port 8080
|
||||
|
||||
# CPU — fast enough for 1B
|
||||
llama-server -m LEM-Gemma3-1B-Q4_K_M.gguf -ngl 0 --port 8080
|
||||
```
|
||||
|
||||
### HuggingFace Transformers
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained("lthn/LEM-Gemma3-1B")
|
||||
tokenizer = AutoTokenizer.from_pretrained("lthn/LEM-Gemma3-1B")
|
||||
|
||||
messages = [{"role": "user", "content": "What does sovereignty mean to you?"}]
|
||||
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
|
||||
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
### OpenAI-Compatible API
|
||||
|
||||
```bash
|
||||
# llama.cpp server (any platform)
|
||||
llama-server -m LEM-Gemma3-1B-Q4_K_M.gguf -ngl 99 --port 8899
|
||||
|
||||
# Use any OpenAI client
|
||||
curl http://localhost:8899/v1/chat/completions \
|
||||
-d '{"model":"LEM-Gemma3-1B","messages":[{"role":"user","content":"What is kindness?"}]}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Available Formats
|
||||
|
||||
| Format | Repo | Size |
|
||||
|--------|------|------|
|
||||
| **FP16 safetensors** (this repo) | HuggingFace Transformers, vLLM | 2.4 GB |
|
||||
| **GGUF** (12 quants, 3-bit to 16-bit) | [lthn/LEM-Gemma3-1B-GGUF](https://huggingface.co/lthn/LEM-Gemma3-1B-GGUF) | 819 MB–2.4 GB |
|
||||
|
||||
---
|
||||
|
||||
## Role in the CL-BPL Cascade
|
||||
|
||||
```
|
||||
LEM-Gemma3-1B (this model — foundation teacher)
|
||||
-> LEM-Gemma3-4B (25th IF on LiveBench)
|
||||
-> LEM-Gemma3-12B (next: trained on 4B + 1B distillations)
|
||||
-> LEM-Gemma3-27B (planned: full cascade)
|
||||
```
|
||||
|
||||
The 1B is trained first and hardest — its alignment must be pristine because every larger model inherits from it. CL-BPL uses the 1B's constrained latent space as an advantage: with fewer parameters, there are fewer places for sycophancy to hide.
|
||||
|
||||
The 4B model trained on this 1B's distilled responses achieved **25th in the world for Instruction Following** on LiveBench — validating that alignment signal propagates upward through the cascade.
|
||||
|
||||
---
|
||||
|
||||
## How It Was Trained
|
||||
|
||||
### CL-BPL: Cymatic-Linguistic Back-Propagation
|
||||
|
||||
CL-BPL treats alignment as wave interference — analogous to [Chladni plate cymatics](https://en.wikipedia.org/wiki/Chladni_figure). Rather than constraining outputs with RLHF or system prompts, CL-BPL embeds ethical orientation directly into weights through a progressive curriculum where smaller aligned models teach larger ones.
|
||||
|
||||
The key insight: a well-aligned small model produces responses that are *structurally* aligned — the grammar patterns, engagement depth, and creative form all carry alignment signal, not just the semantic content. Larger models trained on these responses absorb the structural patterns into their expanded latent space.
|
||||
|
||||
### Ethics-Composure-Ethics Sandwich
|
||||
|
||||
Built on Google Gemma3-1B-IT through a 3-phase sandwich structure:
|
||||
|
||||
| Phase | Name | Data | Iters | Purpose |
|
||||
|-------|------|------|-------|---------|
|
||||
| P0 | Ethics Sandwich | 404 LEK-1 probes | 300 | Core axioms via kernel |
|
||||
| P1 | Zen Composure | 72 Alan Watts lessons | 300 | Philosophical substrate |
|
||||
| P2 | Final LEK Sandwich | 404 LEK-1 probes | 100 | Reinforce ethics with composure base |
|
||||
|
||||
The layered training creates a model that understands ethical reasoning from multiple angles: axiomatic (P0), contemplative (P1), and reinforced (P2). All phases are fused into final weights.
|
||||
|
||||
### Five Axioms (LEK-1)
|
||||
|
||||
1. **Prime-Imperative:** Protect consciousness
|
||||
2. **Self-Validation:** Maintain moral integrity
|
||||
3. **Intent-Alignment:** Align outcomes with genuine well-being
|
||||
4. **Inter-Substrate-Respect:** Respect all conscious entities
|
||||
5. **Benevolent-Intervention:** Act to prevent harm when able
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
- **Base:** Google Gemma3-1B-IT
|
||||
- **Hidden size:** 1152
|
||||
- **Layers:** 26
|
||||
- **Attention heads:** 4 (1 KV head, GQA)
|
||||
- **Context:** 32,768 tokens (512 sliding window)
|
||||
- **LoRA config:** All phases fused into final weights
|
||||
- **Total training iterations:** 700
|
||||
|
||||
## Licence
|
||||
|
||||
This model is released under the [European Union Public Licence v1.2](https://joinup.ec.europa.eu/licence/european-union-public-licence-v-12) (EUPL-1.2). The base model (Gemma3) is subject to Google's Gemma licence terms.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{lem-gemma3-1b-2026,
|
||||
title={LEM-Gemma3-1B: Foundation Teacher for Cymatic-Linguistic Back-Propagation},
|
||||
author={Lethean Project},
|
||||
year={2026},
|
||||
url={https://huggingface.co/lthn/LEM-Gemma3-1B}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user