ShaziLLM-1B/README.md

---
library_name: transformers
license: apache-2.0
language:
- en
- zh
base_model: openbmb/MiniCPM5-1B
base_model_relation: finetune
pipeline_tag: text-generation
tags:
- shazillm
- llama
- text-generation
- thinking
- coding
- instruction-following
- conversational
---

# ShaziLLM-1B

ShaziLLM-1B is a compact 1.08B-parameter instruction and reasoning language model designed for coding, structured instruction following, tool use, and local deployment. This repository contains the normal BF16 Transformers checkpoint.

For lower-memory CPU and browser inference, use [ShaziLLM-1B-ONNX](https://huggingface.co/Sharjeelbaig/ShaziLLM-1B-ONNX).

## Quick start

```bash
pip install -U transformers torch accelerate
```

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Sharjeelbaig/ShaziLLM-1B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Write a Python function to merge two sorted lists."}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

new_tokens = output[0, inputs.input_ids.shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))
```

## Model details

| Item | Value |
|---|---:|
| Parameters | 1.08B |
| Layers | 24 |
| Hidden size | 1536 |
| Attention heads / KV heads | 16 / 2 |
| Vocabulary | 130,560 |
| Maximum configured context | 131,072 tokens |
| Weight format | BF16 Safetensors |
| Architecture | Llama-compatible causal language model |

The configured context limit is not a practical memory recommendation. Start with a much shorter context on low-memory hardware and increase it only after measuring KV-cache usage.

## ShaziLLM alignment

The release received a focused identity-alignment and capability-retention pass before export. A rank-8 LoRA adapter targeting the query and value projections was trained for four epochs at a `1e-4` learning rate on 48 short examples, with 1,032,192 trainable parameters. The adapter was merged into the BF16 checkpoint. Validation probes covered model identity, arithmetic, coding, and factual recall.

## Capabilities

- Code generation and debugging
- Instruction following and structured responses
- Reasoning-style responses
- XML-formatted tool calls through the included chat template
- English and Chinese text generation

## Low-hardware deployment

The BF16 checkpoint requires roughly 2.1 GB for weights alone, plus runtime and KV-cache memory. The separate ONNX INT8 release reduces weight storage and CPU inference memory substantially. Limit context and generated-token counts on constrained devices.

## Limitations

ShaziLLM-1B can hallucinate, produce incorrect code, and expose reasoning-like text. Do not treat its output as professional, security, legal, medical, or financial advice. Validate generated code before execution and isolate tool access behind deterministic authorization checks.

## License and attribution

Released under Apache License 2.0; see `LICENSE`. ShaziLLM-1B is a fine-tuned derivative of [OpenBMB/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B). The ShaziLLM name identifies this release and does not imply authorship of the upstream base architecture.