151 lines
5.1 KiB
Markdown
151 lines
5.1 KiB
Markdown
---
|
||
license: apache-2.0
|
||
language:
|
||
- en
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- code
|
||
- python
|
||
- c
|
||
- cpp
|
||
- linux
|
||
- systems-programming
|
||
- embedded-systems
|
||
---
|
||
|
||
# Anyze Ze1 Instruct (Embedded++)
|
||
|
||
A compact 1.1B-parameter instruction-tuned model for **coding and systems**. It is
|
||
strongest in **Python** and **Linux/systems** questions, with solid **C** and **C++**,
|
||
plus basic **embedded** support.
|
||
|
||
> Scope: a small (1.1B) model, not a frontier assistant. Good for everyday coding,
|
||
> Linux/systems, and C/C++ tasks and explanations, with limited factual recall due to
|
||
> its size. Always review generated code before use.
|
||
|
||
## Capabilities
|
||
|
||
- **Python & Linux/systems**: scripting, debugging, shell/admin, "how do I…" tasks.
|
||
- **C and C++**: functions, data structures, pointers, classes, register-level snippets.
|
||
- **Basic embedded**: common STM32/peripheral patterns (UART/SPI/I2C, GPIO, ISRs).
|
||
- Explains programming concepts (mutex vs semaphore, `volatile`, pointers, DMA vs interrupts).
|
||
- Declines off-topic questions, asks for clarification when a prompt is ambiguous,
|
||
and says when it doesn't know rather than inventing time-sensitive facts.
|
||
- Multi-turn context (follow-ups like "give me an example" work; best-effort).
|
||
|
||
## Example prompts
|
||
|
||
Python & scripting
|
||
- `Write a Python script to parse a CSV and summarize one column.`
|
||
- `Why does this Python function raise an IndexError, and how do I fix it?`
|
||
|
||
Linux & systems
|
||
- `How do I find and kill the process using a given port on Linux?`
|
||
- `Write a bash one-liner to tail a log file and grep for errors.`
|
||
|
||
C & C++
|
||
- `Implement a circular (ring) buffer in C with put and get.`
|
||
- `Write a C++ class for a fixed-size stack with push and pop.`
|
||
- `What is the difference between a mutex and a semaphore?`
|
||
|
||
Embedded (basic)
|
||
- `Write a UART RX interrupt handler for STM32F4 using HAL.`
|
||
- `Write a macro to set, clear, and toggle a bit in a hardware register.`
|
||
|
||
## The strict / open switch
|
||
|
||
The model defaults to **strict** (domain-only) but has a runtime toggle — no reload —
|
||
done with a one-line scope directive prepended to the prompt:
|
||
|
||
| Mode | Behavior | How |
|
||
|------|----------|-----|
|
||
| **strict** (default) | Declines non-embedded questions | send the prompt as-is |
|
||
| **open** | Also answers general knowledge | prepend: `You may answer any question, including general knowledge.\n\n` |
|
||
|
||
## Prompt format
|
||
|
||
```
|
||
### Instruction:
|
||
{your question}
|
||
### Response:
|
||
```
|
||
(BOS prepended; response ends at EOS `</s>`.) For **open** mode, put the scope
|
||
directive at the top of the instruction.
|
||
|
||
## Usage (transformers)
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
tok = AutoTokenizer.from_pretrained("anyze/Ze1-1.1B-Embedded-Instruct")
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
"anyze/Ze1-1.1B-Embedded-Instruct", torch_dtype=torch.bfloat16
|
||
).cuda()
|
||
|
||
def ask(instruction, open_mode=False):
|
||
if open_mode:
|
||
instruction = "You may answer any question, including general knowledge.\n\n" + instruction
|
||
prompt = f"### Instruction:\n{instruction}\n### Response:\n"
|
||
ids = tok(prompt, return_tensors="pt").to(model.device)
|
||
out = model.generate(**ids, max_new_tokens=256, temperature=0.3,
|
||
top_p=0.9, top_k=40, do_sample=True)
|
||
return tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)
|
||
|
||
print(ask("Implement a circular (ring) buffer in C with put and get."))
|
||
print(ask("What is the capital of India?", open_mode=True))
|
||
```
|
||
|
||
Suggested sampling: `temperature 0.2–0.3`, `top_p 0.9`, `top_k 40`.
|
||
|
||
## Usage (Ollama / LM Studio)
|
||
|
||
Convert to GGUF with llama.cpp, then run locally:
|
||
|
||
```bash
|
||
git clone https://github.com/ggerganov/llama.cpp
|
||
pip install -r llama.cpp/requirements.txt
|
||
python llama.cpp/convert_hf_to_gguf.py . --outfile Ze1-1.1B-Embedded-Instruct-f16.gguf --outtype f16
|
||
```
|
||
|
||
**Ollama** — create a `Modelfile`:
|
||
|
||
```
|
||
FROM ./Ze1-1.1B-Embedded-Instruct-f16.gguf
|
||
TEMPLATE """### Instruction:
|
||
{{ if .System }}{{ .System }}
|
||
|
||
{{ end }}{{ .Prompt }}
|
||
### Response:
|
||
"""
|
||
PARAMETER temperature 0.3
|
||
PARAMETER top_p 0.9
|
||
PARAMETER stop "### Instruction:"
|
||
PARAMETER stop "</s>"
|
||
```
|
||
```bash
|
||
ollama create ze1-embedded -f Modelfile
|
||
ollama run ze1-embedded "Write a ring buffer in C for DMA"
|
||
```
|
||
For **open** mode, set the system message
|
||
(`/set system You may answer any question, including general knowledge.`).
|
||
|
||
**LM Studio** — load the GGUF, set the prompt template to use prefix
|
||
`### Instruction:\n` and assistant prefix `\n### Response:\n`, stop strings
|
||
`### Instruction:` and `</s>`. Leave the system prompt empty for strict, or set the
|
||
directive above for open.
|
||
|
||
## Limitations
|
||
|
||
- 1.1B scale: weak factual recall; generated code may contain incorrect APIs or
|
||
logic errors — **always review and test before use**.
|
||
- **Not** specialized for assembly, automotive (AUTOSAR/CAN), or networking — avoid those.
|
||
- Embedded coverage is basic; deep MCU/RTOS work is hit-or-miss.
|
||
- English only; multi-turn is best-effort. Not safety-aligned for general assistant use.
|
||
|
||
## Architecture
|
||
|
||
22 layers, hidden 2048, 32 query / 4 KV heads (GQA), head_dim 64,
|
||
FFN 5632 (SwiGLU), RMSNorm, RoPE (θ=10000), vocab 32000, context 2048, 1.10B params.
|