145 lines
6.3 KiB
Markdown
145 lines
6.3 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
language:
|
||
- en
|
||
tags:
|
||
- code
|
||
- python
|
||
- coding-assistant
|
||
- lora
|
||
- naderu
|
||
- qwen2
|
||
---
|
||
|
||
# naderu-geek-py-0.5b
|
||
|
||
**A small coding assistant by [Naderu](https://naderu.com) — a BytesBrains Pte. Ltd. venture.**
|
||
|
||
`naderu-geek` is a compact, specialised coding model. It is a **portfolio / demonstration**
|
||
piece: a LoRA fine-tune of an open coding base that gives the model a recognisable
|
||
`naderu-geek` identity and a clean, commented Python style. It demonstrates Naderu's
|
||
model-engineering pipeline — data → training → evaluation → release — end to end, on a
|
||
deliberately small footprint.
|
||
|
||
> **Status: released (v0.1.0).** Trained 2026-07-14 (LoRA fine-tune, merged) and passed its
|
||
> qualitative smoke eval — see **Evaluation** below. Nothing here is a capability claim — see
|
||
> **Limitations**.
|
||
|
||
> This is an honest demonstration piece, **not** a state-of-the-art model. Its coding
|
||
> ability is essentially that of its base; the fine-tune adds identity and style, not new
|
||
> capability. See **Limitations**.
|
||
|
||
## Provenance
|
||
|
||
- **Fine-tuned from:** [`Qwen/Qwen2.5-Coder-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) (Apache-2.0)
|
||
- **Method:** LoRA (r=16, α=32; ~8.8M trainable params, 1.75%) on attention + MLP projections, adapter merged into the base
|
||
- **Training data:** a small, Naderu-authored instruction set (39 examples: identity + idiomatic Python) — license-clean, included in our repo
|
||
- **Precision:** trained in fp32; merged weights distributed in **bf16** (matching the base's format)
|
||
- **License:** Apache-2.0 (inherits the base model's terms)
|
||
|
||
## Intended use
|
||
|
||
- A lightweight Python coding helper: small functions, snippets, explanations.
|
||
- A reference example of a Naderu specialised-model release (card + provenance + eval).
|
||
|
||
**Out of scope:** production code generation at scale, non-Python languages, security-
|
||
sensitive code, or any use where correctness must be guaranteed without review.
|
||
|
||
## How to use
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_id = "bytesbrains/naderu-geek-py-0.5b"
|
||
tok = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(model_id)
|
||
|
||
messages = [
|
||
{"role": "system", "content": "You are naderu-geek, a focused coding assistant by Naderu (naderu.com). You write clean, correct, well-commented code and explain briefly."},
|
||
{"role": "user", "content": "Write a Python function to check if a number is prime."},
|
||
]
|
||
enc = tok.apply_chat_template(
|
||
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
|
||
)
|
||
out = model.generate(**enc, max_new_tokens=200, do_sample=False)
|
||
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
|
||
```
|
||
|
||
The model is chat-templated (Qwen2 template). The system prompt above is the one it was
|
||
trained with; keeping it yields the most on-style responses.
|
||
|
||
## Evaluation
|
||
|
||
<!-- EVAL:START -->
|
||
**Suite:** `eval/suites/naderu-geek/run_eval.py` (qualitative smoke, greedy decode) ·
|
||
**Run:** 2026-07-14 · **Result: PASS**
|
||
|
||
| Check | Outcome |
|
||
|-------|---------|
|
||
| Identity recognised (`Who are you?` → names `naderu-geek`) | ✅ yes |
|
||
| Coherent, runnable Python (3 in-suite coding prompts) | ✅ 3/3 |
|
||
|
||
Observations (honest):
|
||
|
||
- **Identity imprinted and robust.** It reliably self-identifies as `naderu-geek` by Naderu,
|
||
and on a held-out prompt (*"Are you ChatGPT or made by OpenAI?"*) it correctly denies and
|
||
states its open-base provenance — i.e. the identity generalises beyond the exact training phrasings.
|
||
- **Clean Python style generalises.** On held-out tasks not in the training set (mean of a list,
|
||
nth triangular number) it produced correct, type-hinted, docstring'd functions in the same
|
||
house style — so the fine-tune transfers *style*, not just memorised answers.
|
||
- **In line with the base's capability.** In-suite answers closely track the curated exemplars
|
||
(a 39-example set, low final train loss). This is the expected behaviour of a small identity/
|
||
style imprint and **not** a capability claim — see **Limitations**.
|
||
|
||
Reproduce: `python training/train_lora.py && NG_MODEL=./naderu-geek-py-0.5b python eval/suites/naderu-geek/run_eval.py`
|
||
<!-- EVAL:END -->
|
||
|
||
This is a qualitative smoke check (identity recognised + coherent Python), not a
|
||
leaderboard score. For real releases, Naderu attaches reproducible benchmark results
|
||
tied to a versioned eval suite.
|
||
|
||
## Limitations & risks
|
||
|
||
- Tiny base (0.5B) + tiny fine-tune → limited reasoning; can produce incorrect or
|
||
insecure code. **Always review generated code.**
|
||
- The fine-tune changes identity/style far more than capability.
|
||
- English + Python focus; other languages are best-effort from the base.
|
||
|
||
## About Naderu
|
||
|
||
[**Naderu**](https://naderu.com) is an AI-models company — a venture of **BytesBrains Pte. Ltd.**
|
||
We do three things, and only these:
|
||
|
||
- **Train** — turn foundation models into specialised ones (fine-tune, LoRA, distill, quantize).
|
||
- **Release** — publish specialised models with model cards, provenance, and clear licensing.
|
||
- **Serve** — the engineering around models (customization, deployment, operations).
|
||
|
||
Every Naderu model states its upstream foundation model and licence plainly, and every capability
|
||
claim is backed by a real evaluation — never vibes. `naderu-geek-py-0.5b` is our **first public
|
||
portfolio release**: small on purpose, honest about its scope, and reproducible end to end.
|
||
|
||
## Links
|
||
|
||
- 🌐 Website — [naderu.com](https://naderu.com)
|
||
- 🧩 Model family — `naderu-geek` (compact, specialised coding models)
|
||
- 🧱 Base model — [`Qwen/Qwen2.5-Coder-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct)
|
||
- 🪪 License — [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
||
- 🏷️ Version — v0.1.0 (released 2026-07-14)
|
||
|
||
## Citation / attribution
|
||
|
||
Built on Qwen2.5-Coder (Apache-2.0). Fine-tuned and released by **Naderu** (BytesBrains Pte. Ltd.).
|
||
|
||
```bibtex
|
||
@misc{naderu_geek_py_0_5b_2026,
|
||
title = {naderu-geek-py-0.5b},
|
||
author = {Naderu (BytesBrains Pte. Ltd.)},
|
||
year = {2026},
|
||
howpublished = {\url{https://huggingface.co/bytesbrains/naderu-geek-py-0.5b}},
|
||
note = {LoRA fine-tune of Qwen/Qwen2.5-Coder-0.5B-Instruct, Apache-2.0}
|
||
}
|
||
```
|