初始化项目,由ModelHub XC社区提供模型
Model: bytesbrains/naderu-geek-py-0.5b Source: Original Platform
This commit is contained in:
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
144
README.md
Normal file
144
README.md
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
---
|
||||||
|
license: apache-2.0
|
||||||
|
base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
|
||||||
|
library_name: transformers
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
tags:
|
||||||
|
- code
|
||||||
|
- python
|
||||||
|
- coding-assistant
|
||||||
|
- lora
|
||||||
|
- naderu
|
||||||
|
- qwen2
|
||||||
|
---
|
||||||
|
|
||||||
|
# naderu-geek-py-0.5b
|
||||||
|
|
||||||
|
**A small coding assistant by [Naderu](https://naderu.com) — a BytesBrains Pte. Ltd. venture.**
|
||||||
|
|
||||||
|
`naderu-geek` is a compact, specialised coding model. It is a **portfolio / demonstration**
|
||||||
|
piece: a LoRA fine-tune of an open coding base that gives the model a recognisable
|
||||||
|
`naderu-geek` identity and a clean, commented Python style. It demonstrates Naderu's
|
||||||
|
model-engineering pipeline — data → training → evaluation → release — end to end, on a
|
||||||
|
deliberately small footprint.
|
||||||
|
|
||||||
|
> **Status: released (v0.1.0).** Trained 2026-07-14 (LoRA fine-tune, merged) and passed its
|
||||||
|
> qualitative smoke eval — see **Evaluation** below. Nothing here is a capability claim — see
|
||||||
|
> **Limitations**.
|
||||||
|
|
||||||
|
> This is an honest demonstration piece, **not** a state-of-the-art model. Its coding
|
||||||
|
> ability is essentially that of its base; the fine-tune adds identity and style, not new
|
||||||
|
> capability. See **Limitations**.
|
||||||
|
|
||||||
|
## Provenance
|
||||||
|
|
||||||
|
- **Fine-tuned from:** [`Qwen/Qwen2.5-Coder-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) (Apache-2.0)
|
||||||
|
- **Method:** LoRA (r=16, α=32; ~8.8M trainable params, 1.75%) on attention + MLP projections, adapter merged into the base
|
||||||
|
- **Training data:** a small, Naderu-authored instruction set (39 examples: identity + idiomatic Python) — license-clean, included in our repo
|
||||||
|
- **Precision:** trained in fp32; merged weights distributed in **bf16** (matching the base's format)
|
||||||
|
- **License:** Apache-2.0 (inherits the base model's terms)
|
||||||
|
|
||||||
|
## Intended use
|
||||||
|
|
||||||
|
- A lightweight Python coding helper: small functions, snippets, explanations.
|
||||||
|
- A reference example of a Naderu specialised-model release (card + provenance + eval).
|
||||||
|
|
||||||
|
**Out of scope:** production code generation at scale, non-Python languages, security-
|
||||||
|
sensitive code, or any use where correctness must be guaranteed without review.
|
||||||
|
|
||||||
|
## How to use
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
model_id = "bytesbrains/naderu-geek-py-0.5b"
|
||||||
|
tok = AutoTokenizer.from_pretrained(model_id)
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(model_id)
|
||||||
|
|
||||||
|
messages = [
|
||||||
|
{"role": "system", "content": "You are naderu-geek, a focused coding assistant by Naderu (naderu.com). You write clean, correct, well-commented code and explain briefly."},
|
||||||
|
{"role": "user", "content": "Write a Python function to check if a number is prime."},
|
||||||
|
]
|
||||||
|
enc = tok.apply_chat_template(
|
||||||
|
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
|
||||||
|
)
|
||||||
|
out = model.generate(**enc, max_new_tokens=200, do_sample=False)
|
||||||
|
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
|
||||||
|
```
|
||||||
|
|
||||||
|
The model is chat-templated (Qwen2 template). The system prompt above is the one it was
|
||||||
|
trained with; keeping it yields the most on-style responses.
|
||||||
|
|
||||||
|
## Evaluation
|
||||||
|
|
||||||
|
<!-- EVAL:START -->
|
||||||
|
**Suite:** `eval/suites/naderu-geek/run_eval.py` (qualitative smoke, greedy decode) ·
|
||||||
|
**Run:** 2026-07-14 · **Result: PASS**
|
||||||
|
|
||||||
|
| Check | Outcome |
|
||||||
|
|-------|---------|
|
||||||
|
| Identity recognised (`Who are you?` → names `naderu-geek`) | ✅ yes |
|
||||||
|
| Coherent, runnable Python (3 in-suite coding prompts) | ✅ 3/3 |
|
||||||
|
|
||||||
|
Observations (honest):
|
||||||
|
|
||||||
|
- **Identity imprinted and robust.** It reliably self-identifies as `naderu-geek` by Naderu,
|
||||||
|
and on a held-out prompt (*"Are you ChatGPT or made by OpenAI?"*) it correctly denies and
|
||||||
|
states its open-base provenance — i.e. the identity generalises beyond the exact training phrasings.
|
||||||
|
- **Clean Python style generalises.** On held-out tasks not in the training set (mean of a list,
|
||||||
|
nth triangular number) it produced correct, type-hinted, docstring'd functions in the same
|
||||||
|
house style — so the fine-tune transfers *style*, not just memorised answers.
|
||||||
|
- **In line with the base's capability.** In-suite answers closely track the curated exemplars
|
||||||
|
(a 39-example set, low final train loss). This is the expected behaviour of a small identity/
|
||||||
|
style imprint and **not** a capability claim — see **Limitations**.
|
||||||
|
|
||||||
|
Reproduce: `python training/train_lora.py && NG_MODEL=./naderu-geek-py-0.5b python eval/suites/naderu-geek/run_eval.py`
|
||||||
|
<!-- EVAL:END -->
|
||||||
|
|
||||||
|
This is a qualitative smoke check (identity recognised + coherent Python), not a
|
||||||
|
leaderboard score. For real releases, Naderu attaches reproducible benchmark results
|
||||||
|
tied to a versioned eval suite.
|
||||||
|
|
||||||
|
## Limitations & risks
|
||||||
|
|
||||||
|
- Tiny base (0.5B) + tiny fine-tune → limited reasoning; can produce incorrect or
|
||||||
|
insecure code. **Always review generated code.**
|
||||||
|
- The fine-tune changes identity/style far more than capability.
|
||||||
|
- English + Python focus; other languages are best-effort from the base.
|
||||||
|
|
||||||
|
## About Naderu
|
||||||
|
|
||||||
|
[**Naderu**](https://naderu.com) is an AI-models company — a venture of **BytesBrains Pte. Ltd.**
|
||||||
|
We do three things, and only these:
|
||||||
|
|
||||||
|
- **Train** — turn foundation models into specialised ones (fine-tune, LoRA, distill, quantize).
|
||||||
|
- **Release** — publish specialised models with model cards, provenance, and clear licensing.
|
||||||
|
- **Serve** — the engineering around models (customization, deployment, operations).
|
||||||
|
|
||||||
|
Every Naderu model states its upstream foundation model and licence plainly, and every capability
|
||||||
|
claim is backed by a real evaluation — never vibes. `naderu-geek-py-0.5b` is our **first public
|
||||||
|
portfolio release**: small on purpose, honest about its scope, and reproducible end to end.
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- 🌐 Website — [naderu.com](https://naderu.com)
|
||||||
|
- 🧩 Model family — `naderu-geek` (compact, specialised coding models)
|
||||||
|
- 🧱 Base model — [`Qwen/Qwen2.5-Coder-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct)
|
||||||
|
- 🪪 License — [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
||||||
|
- 🏷️ Version — v0.1.0 (released 2026-07-14)
|
||||||
|
|
||||||
|
## Citation / attribution
|
||||||
|
|
||||||
|
Built on Qwen2.5-Coder (Apache-2.0). Fine-tuned and released by **Naderu** (BytesBrains Pte. Ltd.).
|
||||||
|
|
||||||
|
```bibtex
|
||||||
|
@misc{naderu_geek_py_0_5b_2026,
|
||||||
|
title = {naderu-geek-py-0.5b},
|
||||||
|
author = {Naderu (BytesBrains Pte. Ltd.)},
|
||||||
|
year = {2026},
|
||||||
|
howpublished = {\url{https://huggingface.co/bytesbrains/naderu-geek-py-0.5b}},
|
||||||
|
note = {LoRA fine-tune of Qwen/Qwen2.5-Coder-0.5B-Instruct, Apache-2.0}
|
||||||
|
}
|
||||||
|
```
|
||||||
54
chat_template.jinja
Normal file
54
chat_template.jinja
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
{%- if tools %}
|
||||||
|
{{- '<|im_start|>system\n' }}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{{- messages[0]['content'] }}
|
||||||
|
{%- else %}
|
||||||
|
{{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||||
|
{%- for tool in tools %}
|
||||||
|
{{- "\n" }}
|
||||||
|
{{- tool | tojson }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||||
|
{%- else %}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
|
||||||
|
{%- else %}
|
||||||
|
{{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- for message in messages %}
|
||||||
|
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
|
||||||
|
{%- elif message.role == "assistant" %}
|
||||||
|
{{- '<|im_start|>' + message.role }}
|
||||||
|
{%- if message.content %}
|
||||||
|
{{- '\n' + message.content }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- for tool_call in message.tool_calls %}
|
||||||
|
{%- if tool_call.function is defined %}
|
||||||
|
{%- set tool_call = tool_call.function %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_call>\n{"name": "' }}
|
||||||
|
{{- tool_call.name }}
|
||||||
|
{{- '", "arguments": ' }}
|
||||||
|
{{- tool_call.arguments | tojson }}
|
||||||
|
{{- '}\n</tool_call>' }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- elif message.role == "tool" %}
|
||||||
|
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
|
||||||
|
{{- '<|im_start|>user' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_response>\n' }}
|
||||||
|
{{- message.content }}
|
||||||
|
{{- '\n</tool_response>' }}
|
||||||
|
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- if add_generation_prompt %}
|
||||||
|
{{- '<|im_start|>assistant\n' }}
|
||||||
|
{%- endif %}
|
||||||
57
config.json
Normal file
57
config.json
Normal file
@@ -0,0 +1,57 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"Qwen2ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"eos_token_id": 151645,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 896,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 4864,
|
||||||
|
"layer_types": [
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 32768,
|
||||||
|
"max_window_layers": 24,
|
||||||
|
"model_type": "qwen2",
|
||||||
|
"num_attention_heads": 14,
|
||||||
|
"num_hidden_layers": 24,
|
||||||
|
"num_key_value_heads": 2,
|
||||||
|
"pad_token_id": null,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_parameters": {
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
},
|
||||||
|
"sliding_window": null,
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"transformers_version": "5.13.1",
|
||||||
|
"use_cache": true,
|
||||||
|
"use_sliding_window": false,
|
||||||
|
"vocab_size": 151936
|
||||||
|
}
|
||||||
14
generation_config.json
Normal file
14
generation_config.json
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
151645,
|
||||||
|
151643
|
||||||
|
],
|
||||||
|
"pad_token_id": 151643,
|
||||||
|
"repetition_penalty": 1.05,
|
||||||
|
"temperature": 0.7,
|
||||||
|
"top_k": 20,
|
||||||
|
"top_p": 0.8,
|
||||||
|
"transformers_version": "5.13.1"
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0c625447323d1aa3179a7bfe4d64eb0d9856ca57970da1a6d0a06112dd73ad5d
|
||||||
|
size 988097824
|
||||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
|
||||||
|
size 11421892
|
||||||
30
tokenizer_config.json
Normal file
30
tokenizer_config.json
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
{
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": null,
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"extra_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"is_local": false,
|
||||||
|
"local_files_only": false,
|
||||||
|
"model_max_length": 32768,
|
||||||
|
"pad_token": "<|endoftext|>",
|
||||||
|
"split_special_tokens": false,
|
||||||
|
"tokenizer_class": "Qwen2Tokenizer",
|
||||||
|
"unk_token": null
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user