neko-qwen3-4b/README.md

---
base_model: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
datasets:
- liumindmind/NekoQA-10K
tags:
- text-generation-inference
- transformers
- unsloth
- qwen3
- catgirl
- persona
- roleplay
license: apache-2.0
language:
- zh
- en
---

# neko-qwen3-4b 🐾

- **Developed by:** Laow0v0
- **License:** apache-2.0
- **Finetuned from model:** [unsloth/qwen3-4b-instruct-2507](https://huggingface.co/unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit) (bnb-4bit), merged to 16-bit
- **Training data:** [liumindmind/NekoQA-10K](https://huggingface.co/datasets/liumindmind/NekoQA-10K)
- **Demo:** [Laow0v0/neko-qwen3-4b-demo](https://huggingface.co/spaces/Laow0v0/neko-qwen3-4b-demo)

A catgirl-persona (猫娘) finetune of Qwen3-4B-Instruct-2507.

## Training data

Finetuned on [**NekoQA-10K**](https://huggingface.co/datasets/liumindmind/NekoQA-10K) by
[liumindmind](https://huggingface.co/liumindmind) — 10,000 single-turn QA pairs written in a
consistent catgirl persona. Per the dataset card, every answer follows the same conventions:

- addresses the user as **主人** ("master"),
- ends sentences with characteristic verbal tics (**喵~**, **no desu**, **的说喵**),
- keeps a cute, affectionate, 二次元 register.

The data is **primarily Chinese**, with some mixed Chinese-English. It was built from a mix of
original hand-written pairs, public forum content (e.g. 弱智吧) rewritten by an LLM for
consistency and safety, and ~900 rows rewritten from existing catgirl QA sets. Answers were
mostly LLM-generated and human-filtered. The dataset is Apache-2.0.

The rows are `instruction` / `output` pairs with no system prompt, so the persona is intended to
be baked in rather than prompted.

## Intended use

Style transfer / persona-consistency research, roleplay and companionship-style chat. As the
dataset card notes, this kind of data optimises for tone, **not** factual rigour — the dataset
authors explicitly warn that it may make a model "过于可爱" (too cute) on serious tasks, and ask
that it not be treated as a substitute for real human relationships.

## Limitations

- Persona adherence is inconsistent. In Chinese the model often answers in a plain-assistant
  voice and may still self-identify as 通义千问 (the base model's identity) rather than as a
  catgirl; an explicit system prompt is currently doing most of the persona work.
- Generation scaffolding: replies frequently open with an unterminated `<think>`, a `<tool_call>`
  pair, or a literal `(Dialogue begins)` line before the real answer. In this repo's
  `tokenizer.json` these markers are added tokens flagged `special: false`, so
  `skip_special_tokens=True` does **not** strip them — downstream code has to remove them (see
  the demo Space's `app.py`). Note they cannot be removed via `suppress_tokens`: blocking them
  at sampling time also blocks the good continuation that follows.
- The model occasionally emits `<|im_start|>user …`, opening a fake new turn instead of
  answering.
- Not suitable for tasks requiring factual reliability.

## Citation

Please cite the dataset if you build on this work:

```bibtex
@article{nekoqa2025,
  title={NekoQA-10K: A Catgirl Dialogue Dataset and NekoBench Evaluation},
  author={MindsRiverPonder},
  journal={ZHIHU preprint ZHIHU:2508.22},
  year={2025}
}
```

This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and
Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)