86 lines
3.5 KiB
Markdown
86 lines
3.5 KiB
Markdown
---
|
|
base_model: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
|
|
datasets:
|
|
- liumindmind/NekoQA-10K
|
|
tags:
|
|
- text-generation-inference
|
|
- transformers
|
|
- unsloth
|
|
- qwen3
|
|
- catgirl
|
|
- persona
|
|
- roleplay
|
|
license: apache-2.0
|
|
language:
|
|
- zh
|
|
- en
|
|
---
|
|
|
|
# neko-qwen3-4b 🐾
|
|
|
|
- **Developed by:** Laow0v0
|
|
- **License:** apache-2.0
|
|
- **Finetuned from model:** [unsloth/qwen3-4b-instruct-2507](https://huggingface.co/unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit) (bnb-4bit), merged to 16-bit
|
|
- **Training data:** [liumindmind/NekoQA-10K](https://huggingface.co/datasets/liumindmind/NekoQA-10K)
|
|
- **Demo:** [Laow0v0/neko-qwen3-4b-demo](https://huggingface.co/spaces/Laow0v0/neko-qwen3-4b-demo)
|
|
|
|
A catgirl-persona (猫娘) finetune of Qwen3-4B-Instruct-2507.
|
|
|
|
## Training data
|
|
|
|
Finetuned on [**NekoQA-10K**](https://huggingface.co/datasets/liumindmind/NekoQA-10K) by
|
|
[liumindmind](https://huggingface.co/liumindmind) — 10,000 single-turn QA pairs written in a
|
|
consistent catgirl persona. Per the dataset card, every answer follows the same conventions:
|
|
|
|
- addresses the user as **主人** ("master"),
|
|
- ends sentences with characteristic verbal tics (**喵~**, **no desu**, **的说喵**),
|
|
- keeps a cute, affectionate, 二次元 register.
|
|
|
|
The data is **primarily Chinese**, with some mixed Chinese-English. It was built from a mix of
|
|
original hand-written pairs, public forum content (e.g. 弱智吧) rewritten by an LLM for
|
|
consistency and safety, and ~900 rows rewritten from existing catgirl QA sets. Answers were
|
|
mostly LLM-generated and human-filtered. The dataset is Apache-2.0.
|
|
|
|
The rows are `instruction` / `output` pairs with no system prompt, so the persona is intended to
|
|
be baked in rather than prompted.
|
|
|
|
## Intended use
|
|
|
|
Style transfer / persona-consistency research, roleplay and companionship-style chat. As the
|
|
dataset card notes, this kind of data optimises for tone, **not** factual rigour — the dataset
|
|
authors explicitly warn that it may make a model "过于可爱" (too cute) on serious tasks, and ask
|
|
that it not be treated as a substitute for real human relationships.
|
|
|
|
## Limitations
|
|
|
|
- Persona adherence is inconsistent. In Chinese the model often answers in a plain-assistant
|
|
voice and may still self-identify as 通义千问 (the base model's identity) rather than as a
|
|
catgirl; an explicit system prompt is currently doing most of the persona work.
|
|
- Generation scaffolding: replies frequently open with an unterminated `<think>`, a `<tool_call>`
|
|
pair, or a literal `(Dialogue begins)` line before the real answer. In this repo's
|
|
`tokenizer.json` these markers are added tokens flagged `special: false`, so
|
|
`skip_special_tokens=True` does **not** strip them — downstream code has to remove them (see
|
|
the demo Space's `app.py`). Note they cannot be removed via `suppress_tokens`: blocking them
|
|
at sampling time also blocks the good continuation that follows.
|
|
- The model occasionally emits `<|im_start|>user …`, opening a fake new turn instead of
|
|
answering.
|
|
- Not suitable for tasks requiring factual reliability.
|
|
|
|
## Citation
|
|
|
|
Please cite the dataset if you build on this work:
|
|
|
|
```bibtex
|
|
@article{nekoqa2025,
|
|
title={NekoQA-10K: A Catgirl Dialogue Dataset and NekoBench Evaluation},
|
|
author={MindsRiverPonder},
|
|
journal={ZHIHU preprint ZHIHU:2508.22},
|
|
year={2025}
|
|
}
|
|
```
|
|
|
|
This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and
|
|
Huggingface's TRL library.
|
|
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|