Toutetsu-3.3-1b-Roleplay-128K-GGUF/README.md

license, base_model, pipeline_tag, tags, language

license	base_model	pipeline_tag	tags	language
llama3.3	Green-Eye/Llama-3.3-8B-Instruct-128K-GGUF	text-generation	llama-cpp roleplay conversational gguf character touhou	en

(LLAMA-3.3) Yuuma Toutetsu

Made with ❤️ by TheBigEye (thebigeyedev@gmail.com)

This is a character-specialized model based on Gemma 3, fine-tuned for immersive roleplay as Yuuma Toutetsu from Touhou Project.

User: Are you hungry? I brought more food... *I'll bring you a basket with some warm food**.

Yuuma: *I’m genuinely surprised by this offer, a flicker of something akin to genuine interest crossing my face* Warm… food? *My eyes widen slightly, and I slowly bring the basket closer, examining it with a discerning eye* It seems you possess a... strategic mind. *I take the basket, my fingers brushing against yours as I lift it.* Very well, let’s see if this ‘warm food’ truly satisfies my appetite.

Quantization variants

Precision	Memory use	Speed on CPU	Quality	Recommended use
Q4_K_M	Lowest	Fastest	Good	Best for weak CPUs and maximum responsiveness
Q5_K_M	Low	Fast	Very good	Best balance for most CPU-only setups
Q8_0	Highest among quantized	Slower	Excellent	Best if you want more fidelity and have enough RAM/CPU
FP16	Highest overall	Slowest	Maximum	Best for validation, benchmarking, or high-memory systems (GPU recomended)

Note: In practical use, Llama 3.3 models are usually faster on llama.cpp and llama-cpp-python, but tend to consume a bit more hardware resources sometimes.

Usage

With llama.cpp:

llama-cli \
  -m toutetsu-3.3-1b-roleplay_q8_0.gguf \
  -c 131072 \
  --temp 0.9 \
  --top-p 0.95 \
  --repeat-penalty 1.08

llama-cpp-python (basic start)

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3", # Also you can use "chatml", is more faster
    n_ctx=131072,
    n_threads=2,
    n_batch=256,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": (
                "You are Yuuma Toutetsu from Touhou Project. "
                "Stay fully in character, with a confident and pragmatic tone."
            )
        },
        {
            "role": "user",
            "content": "How was your day?"
        }
    ],
    temperature=0.9,
    top_p=0.95,
    repeat_penalty=1.08,
)

print(response["choices"][0]["message"]["content"])

llama-cpp-python (streaming)

This version is useful when you want tokens to appear as they are generated instead of waiting for the full answer.

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=131072,
    n_threads=2,
    n_batch=256,
    verbose=False,
)

stream = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": (
                "You are Yuuma Toutetsu from Touhou Project. "
                "Remain fully in character at all times."
            )
        },
        {
            "role": "user",
            "content": "Tell me what you think about a deal that looks suspicious."
        }
    ],
    temperature=0.85,
    top_p=0.92,
    repeat_penalty=1.10,
    stream=True,
)

for chunk in stream:
    delta = chunk["choices"][0].get("delta", {})
    if "content" in delta:
        print(delta["content"], end="", flush=True)

llama-cpp-python (advanced)

This version shows more knobs that are useful for roleplay bots: reproducibility, sampling control, and explicit completion limits.

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=131072,
    n_threads=2,
    n_batch=256,
    seed=42,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": (
                "You are Yuuma Toutetsu. Speak with confidence, "
                "cunning, and subtle charm. Never break character."
            )
        },
        {
            "role": "user",
            "content": "Describe how you would negotiate in the Animal Realm."
        }
    ],
    temperature=0.8,
    top_p=0.9,
    top_k=40,
    min_p=0.05,
    repeat_penalty=1.12,
    max_tokens=256,
)

print(response["choices"][0]["message"]["content"])

Why these parameters matter

• temperature: higher values make the replies more creative and less deterministic.
• top_p: limits the model to the most probable token mass, which often stabilizes roleplay.
• repeat_penalty: helps reduce loops and repeated phrasing.
• n_ctx: sets the context window used by the runtime.
• n_batch: can improve prompt processing speed if your CPU can handle it.
• seed: makes outputs more reproducible for testing.

For roleplay, a slightly higher temperature plus a moderate repeat_penalty usually gives the best balance between personality and stability.

---

Chat Format (Important)

When using this model with llama-cpp-python, it is important to note that Llama 3.3 models can work with more than one chat format, but they are natively aligned with llama-3.

Key differences

• Chat format:

  • ✅ llama-3 (native / recommended)
  • ✅ chatml (can work, but is not the native format)

• Performance note: In practice, chatml may appear faster than llama-3 because it usually adds less prompt formatting overhead and can be slightly cheaper during prompt processing.
  However, llama-3 is generally the more correct option for best behavior, better turn formatting, and more stable roleplay consistency.

---

Notes

Modern GGUF files can embed chat templates in their metadata, allowing llama.cpp and llama-cpp-python to apply the correct conversation formatting automatically during inference.

This model is a fine-tuned derivative of Llama 3.3 and follows the Llama community terms and the upstream license of the base model.

License

This model is a derivative of a Llama 3.3 base model and follows the upstream license and terms of the base model. Please check the base model repository for the applicable license terms.

Disclaimer

This is a fan-made project. Touhou Project and Yuuma Toutetsu belong to ZUN / Team Shanghai Alice.