81 lines
3.5 KiB
Markdown
81 lines
3.5 KiB
Markdown
|
---
|
|||
|
|
language: en
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: meta-llama/Llama-3.2-3B-Instruct
|
|||
|
|
tags:
|
|||
|
|
- unsloth
|
|||
|
|
- text-generation-inference
|
|||
|
|
- conversational
|
|||
|
|
- gguf
|
|||
|
|
- llama
|
|||
|
|
datasets:
|
|||
|
|
- KiwiMate/KiwiMate-Mini-training
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# KiwiMate-Mini-Preview
|
|||
|
|
|
|||
|
|
**KiwiMate-Mini-Preview** is a lightweight, New Zealand–flavoured conversational language model, fine-tuned for the [KiwiMate](https://kiwimate.net) AI companion app. It is the smallest model in the KiwiMate model family and is designed for fast, low-cost inference on the app's free and lower-tier subscription plans.
|
|||
|
|
|
|||
|
|
> ⚠️ **Preview status:** This is a prototype release. The name reflects its preview status — expect breaking changes, retraining, and behavioural shifts before a stable v1 release.
|
|||
|
|
|
|||
|
|
## Model Details
|
|||
|
|
|
|||
|
|
| | |
|
|||
|
|
|---|---|
|
|||
|
|
| **Developed by** | KiwiMate / KyleCodeKiwi |
|
|||
|
|
| **Base model** | Llama 3.2 3B Instruct |
|
|||
|
|
| **Architecture** | Llama |
|
|||
|
|
| **Parameters** | ~3.21B |
|
|||
|
|
| **Fine-tuning framework** | [Unsloth](https://github.com/unslothai/unsloth) |
|
|||
|
|
| **License** | Apache 2.0 |
|
|||
|
|
| **Languages** | English (with New Zealand English and Te Reo Māori vocabulary coverage) |
|
|||
|
|
| **Model class** | `AutoModelForCausalLM` |
|
|||
|
|
|
|||
|
|
## Intended Use
|
|||
|
|
|
|||
|
|
KiwiMate-Mini-Preview is intended as the default conversational backend for the KiwiMate app, providing:
|
|||
|
|
|
|||
|
|
- General-purpose chat and assistant-style conversation
|
|||
|
|
- New Zealand cultural and "Kiwi" context awareness (slang, geography, fun facts)
|
|||
|
|
- Light Te Reo Māori vocabulary recognition and use
|
|||
|
|
- Lore and knowledge specific to the KiwiMate app itself ("KiwiMate Origin" data)
|
|||
|
|
- Lightweight knowledge support for in-app mini-games
|
|||
|
|
|
|||
|
|
It is **not** intended for high-stakes, medical, legal, or financial advice, and should not be relied on as an authoritative source on Māori language or tikanga — for genuinely sensitive Te Reo or cultural content, defer to community-governed resources.
|
|||
|
|
|
|||
|
|
## Training Data
|
|||
|
|
|
|||
|
|
Fine-tuned on the [`KiwiMate/KiwiMate-Mini-training`](https://huggingface.co/datasets/KiwiMate/KiwiMate-Mini-training) dataset, organised into categories including:
|
|||
|
|
|
|||
|
|
- NZ English
|
|||
|
|
- Te Reo Māori
|
|||
|
|
- KiwiMate Origin (app-specific lore/identity)
|
|||
|
|
- NZ Fun Facts
|
|||
|
|
- MiniGame Knowledge
|
|||
|
|
|
|||
|
|
## Files & Quantizations
|
|||
|
|
|
|||
|
|
Distributed as `safetensors` (full precision) and GGUF quantizations for efficient local/edge inference:
|
|||
|
|
|
|||
|
|
| Format | Use case |
|
|||
|
|
|---|---|
|
|||
|
|
| F16 | Highest fidelity, largest size |
|
|||
|
|
| Q6_K | Near-lossless, smaller footprint |
|
|||
|
|
| Q4_K_M | Balanced quality/size — recommended default for on-device use |
|
|||
|
|
| Q2_K_L | Smallest footprint, lowest fidelity |
|
|||
|
|
|
|||
|
|
## Deployment
|
|||
|
|
|
|||
|
|
Served in production via a Hugging Face Inference Endpoint on a T4 GPU with scale-to-zero, fronted by a Supabase Edge Function (OpenAI-compatible proxy) that routes KiwiMate app traffic to this and other KiwiMate model endpoints behind a single API.
|
|||
|
|
|
|||
|
|
### Known Limitations
|
|||
|
|
|
|||
|
|
- A server-side mitigation is in place for an occasional role-bleed / over-generation issue (the model sometimes continuing past `<|eot_id|>`), handled via stop-sequence aliases and trimming at the proxy layer.
|
|||
|
|
- The long-term fix — adding `<|eot_id|>` (token ID 128009) properly to the training loss and `generation_config.json` — is planned for a future retraining pass rather than this preview.
|
|||
|
|
- As a 3B-parameter model, reasoning depth and factual reliability are limited compared to larger models; it is tuned for speed and personality over raw capability.
|
|||
|
|
|
|||
|
|
## License
|
|||
|
|
|
|||
|
|
Released under the Apache 2.0 license, consistent with the open weights commitment for the KiwiMate model family.
|