81 lines
3.5 KiB
Markdown
81 lines
3.5 KiB
Markdown
---
|
||
language: en
|
||
license: apache-2.0
|
||
base_model: meta-llama/Llama-3.2-3B-Instruct
|
||
tags:
|
||
- unsloth
|
||
- text-generation-inference
|
||
- conversational
|
||
- gguf
|
||
- llama
|
||
datasets:
|
||
- KiwiMate/KiwiMate-Mini-training
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# KiwiMate-Mini-Preview
|
||
|
||
**KiwiMate-Mini-Preview** is a lightweight, New Zealand–flavoured conversational language model, fine-tuned for the [KiwiMate](https://kiwimate.net) AI companion app. It is the smallest model in the KiwiMate model family and is designed for fast, low-cost inference on the app's free and lower-tier subscription plans.
|
||
|
||
> ⚠️ **Preview status:** This is a prototype release. The name reflects its preview status — expect breaking changes, retraining, and behavioural shifts before a stable v1 release.
|
||
|
||
## Model Details
|
||
|
||
| | |
|
||
|---|---|
|
||
| **Developed by** | KiwiMate / KyleCodeKiwi |
|
||
| **Base model** | Llama 3.2 3B Instruct |
|
||
| **Architecture** | Llama |
|
||
| **Parameters** | ~3.21B |
|
||
| **Fine-tuning framework** | [Unsloth](https://github.com/unslothai/unsloth) |
|
||
| **License** | Apache 2.0 |
|
||
| **Languages** | English (with New Zealand English and Te Reo Māori vocabulary coverage) |
|
||
| **Model class** | `AutoModelForCausalLM` |
|
||
|
||
## Intended Use
|
||
|
||
KiwiMate-Mini-Preview is intended as the default conversational backend for the KiwiMate app, providing:
|
||
|
||
- General-purpose chat and assistant-style conversation
|
||
- New Zealand cultural and "Kiwi" context awareness (slang, geography, fun facts)
|
||
- Light Te Reo Māori vocabulary recognition and use
|
||
- Lore and knowledge specific to the KiwiMate app itself ("KiwiMate Origin" data)
|
||
- Lightweight knowledge support for in-app mini-games
|
||
|
||
It is **not** intended for high-stakes, medical, legal, or financial advice, and should not be relied on as an authoritative source on Māori language or tikanga — for genuinely sensitive Te Reo or cultural content, defer to community-governed resources.
|
||
|
||
## Training Data
|
||
|
||
Fine-tuned on the [`KiwiMate/KiwiMate-Mini-training`](https://huggingface.co/datasets/KiwiMate/KiwiMate-Mini-training) dataset, organised into categories including:
|
||
|
||
- NZ English
|
||
- Te Reo Māori
|
||
- KiwiMate Origin (app-specific lore/identity)
|
||
- NZ Fun Facts
|
||
- MiniGame Knowledge
|
||
|
||
## Files & Quantizations
|
||
|
||
Distributed as `safetensors` (full precision) and GGUF quantizations for efficient local/edge inference:
|
||
|
||
| Format | Use case |
|
||
|---|---|
|
||
| F16 | Highest fidelity, largest size |
|
||
| Q6_K | Near-lossless, smaller footprint |
|
||
| Q4_K_M | Balanced quality/size — recommended default for on-device use |
|
||
| Q2_K_L | Smallest footprint, lowest fidelity |
|
||
|
||
## Deployment
|
||
|
||
Served in production via a Hugging Face Inference Endpoint on a T4 GPU with scale-to-zero, fronted by a Supabase Edge Function (OpenAI-compatible proxy) that routes KiwiMate app traffic to this and other KiwiMate model endpoints behind a single API.
|
||
|
||
### Known Limitations
|
||
|
||
- A server-side mitigation is in place for an occasional role-bleed / over-generation issue (the model sometimes continuing past `<|eot_id|>`), handled via stop-sequence aliases and trimming at the proxy layer.
|
||
- The long-term fix — adding `<|eot_id|>` (token ID 128009) properly to the training loss and `generation_config.json` — is planned for a future retraining pass rather than this preview.
|
||
- As a 3B-parameter model, reasoning depth and factual reliability are limited compared to larger models; it is tuned for speed and personality over raw capability.
|
||
|
||
## License
|
||
|
||
Released under the Apache 2.0 license, consistent with the open weights commitment for the KiwiMate model family.
|