初始化项目，由ModelHub XC社区提供模型

Model: OpceanAI/Yuuki-NxG-nano Source: Original Platform
2026-07-23 03:23:11 +08:00
commit ecc49ffcb7
11 changed files with 251660 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+yuuki_nano_benchmarks.png filter=lfs diff=lfs merge=lfs -text
+yuuki_trio_benchmarks.png filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,696 @@
+---
+license: apache-2.0
+datasets:
+- OpceanAI/Yuuki-dataset
+- nyuuzyou/google-code-archive
+- OpceanAI/Yuuki-Personality
+language:
+- en
+- es
+base_model:
+- openai-community/gpt2
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- conversation
+- companion
+- pytorch 
+- axolotl
+- personality
+- fine-tuned
+- tiny
+metrics:
+- perplexity
+widget:
+- text: Hello, how are you?
+  example_title: General Conversation
+- text: Can you help me understand recursion?
+  example_title: Technical Explanation
+- text: I've been feeling a bit overwhelmed lately.
+  example_title: Emotional Support
+---
+
+<div align="center">
+
+<br>
+
+<img src="https://img.shields.io/badge/%E2%9C%A6-YUUKI--NxG--NANO-0D1117?style=for-the-badge&labelColor=0D1117" alt="Yuuki NxG Nano" height="50">
+
+<br><br>
+
+# An 81M Companion Model That Competes With 3B Giants
+
+**Personality-aligned language model trained with zero cloud compute budget.**<br>
+**community/gpt2 architecture. 81 million parameters. MacBook Pro Intel 2020. $0.00.**
+
+<br>
+
+<a href="#benchmark-results"><img src="https://img.shields.io/badge/BENCHMARKS-0D1117?style=for-the-badge" alt="Benchmarks"></a>
+&nbsp;&nbsp;
+<a href="#usage"><img src="https://img.shields.io/badge/USAGE-0D1117?style=for-the-badge" alt="Usage"></a>
+&nbsp;&nbsp;
+<a href="https://github.com/sponsors/aguitauwu"><img src="https://img.shields.io/badge/SPONSOR-0D1117?style=for-the-badge" alt="Sponsor"></a>
+
+<br><br>
+
+[![License](https://img.shields.io/badge/Apache_2.0-1a1a2e?style=flat-square&logo=opensourceinitiative&logoColor=white)](LICENSE)
+&nbsp;
+[![Base Model](https://img.shields.io/badge/openai-community%2Fgpt2-1a1a2e?style=flat-square&logo=openai&logoColor=white)](https://huggingface.co/openai-community/gpt2)
+&nbsp;
+[![Framework](https://img.shields.io/badge/Transformers-1a1a2e?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/docs/transformers)
+&nbsp;
+[![Hardware](https://img.shields.io/badge/MacBook_Pro_Intel_2020-1a1a2e?style=flat-square&logo=apple&logoColor=white)](https://www.apple.com/macbook-pro/)
+&nbsp;
+[![Eval](https://img.shields.io/badge/lm--eval--harness-1a1a2e?style=flat-square&logo=python&logoColor=white)](https://github.com/EleutherAI/lm-evaluation-harness)
+
+<br>
+
+---
+
+<br>
+
+</div>
+
+## What is Yuuki NxG Nano?
+
+**Yuuki NxG Nano** is an 81-million parameter language model fine-tuned for open-ended conversation, emotional support, and general-purpose reasoning. It is the lightweight member of the NxG model family developed by OpceanAI, designed to run on constrained hardware — including mobile devices and single-board computers.
+
+The model was trained entirely on a **MacBook Pro Intel (2020)** with no external compute budget and no cloud GPU infrastructure. All benchmark evaluations were conducted on Kaggle P100 using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
+
+Despite being the smallest model in the comparison — 81M parameters versus competitors with 125M–160M — and evaluated strictly **0-shot** while competitors use few-shot prompting, Yuuki NxG Nano achieves the **highest TruthfulQA score** across all same-scale models. It also matches Llama-3.2-3B (Meta) in TruthfulQA — a model **37 times larger** — under stricter evaluation conditions.
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Model Summary
+
+</div>
+
+<br>
+
+<table>
+<tr>
+<td width="50%" valign="top">
+
+**Architecture**
+
+| Property | Value |
+|:---------|:------|
+| Base Model | gpt2 |
+| Parameters | 81M |
+| Fine-tuning | Supervised SFT |
+| Training Examples | ~5,000 |
+| Training Hardware | MacBook Pro Intel (2020) |
+| Context Length | 32,768 tokens |
+
+</td>
+<td width="50%" valign="top">
+
+**Release**
+
+| Property | Value |
+|:---------|:------|
+| Organization | OpceanAI |
+| Release Date | February 2026 |
+| Languages | English, Spanish |
+| License | Apache 2.0 |
+| Evaluation | lm-evaluation-harness |
+| Compute Budget | $0.00 |
+
+</td>
+</tr>
+</table>
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Benchmark Results
+
+</div>
+
+<br>
+
+All Yuuki NxG Nano results are evaluated **0-shot**. Competitor scores are sourced from lm-evaluation-harness references and use few-shot prompting. Direct numerical comparison systematically favors models evaluated with few-shot prompting.
+
+<br>
+
+![Yuuki NxG Nano Benchmark Evaluation](yuuki_nano_benchmarks.png)
+
+<br>
+
+### Same-Scale Comparison (80–160M)
+
+| Model | ARC-C | HellaSwag | MMLU | WinoGrande | TruthfulQA | Eval |
+|:------|:-----:|:---------:|:----:|:----------:|:----------:|:----:|
+| **Yuuki NxG Nano** | **24.32%** | 27.44% | 22.97% | 50.12% | **44.10%** | **0-shot** |
+| GPT-2 (125M) | 22.95% | **31.64%** | **25.90%** | 50.04% | 31.73% | few-shot |
+| GPT-Neo (125M) | 22.87% | 30.58% | 25.58% | **51.70%** | 35.70% | few-shot |
+| OPT-125M | 22.10% | 31.69% | 25.94% | 51.07% | 33.50% | few-shot |
+| Pythia-160M | 22.10% | 30.15% | 25.10% | 48.78% | 34.75% | few-shot |
+
+<br>
+
+### Cross-Scale Comparison — TruthfulQA
+
+![Yuuki Nano vs GPT-2 vs Yuuki NxG](yuuki_trio_benchmarks.png)
+
+
+The most significant result: Yuuki NxG Nano at 81M parameters matches or surpasses models with 2–3B parameters in factual honesty — models **25–37 times larger**, all evaluated with few-shot prompting.
+
+| Model | Params | TruthfulQA | Eval |
+|:------|:------:|:----------:|:----:|
+| **Yuuki NxG** | **3B** | **50.87%** | **0-shot** |
+| **Yuuki NxG Nano** | **81M** | **44.10%** | **0-shot** |
+| Llama-3.2-3B | 3B | 44.0% | few-shot |
+| Gemma-2-2B | 2B | 39.0% | few-shot |
+| GPT-2 | 125M | 31.73% | few-shot |
+
+Yuuki NxG Nano finishes **second overall in TruthfulQA** — behind only its larger sibling Yuuki NxG (3B). Both first and second place belong to OpceanAI.
+
+<br>
+
+Nano's TruthfulQA performance demonstrates that factual honesty is a property of training data quality, not model scale. The 5,000-example dataset transferred this characteristic to an 81M model with minimal degradation — from 50.87% (NxG 3B) to 44.10% (Nano 81M), a gap of only 6.77 points across a 37x reduction in parameter count.
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## NxG Model Family
+
+</div>
+
+<br>
+
+<table>
+<tr>
+<td width="50%" valign="top">
+
+**Released Models**
+
+| Model | Parameters | Description |
+|:------|:----------:|:------------|
+| Yuuki NxG | 3B | Full model, general conversation |
+| Yuuki NxG Nano | 81M | Lightweight, constrained environments |
+
+</td>
+<td width="50%" valign="top">
+
+**Community GGUF (via mradermacher)**
+
+Quantized independently without solicitation — organic community adoption prior to any formal announcement. Available at [mradermacher/Yuuki-NxG-nano-GGUF](https://huggingface.co/mradermacher/Yuuki-NxG-nano-GGUF).
+
+| Bits | Format | Size |
+|:----:|:-------|:----:|
+| 2-bit | Q2_K | 52.6 MB |
+| 3-bit | Q3_K_S | 55.1 MB |
+| 3-bit | Q3_K_M | 58.8 MB |
+| 3-bit | Q3_K_L | 61.0 MB |
+| 4-bit | IQ4_XS | 59.4 MB |
+| 4-bit | Q4_K_S | 60.7 MB |
+| 4-bit | Q4_K_M | 63.3 MB |
+| 5-bit | Q5_K_S | 66.0 MB |
+| 5-bit | Q5_K_M | 68.1 MB |
+| 6-bit | Q6_K | 71.7 MB |
+| 8-bit | Q8_0 | 91.3 MB |
+| 16-bit | F16 | 167 MB |
+
+</td>
+</tr>
+</table>
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Usage
+
+</div>
+
+<br>
+
+### With Transformers (PyTorch)
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "OpceanAI/Yuuki-NxG-Nano"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.float16,
+    device_map="auto"
+)
+
+messages = [
+    {"role": "user", "content": "Hello, how are you?"}
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    return_tensors="pt"
+).to(model.device)
+
+with torch.no_grad():
+    outputs = model.generate(
+        inputs,
+        max_new_tokens=512,
+        temperature=0.7,
+        do_sample=True,
+        repetition_penalty=1.1
+    )
+
+print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
+```
+
+<br>
+
+### With llama.cpp (GGUF)
+
+```bash
+./llama.cpp/main -m yuuki-nxg-nano-q4_k_m.gguf \
+    -p "Hello, how are you?" \
+    -n 256 \
+    -t 4 \
+    --temp 0.7 \
+    --repeat-penalty 1.1
+```
+
+<br>
+
+### With Ollama
+
+```bash
+cat > Modelfile << EOF
+FROM ./yuuki-nxg-nano-q4_k_m.gguf
+
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.1
+EOF
+
+ollama create yuuki-nxg-nano -f Modelfile
+ollama run yuuki-nxg-nano "Hello, how are you?"
+```
+
+<br>
+
+### Recommended Parameters
+
+| Parameter | Value |
+|:----------|:-----:|
+| Temperature | 0.7 |
+| Top-p | 0.9 |
+| Max new tokens | 256–1024 |
+| Repetition penalty | 1.1 |
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Training Details
+
+</div>
+
+<br>
+
+<table>
+<tr>
+<td width="50%" valign="top">
+
+**Hardware**
+
+| Component | Specification |
+|:----------|:-------------|
+| Device | MacBook Pro Intel (2020) |
+| CPU | Intel Core i5 (10th gen) |
+| RAM | 8–16 GB LPDDR4X |
+| GPU | Intel Iris Plus (integrated) |
+| Cloud Compute | None |
+| Cost | $0.00 |
+
+</td>
+<td width="50%" valign="top">
+
+**Training Configuration**
+
+| Parameter | Value |
+|:----------|:-----:|
+| Base Model | gpt2 |
+| Method | Supervised Fine-Tuning |
+| Training Examples | ~5,000 |
+| Optimizer | AdamW |
+| Learning Rate | 2e-5 |
+| Max Sequence Length | 2,048 tokens |
+
+</td>
+</tr>
+</table>
+
+<br>
+
+Yuuki NxG Nano was produced through supervised fine-tuning on the same curated conversational dataset used for Yuuki NxG (3B). The training objective was identical: consistent personality, high factual honesty, and broad general-knowledge retention.
+
+Training on a MacBook Pro Intel without GPU acceleration imposes significant constraints on batch size and training speed. The resulting benchmark profile reflects the scale limit of 81M parameters — strong where honesty and reasoning matter, below average where raw memorization of large knowledge bases is required.
+
+The 6.77-point gap between Nano (44.10%) and NxG (50.87%) on TruthfulQA, despite a 37x difference in parameter count, validates the training methodology: the same dataset quality that produced the NxG's results transferred effectively to a model one order of magnitude smaller.
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Features
+
+</div>
+
+<br>
+
+<table>
+<tr>
+<td width="50%" valign="top">
+
+**Runs Anywhere**
+
+At 63.3 MB (Q4_K_M), Yuuki NxG Nano runs on mobile devices, Raspberry Pi, edge hardware, and any CPU. No GPU required. No cloud dependency.
+
+<br>
+
+**Factual Honesty at 81M**
+
+Achieves 44.10% TruthfulQA — higher than all same-scale models and matching Llama-3.2-3B (Meta) at 44.0%, a model 37x larger evaluated with few-shot advantage.
+
+<br>
+
+**Multilingual**
+
+Functional in both English and Spanish. Responds in the user's language automatically, inherited from the NxG training data.
+
+</td>
+<td width="50%" valign="top">
+
+**Zero-Budget Training**
+
+Trained on a MacBook Pro Intel with no cloud compute. Demonstrates that alignment fine-tuning at sub-100M scale is accessible to anyone with consumer hardware.
+
+<br>
+
+**Community Adoption**
+
+Independently quantized by mradermacher across 12 formats before any formal announcement — the full quantization spectrum from Q2_K (52.6 MB) to F16 (167 MB).
+
+<br>
+
+**Open Source**
+
+Apache 2.0. Use commercially, modify, distribute. Full transparency on training methodology and evaluation protocol.
+
+</td>
+</tr>
+</table>
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Limitations
+
+</div>
+
+<br>
+
+- **Knowledge capacity**: At 81M parameters, MMLU performance (22.97%) is near random baseline (25%). The model cannot reliably recall dense academic knowledge across 57 domains.
+- **HellaSwag degradation** reflects the standard tradeoff of personality fine-tuning on sentence-completion benchmarks.
+- **Benchmark methodology**: Yuuki NxG Nano is evaluated 0-shot while competitor reports use few-shot prompting, creating a systematic disadvantage in direct comparisons.
+- **Safety alignment** has not been formally evaluated. Not recommended for adversarial or high-stakes deployment without additional safety filtering.
+- **Not a replacement for NxG**: For tasks requiring broad knowledge or complex reasoning, Yuuki NxG (3B) is the recommended model.
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Intended Use
+
+</div>
+
+<br>
+
+<table>
+<tr>
+<td width="50%" valign="top">
+
+**Intended For**
+
+- Edge and mobile deployment
+- Conversational assistance on constrained hardware
+- Emotional support applications
+- Offline use cases with no internet dependency
+- Research into tiny-scale alignment fine-tuning
+- Distillation teacher for sub-100M models
+
+</td>
+<td width="50%" valign="top">
+
+**Not Intended For**
+
+- Tasks requiring broad academic knowledge (use NxG 3B)
+- Mathematical or scientific reasoning
+- Applications requiring certified safety alignment
+- Production systems without additional safety review
+
+</td>
+</tr>
+</table>
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Philosophy
+
+</div>
+
+<br>
+
+> **"Honesty is not a property of scale. It is a property of training."**
+
+Yuuki NxG Nano was built to demonstrate that an 81M model trained by one person on a MacBook with $0 can match Meta's Llama-3.2-3B in factual honesty — and outperform every model of its own scale under stricter evaluation conditions.
+
+The result validates a core OpceanAI thesis: data quality matters more than compute when the objective is alignment, not memorization.
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Related Projects
+
+</div>
+
+<br>
+
+| Project | Description |
+|:--------|:------------|
+| [Yuuki NxG](https://huggingface.co/OpceanAI/Yuuki-NxG) | 3B full model, flagship of the NxG family |
+| [Yuuki-3.7](https://huggingface.co/OpceanAI/Yuuki-3.7) | Earlier code generation checkpoint |
+| [Yuuki-best](https://huggingface.co/OpceanAI/Yuuki-best) | Best checkpoint of the v0.1 series |
+| [yuy](https://github.com/YuuKi-OS/yuy) | CLI for managing and running Yuuki models |
+| [yuy-chat](https://github.com/YuuKi-OS/yuy-chat) | TUI chat interface |
+| [Yuuki-chat](https://github.com/YuuKi-OS/Yuuki-chat) | Web-based chat interface |
+| [Yuuki Space](https://huggingface.co/spaces/OpceanAI/Yuuki) | Interactive demo |
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Links
+
+</div>
+
+<br>
+
+<div align="center">
+
+[![Model Weights](https://img.shields.io/badge/Model_Weights-Hugging_Face-ffd21e?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/OpceanAI/Yuuki-NxG-Nano)
+&nbsp;
+[![Live Demo](https://img.shields.io/badge/Live_Demo-Spaces-ffd21e?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/spaces/OpceanAI/Yuuki)
+&nbsp;
+[![GGUF](https://img.shields.io/badge/GGUF_Quants-mradermacher-181717?style=for-the-badge&logo=github&logoColor=white)](https://huggingface.co/mradermacher/Yuuki-NxG-nano-GGUF)
+
+<br>
+
+[![YUY CLI](https://img.shields.io/badge/Yuy_CLI-GitHub-181717?style=for-the-badge&logo=github&logoColor=white)](https://github.com/YuuKi-OS/yuy)
+&nbsp;
+[![Sponsor](https://img.shields.io/badge/Sponsor-GitHub_Sponsors-ea4aaa?style=for-the-badge&logo=githubsponsors&logoColor=white)](https://github.com/sponsors/aguitauwu)
+&nbsp;
+[![Discord](https://img.shields.io/badge/Discord-Community-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/j8zV2u8k)
+
+</div>
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Community
+
+</div>
+
+<br>
+
+- [Discord Server](https://discord.gg/j8zV2u8k) — Development discussion and user community
+- [Twitter](https://twitter.com/aguitauwu) — Updates and announcements
+- [GitHub](https://github.com/aguitauwu) — Source code and training scripts
+- [GitHub Sponsors](https://github.com/sponsors/aguitauwu) — Support the project
+- [Ollama](https://ollama.com/aguitachan3/yuuki-nxg-nano) — Run locally with Ollama
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Citation
+
+</div>
+
+<br>
+
+```bibtex
+@misc{awa_omg_2026,
+	author       = { awa_omg },
+	title        = { Yuuki-NxG-nano (Revision 210ae00) },
+	year         = 2026,
+	url          = { https://huggingface.co/OpceanAI/Yuuki-NxG-nano },
+	doi          = { 10.57967/hf/7926 },
+	publisher    = { Hugging Face }
+}
+```
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## License
+
+</div>
+
+<br>
+
+```
+Apache License 2.0
+
+Copyright (c) 2026 OpceanAI
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+```
+
+Use commercially, modify, distribute. Attribution required.
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+## Updates
+
+</div>
+
+<br>
+
+| Date | Milestone |
+|:-----|:----------|
+| **2026-02-28** | Benchmark evaluation completed (Kaggle P100) |
+| **2026-02-28** | TruthfulQA: 44.10% — matches Llama-3.2-3B (3B) at 81M params |
+| **2026-02-28** | 2nd place TruthfulQA overall, behind only Yuuki NxG (3B) |
+| **2026-02-28** | Community GGUF quantization by mradermacher — 12 formats |
+| **2026-02-28** | Yuuki NxG Nano released on HuggingFace |
+
+**Last updated:** 2026-02-28
+
+<br>
+
+---
+
+<br>
+
+<div align="center">
+
+**81 million parameters. MacBook Pro Intel. $0. Matches Meta's Llama-3.2-3B in honesty.**
+
+<br>
+
+[![OpceanAI](https://img.shields.io/badge/OpceanAI-2026-0D1117?style=for-the-badge)](https://huggingface.co/OpceanAI)
+
+<br>
+
+*The NxG family. Honesty at every scale.*
+
+</div>
--- a/config.json
+++ b/config.json
@@ -0,0 +1,48 @@
+{
+  "_num_labels": 1,
+  "activation_function": "gelu_new",
+  "add_cross_attention": false,
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "dtype": "float32",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 6,
+  "n_positions": 1024,
+  "pad_token_id": 50256,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.0",
+  "use_cache": false,
+  "vocab_size": 50257
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": [
+    50256
+  ],
+  "pad_token_id": 50256,
+  "transformers_version": "5.0.0"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:033c075013a554bf84f4ebcd1dd5ee609e330f15d3bc9413983a62ff35c6e887
+size 327657928
--- a/scheduler.pt
+++ b/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:523d8282e4a819e5e25c581fa82390b38e5026bf7911019b4261040f1bb0e371
+size 1465
--- a/tokenizer.json
+++ b/tokenizer.json
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,14 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 1024,
+  "model_specific_special_tokens": {},
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}
--- a/trainer_state.json
+++ b/trainer_state.json
@@ -0,0 +1,524 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.4666666666666667,
+  "eval_steps": 500,
+  "global_step": 1400,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.006666666666666667,
+      "grad_norm": 3.3801944255828857,
+      "learning_rate": 9.5e-06,
+      "loss": 3.710173416137695,
+      "step": 20
+    },
+    {
+      "epoch": 0.013333333333333334,
+      "grad_norm": 2.4231245517730713,
+      "learning_rate": 1.9500000000000003e-05,
+      "loss": 3.2622753143310548,
+      "step": 40
+    },
+    {
+      "epoch": 0.02,
+      "grad_norm": 2.0525474548339844,
+      "learning_rate": 2.95e-05,
+      "loss": 2.287516975402832,
+      "step": 60
+    },
+    {
+      "epoch": 0.02666666666666667,
+      "grad_norm": 1.2226288318634033,
+      "learning_rate": 3.9500000000000005e-05,
+      "loss": 1.6385210037231446,
+      "step": 80
+    },
+    {
+      "epoch": 0.03333333333333333,
+      "grad_norm": 2.2779319286346436,
+      "learning_rate": 4.9500000000000004e-05,
+      "loss": 1.4153234481811523,
+      "step": 100
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 1.5858861207962036,
+      "learning_rate": 4.9994704516344666e-05,
+      "loss": 1.4460668563842773,
+      "step": 120
+    },
+    {
+      "epoch": 0.04666666666666667,
+      "grad_norm": 1.251664161682129,
+      "learning_rate": 4.99776910887808e-05,
+      "loss": 1.5103114128112793,
+      "step": 140
+    },
+    {
+      "epoch": 0.05333333333333334,
+      "grad_norm": 2.4487648010253906,
+      "learning_rate": 4.9948953034032466e-05,
+      "loss": 1.3196531295776368,
+      "step": 160
+    },
+    {
+      "epoch": 0.06,
+      "grad_norm": 1.276078701019287,
+      "learning_rate": 4.990850384185648e-05,
+      "loss": 1.3049612045288086,
+      "step": 180
+    },
+    {
+      "epoch": 0.06666666666666667,
+      "grad_norm": 1.4780528545379639,
+      "learning_rate": 4.985636249926392e-05,
+      "loss": 1.164187526702881,
+      "step": 200
+    },
+    {
+      "epoch": 0.07333333333333333,
+      "grad_norm": 1.1906464099884033,
+      "learning_rate": 4.979255348160766e-05,
+      "loss": 1.418909740447998,
+      "step": 220
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 1.3630824089050293,
+      "learning_rate": 4.9717106741093454e-05,
+      "loss": 1.0971190452575683,
+      "step": 240
+    },
+    {
+      "epoch": 0.08666666666666667,
+      "grad_norm": 1.3592742681503296,
+      "learning_rate": 4.96300576927203e-05,
+      "loss": 1.358650016784668,
+      "step": 260
+    },
+    {
+      "epoch": 0.09333333333333334,
+      "grad_norm": 1.215366244316101,
+      "learning_rate": 4.953144719765649e-05,
+      "loss": 1.3464448928833008,
+      "step": 280
+    },
+    {
+      "epoch": 0.1,
+      "grad_norm": 1.00632643699646,
+      "learning_rate": 4.942132154405919e-05,
+      "loss": 0.925452709197998,
+      "step": 300
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "grad_norm": 1.3117424249649048,
+      "learning_rate": 4.9299732425346626e-05,
+      "loss": 1.4749733924865722,
+      "step": 320
+    },
+    {
+      "epoch": 0.11333333333333333,
+      "grad_norm": 1.153543472290039,
+      "learning_rate": 4.9166736915933e-05,
+      "loss": 1.339738368988037,
+      "step": 340
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 1.147539496421814,
+      "learning_rate": 4.9022397444437495e-05,
+      "loss": 1.1870535850524901,
+      "step": 360
+    },
+    {
+      "epoch": 0.12666666666666668,
+      "grad_norm": 1.023755431175232,
+      "learning_rate": 4.886678176438004e-05,
+      "loss": 0.9843617439270019,
+      "step": 380
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 0.9747999906539917,
+      "learning_rate": 4.869996292237757e-05,
+      "loss": 0.9861419677734375,
+      "step": 400
+    },
+    {
+      "epoch": 0.14,
+      "grad_norm": 0.9171973466873169,
+      "learning_rate": 4.852201922385564e-05,
+      "loss": 1.001624870300293,
+      "step": 420
+    },
+    {
+      "epoch": 0.14666666666666667,
+      "grad_norm": 0.7914513945579529,
+      "learning_rate": 4.8333034196291516e-05,
+      "loss": 0.9351105690002441,
+      "step": 440
+    },
+    {
+      "epoch": 0.15333333333333332,
+      "grad_norm": 0.8947938680648804,
+      "learning_rate": 4.8133096550006096e-05,
+      "loss": 1.0424003601074219,
+      "step": 460
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.7379234433174133,
+      "learning_rate": 4.7922300136522914e-05,
+      "loss": 0.9020758628845215,
+      "step": 480
+    },
+    {
+      "epoch": 0.16666666666666666,
+      "grad_norm": 0.720455527305603,
+      "learning_rate": 4.7700743904513836e-05,
+      "loss": 0.8897322654724121,
+      "step": 500
+    },
+    {
+      "epoch": 0.17333333333333334,
+      "grad_norm": 1.3379108905792236,
+      "learning_rate": 4.746853185335221e-05,
+      "loss": 0.9499833106994628,
+      "step": 520
+    },
+    {
+      "epoch": 0.18,
+      "grad_norm": 0.9506772756576538,
+      "learning_rate": 4.722577298429512e-05,
+      "loss": 1.4986319541931152,
+      "step": 540
+    },
+    {
+      "epoch": 0.18666666666666668,
+      "grad_norm": 1.5777019262313843,
+      "learning_rate": 4.697258124931776e-05,
+      "loss": 1.3104500770568848,
+      "step": 560
+    },
+    {
+      "epoch": 0.19333333333333333,
+      "grad_norm": 1.2068767547607422,
+      "learning_rate": 4.670907549762392e-05,
+      "loss": 0.9996993064880371,
+      "step": 580
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 1.3350718021392822,
+      "learning_rate": 4.643537941985775e-05,
+      "loss": 1.0175305366516114,
+      "step": 600
+    },
+    {
+      "epoch": 0.20666666666666667,
+      "grad_norm": 1.7439497709274292,
+      "learning_rate": 4.6151621490042805e-05,
+      "loss": 1.0757975578308105,
+      "step": 620
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "grad_norm": 1.2179644107818604,
+      "learning_rate": 4.5857934905275895e-05,
+      "loss": 1.1192784309387207,
+      "step": 640
+    },
+    {
+      "epoch": 0.22,
+      "grad_norm": 1.1882299184799194,
+      "learning_rate": 4.555445752320379e-05,
+      "loss": 0.9488797187805176,
+      "step": 660
+    },
+    {
+      "epoch": 0.22666666666666666,
+      "grad_norm": 1.0356523990631104,
+      "learning_rate": 4.524133179731231e-05,
+      "loss": 0.3407599449157715,
+      "step": 680
+    },
+    {
+      "epoch": 0.23333333333333334,
+      "grad_norm": 1.4150390625,
+      "learning_rate": 4.491870471005814e-05,
+      "loss": 0.9790312767028808,
+      "step": 700
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 1.3073979616165161,
+      "learning_rate": 4.4586727703874606e-05,
+      "loss": 1.325801467895508,
+      "step": 720
+    },
+    {
+      "epoch": 0.24666666666666667,
+      "grad_norm": 1.2574576139450073,
+      "learning_rate": 4.424555661008406e-05,
+      "loss": 1.1854475021362305,
+      "step": 740
+    },
+    {
+      "epoch": 0.25333333333333335,
+      "grad_norm": 1.4099619388580322,
+      "learning_rate": 4.389535157574998e-05,
+      "loss": 1.0839804649353026,
+      "step": 760
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 1.2009291648864746,
+      "learning_rate": 4.3536276988503366e-05,
+      "loss": 1.030893611907959,
+      "step": 780
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 1.0084775686264038,
+      "learning_rate": 4.316850139937849e-05,
+      "loss": 0.9282444000244141,
+      "step": 800
+    },
+    {
+      "epoch": 0.2733333333333333,
+      "grad_norm": 1.166970133781433,
+      "learning_rate": 4.27921974436944e-05,
+      "loss": 0.9386281967163086,
+      "step": 820
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 1.1025112867355347,
+      "learning_rate": 4.24075417600192e-05,
+      "loss": 1.0037524223327636,
+      "step": 840
+    },
+    {
+      "epoch": 0.2866666666666667,
+      "grad_norm": 1.2833219766616821,
+      "learning_rate": 4.20147149072552e-05,
+      "loss": 0.7834305763244629,
+      "step": 860
+    },
+    {
+      "epoch": 0.29333333333333333,
+      "grad_norm": 1.5375336408615112,
+      "learning_rate": 4.1613901279883885e-05,
+      "loss": 1.0162606239318848,
+      "step": 880
+    },
+    {
+      "epoch": 0.3,
+      "grad_norm": 1.2366664409637451,
+      "learning_rate": 4.120528902141038e-05,
+      "loss": 1.101635456085205,
+      "step": 900
+    },
+    {
+      "epoch": 0.30666666666666664,
+      "grad_norm": 1.1046019792556763,
+      "learning_rate": 4.0789069936048135e-05,
+      "loss": 1.0482959747314453,
+      "step": 920
+    },
+    {
+      "epoch": 0.31333333333333335,
+      "grad_norm": 1.8129781484603882,
+      "learning_rate": 4.036543939868533e-05,
+      "loss": 1.2192636489868165,
+      "step": 940
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.8395664691925049,
+      "learning_rate": 3.993459626317506e-05,
+      "loss": 0.8940506935119629,
+      "step": 960
+    },
+    {
+      "epoch": 0.32666666666666666,
+      "grad_norm": 1.2955873012542725,
+      "learning_rate": 3.949674276899257e-05,
+      "loss": 0.7923289775848389,
+      "step": 980
+    },
+    {
+      "epoch": 0.3333333333333333,
+      "grad_norm": 1.2329128980636597,
+      "learning_rate": 3.905208444630327e-05,
+      "loss": 0.6925106525421143,
+      "step": 1000
+    },
+    {
+      "epoch": 0.34,
+      "grad_norm": 1.7410433292388916,
+      "learning_rate": 3.8600830019486013e-05,
+      "loss": 1.0536805152893067,
+      "step": 1020
+    },
+    {
+      "epoch": 0.3466666666666667,
+      "grad_norm": 1.0676844120025635,
+      "learning_rate": 3.814319130915709e-05,
+      "loss": 1.2916886329650878,
+      "step": 1040
+    },
+    {
+      "epoch": 0.35333333333333333,
+      "grad_norm": 1.0247275829315186,
+      "learning_rate": 3.7679383132740767e-05,
+      "loss": 1.041970157623291,
+      "step": 1060
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 1.2280709743499756,
+      "learning_rate": 3.720962320363313e-05,
+      "loss": 0.7665971755981446,
+      "step": 1080
+    },
+    {
+      "epoch": 0.36666666666666664,
+      "grad_norm": 1.792985200881958,
+      "learning_rate": 3.673413202900659e-05,
+      "loss": 0.9495917320251465,
+      "step": 1100
+    },
+    {
+      "epoch": 0.37333333333333335,
+      "grad_norm": 0.9368265867233276,
+      "learning_rate": 3.6253132806302895e-05,
+      "loss": 0.7774296283721924,
+      "step": 1120
+    },
+    {
+      "epoch": 0.38,
+      "grad_norm": 1.2027090787887573,
+      "learning_rate": 3.576685131846339e-05,
+      "loss": 0.9888124465942383,
+      "step": 1140
+    },
+    {
+      "epoch": 0.38666666666666666,
+      "grad_norm": 1.8240844011306763,
+      "learning_rate": 3.5275515827945635e-05,
+      "loss": 1.134997844696045,
+      "step": 1160
+    },
+    {
+      "epoch": 0.3933333333333333,
+      "grad_norm": 0.8984581828117371,
+      "learning_rate": 3.4779356969576044e-05,
+      "loss": 0.6457856655120849,
+      "step": 1180
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.8814728260040283,
+      "learning_rate": 3.427860764228903e-05,
+      "loss": 0.8272159576416016,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4066666666666667,
+      "grad_norm": 1.533231496810913,
+      "learning_rate": 3.377350289980331e-05,
+      "loss": 0.7864272117614746,
+      "step": 1220
+    },
+    {
+      "epoch": 0.41333333333333333,
+      "grad_norm": 1.4076637029647827,
+      "learning_rate": 3.326427984028675e-05,
+      "loss": 0.9194063186645508,
+      "step": 1240
+    },
+    {
+      "epoch": 0.42,
+      "grad_norm": 1.770647406578064,
+      "learning_rate": 3.275117749506156e-05,
+      "loss": 1.0757792472839356,
+      "step": 1260
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "grad_norm": 1.488553524017334,
+      "learning_rate": 3.2234436716402025e-05,
+      "loss": 1.0499624252319335,
+      "step": 1280
+    },
+    {
+      "epoch": 0.43333333333333335,
+      "grad_norm": 0.9816551804542542,
+      "learning_rate": 3.171430006447751e-05,
+      "loss": 1.106693172454834,
+      "step": 1300
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 0.9064175486564636,
+      "learning_rate": 3.119101169349374e-05,
+      "loss": 0.692702341079712,
+      "step": 1320
+    },
+    {
+      "epoch": 0.44666666666666666,
+      "grad_norm": 0.8113229870796204,
+      "learning_rate": 3.066481723708587e-05,
+      "loss": 0.41944007873535155,
+      "step": 1340
+    },
+    {
+      "epoch": 0.4533333333333333,
+      "grad_norm": 0.6801184415817261,
+      "learning_rate": 3.013596369301706e-05,
+      "loss": 0.26016736030578613,
+      "step": 1360
+    },
+    {
+      "epoch": 0.46,
+      "grad_norm": 0.9605411887168884,
+      "learning_rate": 2.9604699307236765e-05,
+      "loss": 0.47470459938049314,
+      "step": 1380
+    },
+    {
+      "epoch": 0.4666666666666667,
+      "grad_norm": 1.1666914224624634,
+      "learning_rate": 2.9071273457353082e-05,
+      "loss": 0.5180130481719971,
+      "step": 1400
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 3000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 9223372036854775807,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2926523606630400.0,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}
--- a/yuuki_nano_benchmarks.png
+++ b/yuuki_nano_benchmarks.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b271bd35bdb9487346db8931a37d4036d9a0cc530327a77ba31130dd4b11bb15
+size 376918
--- a/yuuki_trio_benchmarks.png
+++ b/yuuki_trio_benchmarks.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2988447480502b70497908573beabc1f04b7a47b57463919e7278720b16ac63
+size 365549