64 lines
2.3 KiB
Markdown
64 lines
2.3 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
base_model: Qwen/Qwen3-8B-Base
|
|||
|
|
tags:
|
|||
|
|
- cantonese
|
|||
|
|
- orthography
|
|||
|
|
- liujgoj
|
|||
|
|
- speech-first
|
|||
|
|
- cpt
|
|||
|
|
- text-generation
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Cantonese-Qwen3-8B-Base (Powered by Liujgoj)
|
|||
|
|
|
|||
|
|
## 📌 簡介 / Overview
|
|||
|
|
|
|||
|
|
6月7日重新訓練。
|
|||
|
|
|
|||
|
|
**Cantonese-Qwen3-8B-Base** 係一個專門為 **溜歌粵語書寫系統（Liujgoj Cantonese Orthography）** 進行增量預訓練（Continued Pre-Training, CPT）嘅大語言模型。
|
|||
|
|
|
|||
|
|
本模型以 **Qwen3-8B-Base** 為基座，透過大規模嘅語音原生（Speech-native）拉丁化粵語文本語料庫進行微調。核心目的係讓大模型深度內化溜歌系統的音節結構與聲調編碼，擺脫漢字（Hanzi）嘅依賴，實現機器原生的粵語文本理解與生成。
|
|||
|
|
|
|||
|
|
### ⚠️ 重要聲明 (Important Clarification)
|
|||
|
|
**Liujgoj（溜歌）發展自粵拼 Jyutping 系統，是一套設計用於獨立寫作、閱讀、且音位結構確定（Deterministic Syllable Structure）嘅完整粵語拉丁正字法（Independent Orthography）。** 本模型在預訓練階段將溜歌正詞法直接焊死在模型的 Token 概率分佈中。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📈 訓練數據與成果 / Training Details
|
|||
|
|
|
|||
|
|
- **基座模型 (Base Model)**: `Qwen3-8B-Base` (ModelScope / Hugging Face)
|
|||
|
|
- **訓練管道 (Pipeline)**: LLaMA-Factory (LoRA CPT, 3 Epochs)
|
|||
|
|
- **核心語料 (Corpus)**:
|
|||
|
|
- 精選超過 60 部香港經典電影部分對白之文本口語對白轉錄。
|
|||
|
|
- **訓練指標 (Metrics)**:
|
|||
|
|
- 初始 Loss: ~2.6
|
|||
|
|
- **最終 Train Loss: 1.5951** (收斂表現極佳，語意對齊深度完成)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🚀 快速開始 / Quick Start
|
|||
|
|
|
|||
|
|
本模型目前為 **CPT 基座模型 (Base Model)**，具備極強嘅 Liujgoj 文字續寫與接龍直覺。推薦使用 OpenAI 相容的 `completions` 或 `chat/completions` 接口進行純文字續寫。
|
|||
|
|
|
|||
|
|
### Python (使用 Transformers 載入)
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
import torch
|
|||
|
|
|
|||
|
|
model_id = "Yvthyvq/cantonese-qwen3-8b-base"
|
|||
|
|
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
model_id,
|
|||
|
|
torch_dtype=torch.bfloat16,
|
|||
|
|
device_map="auto"
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
prompt = "Yixgaaj heoi binj douh? "
|
|||
|
|
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
|
|||
|
|
|
|||
|
|
outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.7)
|
|||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|