初始化项目，由ModelHub XC社区提供模型

Model: KordAI/Kord-Translate-ENTH-V2-1.7B Source: Original Platform
2026-07-21 01:47:11 +08:00
commit 206a3e0b28
12 changed files with 152068 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,166 @@
+---
+language:
+- en
+- th
+license: apache-2.0
+base_model: Qwen/Qwen3-1.7B
+tags:
+- translation
+- thai
+- english
+- machine-translation
+- qwen3
+- distillation
+datasets:
+- KordAI/Translation-Pairs-8K
+pipeline_tag: translation
+---
+
+# Kord Translate ENTH V2 — 1.7B
+
+Bidirectional Thai ⇄ English translation model, fine-tuned from **Qwen3-1.7B** via **rationale-free distillation** from a large reasoning teacher (DeepSeek-V4-Flash).
+
+This model is part of the Kord Translate ENTH V2 family, accompanying the paper *"Teaching the Student to Skip the Homework: Rationale-Free Distillation for Thai-English Translation"* (KordAI, 2026). Other models in the family: [4B](https://huggingface.co/KordAI/Kord-Translate-ENTH-V2-4B), [8B](https://huggingface.co/KordAI/Kord-Translate-ENTH-V2-8B), [mBART50](https://huggingface.co/KordAI/Kord-Translate-MBART50-ENTH-V2).
+
+## Model Description
+
+- **Base model:** Qwen3-1.7B
+- **Adaptation:** LoRA (rank 8, alpha 16, dropout 0.02) applied to all attention and MLP projection matrices, trained in 4-bit precision with gradient checkpointing (Unsloth)
+- **Training data:** [`KordAI/Translation-Pairs-8K`](https://huggingface.co/datasets/KordAI/Translation-Pairs-8K) — ~8,000 bidirectional Thai/English pairs generated by prompting DeepSeek-V4-Flash through an explicit 4-stage reasoning procedure (literal meaning → genre/formality → vocabulary/honorifics → natural rewrite), keeping **only the final translation** and discarding the reasoning trace
+- **Loss masking:** assistant-only, so gradients only flow through the translation output, not the system prompt or source text
+- **Epochs:** 3, LoRA learning rate 2e-4 (cosine decay, 5 warmup steps), paged AdamW 8-bit optimizer
+- **Compute:** 1× NVIDIA Tesla T4 (16GB), per-device batch 2, gradient accumulation 32
+
+## Results (FLORES devtest, 1,012 samples/direction)
+
+| Direction | Model | BLEU | chrF | chrF++ | BERTScore-P | BERTScore-F1 | COMET |
+|---|---|---|---|---|---|---|---|
+| en→th | Qwen3-1.7B (base) | 8.00 | 42.20 | 35.21 | 0.80 | 0.79 | 0.81 |
+| en→th | **Kord Translate 1.7B** | **8.27** | **42.68** | **35.37** | **0.94** | **0.94** | **0.85** |
+| th→en | Qwen3-1.7B (base) | 22.11 | 52.84 | 50.39 | 0.94 | 0.94 | 0.85 |
+| th→en | **Kord Translate 1.7B** | **22.18** | 52.09 | 49.62 | 0.94 | 0.94 | **0.86** |
+
+At this scale, rationale-free distillation produces small, consistent improvements over the untuned Qwen3-1.7B base in both directions, most notably in BERTScore and COMET on en→th. See the paper for comparison against larger scales and specialized Thai-English systems.
+
+## Inference
+
+This is a chat/instruction-tuned model. Prompt with a system message asking for translation and a user message containing the source text.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+MODEL_ID = "KordAI/Kord-Translate-ENTH-V2-1.7B"
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+model = AutoModelForCausalLM.from_pretrained(
+    MODEL_ID,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+def translate(text: str, direction: str = "en2th") -> str:
+    """direction: 'en2th' or 'th2en'"""
+    src_lang, tgt_lang = ("English", "Thai") if direction == "en2th" else ("Thai", "English")
+    messages = [
+        {
+            "role": "system",
+            "content": (
+                f"You are a professional {src_lang}-{tgt_lang} translator. "
+                f"Translate the user's text from {src_lang} to {tgt_lang}. "
+                "Output only the translation, with no explanation, notes, or extra text."
+            ),
+        },
+        {"role": "user", "content": text},
+    ]
+
+    prompt = tokenizer.apply_chat_template(
+        messages, tokenize=False, add_generation_prompt=True
+    )
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+    with torch.no_grad():
+        output_ids = model.generate(
+            **inputs,
+            max_new_tokens=256,
+            do_sample=False,
+            temperature=None,
+            top_p=None,
+            top_k=None,
+        )
+
+    generated = output_ids[0][inputs["input_ids"].shape[-1]:]
+    return tokenizer.decode(generated, skip_special_tokens=True).strip()
+
+
+print(translate("How is the weather today in Bangkok?", direction="en2th"))
+print(translate("วันนี้อากาศที่กรุงเทพเป็นอย่างไรบ้าง", direction="th2en"))
+```
+
+**Using Unsloth (faster 4-bit inference, matches training setup):**
+
+```python
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="KordAI/Kord-Translate-ENTH-V2-1.7B",
+    max_seq_length=2048,
+    load_in_4bit=True,
+)
+FastLanguageModel.for_inference(model)  # enable native 2x faster inference
+
+messages = [
+    {"role": "system", "content": "You are a professional English-Thai translator. Translate the user's text from English to Thai. Output only the translation, with no explanation, notes, or extra text."},
+    {"role": "user", "content": "How is the weather today in Bangkok?"},
+]
+inputs = tokenizer.apply_chat_template(
+    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
+).to("cuda")
+
+output_ids = model.generate(input_ids=inputs, max_new_tokens=256, do_sample=False)
+print(tokenizer.decode(output_ids[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+---
+
+## Sample Translations
+
+### English → Thai
+
+| Source | Translation |
+|--------|-------------|
+| *Efforts to search for the crash site are being met by bad weather and harsh terrain.* | ความพยายามในการค้นหาที่เกิดเหตุยังคงถูกขัดขวางด้วยสภาพอากาศแย่และภูเขาที่รุนแรง|
+| *They are cooler than the surrounding surface in the day and warmer at night.* | พวกมันเย็นกว่าพื้นผิวรอบๆ ในช่วงกลางวัน และร้อนกว่าในช่วงกลางคืน |
+
+### Thai → English
+
+| Source | Translation |
+|--------|-------------|
+| *ระหว่างเดินทาง อิวาซากิประสบปัญหาหลายประการ* | While on the way, Ivaizaki encountered several problems. |
+| *วันนี้เมื่อเวลา 12.00 น. ตามเวลามาตรฐานสากล กลุ่ม Iraq Study Group ได้นำเสนอรายงานของพวกเขา* | At 12:00 noon local time, the Iraq Study Group presented their report. |
+
+---
+
+## Limitations
+
+- Trained on a small (~8K pair), single-teacher distillation set; may not generalize to document-level or highly colloquial Thai.
+- Evaluated only on FLORES devtest (sentence-level general-domain text).
+- Smallest model in the family — larger scales (4B, 8B) achieve higher absolute translation quality; this 1.7B model favors low compute/latency over peak quality.
+
+## Citation
+
+```bibtex
+@article{kordai2026rationalefree,
+  title   = {Teaching the Student to Skip the Homework: Rationale-Free Distillation for Thai-English Translation},
+  author  = {Jangjit, Naphon and Komsang, Jeerawat and Boran, Kord C.},
+  year    = {2026},
+  organization = {KordAI}
+}
+```
+
+## Acknowledgements
+
+Built on [Qwen3](https://arxiv.org/abs/2505.09388), with teacher supervision from [DeepSeek-V4](https://arxiv.org/html/2606.19348v1). LoRA fine-tuning follows [Hu et al., 2021](https://arxiv.org/abs/2106.09685) and the 4-bit recipe popularized by [QLoRA](https://arxiv.org/abs/2305.14314).
+
+This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+
+[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)