--- language: - en - th license: apache-2.0 base_model: Qwen/Qwen3-8B tags: - translation - thai - english - machine-translation - qwen3 - distillation datasets: - KordAI/Translation-Pairs-8K pipeline_tag: translation --- # Kord Translate ENTH V2 — 8B Bidirectional Thai ⇄ English translation model, fine-tuned from **Qwen3-8B** via **rationale-free distillation** from a large reasoning teacher (DeepSeek-V4-Flash). This model is part of the Kord Translate ENTH V2 family, accompanying the paper *"Teaching the Student to Skip the Homework: Rationale-Free Distillation for Thai-English Translation"* (KordAI, 2026). Other models in the family: [1.7B](https://huggingface.co/KordAI/Kord-Translate-ENTH-V2-1.7B), [4B](https://huggingface.co/KordAI/Kord-Translate-ENTH-V2-4B), [mBART50](https://huggingface.co/KordAI/Kord-Translate-MBART50-ENTH-V2). ## Model Description - **Base model:** Qwen3-8B - **Adaptation:** LoRA (rank 8, alpha 16, dropout 0.02) applied to all attention and MLP projection matrices, trained in 4-bit precision with gradient checkpointing (Unsloth) - **Training data:** [`KordAI/Translation-Pairs-8K`](https://huggingface.co/datasets/KordAI/Translation-Pairs-8K) — ~8,000 bidirectional Thai/English pairs generated by prompting DeepSeek-V4-Flash through an explicit 4-stage reasoning procedure (literal meaning → genre/formality → vocabulary/honorifics → natural rewrite), keeping **only the final translation** and discarding the reasoning trace - **Loss masking:** assistant-only, so gradients only flow through the translation output, not the system prompt or source text - **Epochs:** 3, LoRA learning rate 2e-4 (cosine decay, 5 warmup steps), paged AdamW 8-bit optimizer - **Compute:** 1× NVIDIA L40S (48GB), per-device batch 32, gradient accumulation 2 ## Results (FLORES devtest, 1,012 samples/direction) | Direction | Model | BLEU | chrF | chrF++ | BERTScore-P | BERTScore-F1 | COMET | |---|---|---|---|---|---|---|---| | en→th | Qwen3-8B (base) | **10.60** | **52.08** | **43.26** | 0.83 | 0.83 | **0.88** | | en→th | **Kord Translate 8B** | 9.29 | 51.38 | 42.40 | 0.83 | 0.83 | 0.87 | | th→en | Qwen3-8B (base) | **28.82** | **58.72** | **56.29** | 0.95 | 0.95 | **0.88** | | th→en | **Kord Translate 8B** | 26.80 | 57.20 | 54.61 | 0.94 | 0.94 | **0.88** | > **Note:** at 8B parameters, distillation on this ~8K-pair rationale-free set is slightly *behind* the untuned Qwen3-8B base on BLEU and COMET in both directions. The paper reads this as diminishing/negative returns rather than active harm: an 8B model already has substantial latent translation competence from pretraining, and 8,000 pairs are not enough additional signal to move it forward — narrow LoRA fine-tuning can mildly narrow the model's broader coverage. If peak raw translation quality is the priority, the untuned Qwen3-8B base or the 4B distilled model may be preferable; see the paper for full discussion. ## Inference This is a chat/instruction-tuned model. Prompt with a system message asking for translation and a user message containing the source text. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer MODEL_ID = "KordAI/Kord-Translate-ENTH-V2-8B" tokenizer = AutoTokenizer.from_pretrained(MODEL_ID) model = AutoModelForCausalLM.from_pretrained( MODEL_ID, torch_dtype=torch.bfloat16, device_map="auto", ) def translate(text: str, direction: str = "en2th") -> str: """direction: 'en2th' or 'th2en'""" src_lang, tgt_lang = ("English", "Thai") if direction == "en2th" else ("Thai", "English") messages = [ { "role": "system", "content": ( f"You are a professional {src_lang}-{tgt_lang} translator. " f"Translate the user's text from {src_lang} to {tgt_lang}. " "Output only the translation, with no explanation, notes, or extra text." ), }, {"role": "user", "content": text}, ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): output_ids = model.generate( **inputs, max_new_tokens=256, do_sample=False, temperature=None, top_p=None, top_k=None, ) generated = output_ids[0][inputs["input_ids"].shape[-1]:] return tokenizer.decode(generated, skip_special_tokens=True).strip() print(translate("How is the weather today in Bangkok?", direction="en2th")) print(translate("วันนี้อากาศที่กรุงเทพเป็นอย่างไรบ้าง", direction="th2en")) ``` **Using Unsloth (faster 4-bit inference, matches training setup):** ```python from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="KordAI/Kord-Translate-ENTH-V2-8B", max_seq_length=2048, load_in_4bit=True, ) FastLanguageModel.for_inference(model) # enable native 2x faster inference messages = [ {"role": "system", "content": "You are a professional English-Thai translator. Translate the user's text from English to Thai. Output only the translation, with no explanation, notes, or extra text."}, {"role": "user", "content": "How is the weather today in Bangkok?"}, ] inputs = tokenizer.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_tensors="pt" ).to("cuda") output_ids = model.generate(input_ids=inputs, max_new_tokens=256, do_sample=False) print(tokenizer.decode(output_ids[0][inputs.shape[-1]:], skip_special_tokens=True)) ``` --- ## Sample Translations ### English → Thai | Source | Translation | |--------|-------------| | *Ring also settled a lawsuit with competing security company, the ADT Corporation.* | ริงก์ยังได้แก้ไขปัญหาคดีพิพาทกับบริษัทความปลอดภัยคู่แข่งอย่าง ADT Corporation ด้วย | | *USA Gymnastics and the USOC have the same goal — making the sport of gymnastics, and others, as safe as possible for athletes to follow their dreams in a safe, positive and empowered environment.* | USA Gymnastics และ USOC มีเป้าหมายเดียวกัน คือ การทำให้กีฬาโกลด์มัตติกับกีฬาอื่นๆ เป็นไปอย่างปลอดภัยที่สุดสำหรับนักกีฬาที่จะตามฝันของพวกเขาในสภาพแวดล้อมที่ปลอดภัย มีบวก และเต็มไปด้วยพลัง | ### Thai → English | Source | Translation | |--------|-------------| | *แกงอาจมีทั้งชนิด "แห้ง" หรือ "น้ำ" ขึ้นอยู่กับปริมาณของเหลว* | Curry can be either \"dry\" or \"wet,\" depending on the amount of liquid it contains. | | *เนื่องจากมีหมู่เกาะให้เลือกถึง 17,000 เกาะ คำว่าอาหารอินโดนีเซียจึงเป็นคำเรียกกว้าง ๆ ที่ครอบคลุมถึงอาหารประจำภูมิภาคทั่วประเทศ* | With over 17,000 islands to choose from, the term \"Indonesian food\" is a broad label that encompasses regional specialties across the entire country. | --- ## Limitations - Trained on a small (~8K pair), single-teacher distillation set; may not generalize to document-level or highly colloquial Thai. - Evaluated only on FLORES devtest (sentence-level general-domain text). - At this scale, distillation slightly *underperforms* the untuned Qwen3-8B base on BLEU/COMET — consider the base model if raw metric performance matters more than any qualitative benefits of the fine-tune. ## Citation ```bibtex @article{kordai2026rationalefree, title = {Teaching the Student to Skip the Homework: Rationale-Free Distillation for Thai-English Translation}, author = {Jangjit, Naphon and Komsang, Jeerawat and Boran, Kord C.}, year = {2026}, organization = {KordAI} } ``` ## Acknowledgements Built on [Qwen3](https://arxiv.org/abs/2505.09388), with teacher supervision from [DeepSeek-V4](https://arxiv.org/html/2606.19348v1). LoRA fine-tuning follows [Hu et al., 2021](https://arxiv.org/abs/2106.09685) and the 4-bit recipe popularized by [QLoRA](https://arxiv.org/abs/2305.14314). This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. [](https://github.com/unslothai/unsloth)