802 B
802 B
| 1 | Rank | Model | Type | BLEU | chrF++ | Our Model |
|---|---|---|---|---|---|---|
| 2 | 1 | Google Translate | Commercial Service | 9.27 | 46.44 | FALSE |
| 3 | 2 | Claude Sonnet 4 | Anthropic | 8.07 | 43.54 | FALSE |
| 4 | 3 | Gemini 2.0 Flash 001 | 7.94 | 43.38 | FALSE | |
| 5 | 4 | Gemini 2.5 Pro | 7.46 | 44.74 | FALSE | |
| 6 | 5 | Ganda Gemma 1B (Our Model) | Specialized Fine-tuned | 6.99 | 40.32 | TRUE |
| 7 | 6 | Gemini 2.5 Flash | 6.28 | 40.51 | FALSE | |
| 8 | 7 | Chatgpt 4o Latest | OpenAI | 6.19 | 40 | FALSE |
| 9 | 8 | Gpt Oss 120B | OpenAI | 5.16 | 34.7 | FALSE |
| 10 | 9 | Gpt 5 Mini | OpenAI | 5.14 | 36.55 | FALSE |
| 11 | 10 | Gpt 5 Nano | OpenAI | 4.93 | 32.86 | FALSE |
| 12 | 11 | Llama 4 Maverick | Meta | 4.52 | 33.75 | FALSE |
| 13 | 12 | Gemma 3 27B | 3.65 | 31.37 | FALSE | |
| 14 | 13 | Llama 4 Scout | Meta | 3.59 | 27.63 | FALSE |
| 15 | 14 | Gpt Oss 20B | OpenAI | 3.39 | 27.98 | FALSE |
| 16 | 15 | Gemma 3 4B | 1.1 | 20.05 | FALSE | |
| 17 | 16 | Gemma 3N E4B | 0.84 | 17.6 | FALSE | |
| 18 | 17 | Gemma 3 1B (vLLM Baseline) | General Model | 0.51 | 9.79 | FALSE |