---
library_name: transformers
license: apache-2.0
base_model: Qwen/Qwen3-1.7B-Base
tags:
- generated_from_trainer
model-index:
- name: qwen3-1.7b-arabic-standard-kd
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# qwen3-1.7b-arabic-standard-kd

This model is a fine-tuned version of [Qwen/Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0547

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.05
- num_epochs: 3

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 22.8230       | 0.1146 | 100  | 2.6974          |
| 21.2561       | 0.2292 | 200  | 2.4449          |
| 19.5810       | 0.3438 | 300  | 2.3259          |
| 18.0917       | 0.4585 | 400  | 2.2423          |
| 18.3612       | 0.5731 | 500  | 2.1928          |
| 17.0505       | 0.6877 | 600  | 2.1589          |
| 17.5159       | 0.8023 | 700  | 2.1294          |
| 17.3185       | 0.9169 | 800  | 2.1146          |
| 16.9705       | 1.0309 | 900  | 2.0997          |
| 16.5070       | 1.1456 | 1000 | 2.0920          |
| 15.2193       | 1.2602 | 1100 | 2.0815          |
| 16.0431       | 1.3748 | 1200 | 2.0748          |
| 16.2390       | 1.4894 | 1300 | 2.0687          |
| 16.1330       | 1.6040 | 1400 | 2.0644          |
| 16.7163       | 1.7186 | 1500 | 2.0605          |
| 15.7081       | 1.8332 | 1600 | 2.0585          |
| 16.4041       | 1.9479 | 1700 | 2.0572          |
| 16.4091       | 2.0619 | 1800 | 2.0562          |
| 16.0514       | 2.1765 | 1900 | 2.0552          |
| 15.4978       | 2.2911 | 2000 | 2.0550          |
| 16.5213       | 2.4057 | 2100 | 2.0548          |
| 15.9741       | 2.5203 | 2200 | 2.0554          |
| 15.5030       | 2.6350 | 2300 | 2.0547          |
| 16.0256       | 2.7496 | 2400 | 2.0552          |
| 15.7906       | 2.8642 | 2500 | 2.0550          |
| 16.2231       | 2.9788 | 2600 | 2.0548          |
| 15.8414       | 3.0    | 2619 | 2.0547          |


### Framework versions

- Transformers 5.4.0
- Pytorch 2.10.0+cu128
- Datasets 4.8.4
- Tokenizers 0.22.2