qwen3-1.7b-arabic-standard-kd/README.md

library_name, license, base_model, tags, model-index

library_name	license	base_model	tags	model-index
transformers	apache-2.0	Qwen/Qwen3-1.7B-Base	generated_from_trainer	name results qwen3-1.7b-arabic-standard-kd

qwen3-1.7b-arabic-standard-kd

This model is a fine-tuned version of Qwen/Qwen3-1.7B-Base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.0547

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 0.05
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
22.8230	0.1146	100	2.6974
21.2561	0.2292	200	2.4449
19.5810	0.3438	300	2.3259
18.0917	0.4585	400	2.2423
18.3612	0.5731	500	2.1928
17.0505	0.6877	600	2.1589
17.5159	0.8023	700	2.1294
17.3185	0.9169	800	2.1146
16.9705	1.0309	900	2.0997
16.5070	1.1456	1000	2.0920
15.2193	1.2602	1100	2.0815
16.0431	1.3748	1200	2.0748
16.2390	1.4894	1300	2.0687
16.1330	1.6040	1400	2.0644
16.7163	1.7186	1500	2.0605
15.7081	1.8332	1600	2.0585
16.4041	1.9479	1700	2.0572
16.4091	2.0619	1800	2.0562
16.0514	2.1765	1900	2.0552
15.4978	2.2911	2000	2.0550
16.5213	2.4057	2100	2.0548
15.9741	2.5203	2200	2.0554
15.5030	2.6350	2300	2.0547
16.0256	2.7496	2400	2.0552
15.7906	2.8642	2500	2.0550
16.2231	2.9788	2600	2.0548
15.8414	3.0	2619	2.0547

Framework versions

• Transformers 5.4.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.4
• Tokenizers 0.22.2