library_name, license, base_model, tags, model-index

library_name	license	base_model	tags	model-index
transformers	apache-2.0	HuggingFaceTB/SmolLM-135M	generated_from_trainer	name results business-news-generator

business-news-generator

This model is a fine-tuned version of HuggingFaceTB/SmolLM-135M on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1695

Model description

This model is a fine-tuned version of SmolLM-135M trained on the AG News dataset to generate business news-style text. The model learns patterns from financial and economic news articles and generates short business-related text based on prompts.

Intended uses & limitations

This model is intended for educational purposes and experimentation with text generation. It can generate simple business news-style text based on prompts such as earnings reports, stock market updates, and merger announcements.

Limitations include occasional incoherent sentences, lack of factual accuracy, and reduced performance when using small training subsets or parameter-efficient fine-tuning methods like LoRA.

Training and evaluation data

The model was trained on the AG News dataset, specifically filtered to include only business-related articles. The dataset contains labeled news text across categories such as World, Sports, Business, and Sci/Tech. Only the Business category was used for fine-tuning.

Training procedure

The model was fine-tuned using the Hugging Face Transformers library. Training was performed for 2 epochs using a batch size of 8 and a cosine learning rate scheduler. Both full fine-tuning and LoRA-based fine-tuning approaches were implemented and compared.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
3.1445	0.32	200	3.2506
2.8323	0.64	400	3.1549
2.6594	0.96	600	3.0463
1.689	1.28	800	3.1806
1.5107	1.6	1000	3.1657
1.4594	1.92	1200	3.1695

Framework versions

• Transformers 4.57.6
• Pytorch 2.10.0+cu128
• Datasets 4.8.4
• Tokenizers 0.22.2