---
library_name: transformers
license: apache-2.0
base_model: HuggingFaceTB/SmolLM-135M
tags:
- generated_from_trainer
model-index:
- name: business-news-generator
  results: []
---


# business-news-generator

This model is a fine-tuned version of [HuggingFaceTB/SmolLM-135M](https://huggingface.co/HuggingFaceTB/SmolLM-135M) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 3.1695

## Model description

This model is a fine-tuned version of SmolLM-135M trained on the AG News dataset to generate business news-style text. The model learns patterns from financial and economic news articles and generates short business-related text based on prompts.

## Intended uses & limitations

This model is intended for educational purposes and experimentation with text generation. It can generate simple business news-style text based on prompts such as earnings reports, stock market updates, and merger announcements.

Limitations include occasional incoherent sentences, lack of factual accuracy, and reduced performance when using small training subsets or parameter-efficient fine-tuning methods like LoRA.

## Training and evaluation data

The model was trained on the AG News dataset, specifically filtered to include only business-related articles. The dataset contains labeled news text across categories such as World, Sports, Business, and Sci/Tech. Only the Business category was used for fine-tuning.

## Training procedure

The model was fine-tuned using the Hugging Face Transformers library. Training was performed for 2 epochs using a batch size of 8 and a cosine learning rate scheduler. Both full fine-tuning and LoRA-based fine-tuning approaches were implemented and compared.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- num_epochs: 2

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 3.1445        | 0.32  | 200  | 3.2506          |
| 2.8323        | 0.64  | 400  | 3.1549          |
| 2.6594        | 0.96  | 600  | 3.0463          |
| 1.689         | 1.28  | 800  | 3.1806          |
| 1.5107        | 1.6   | 1000 | 3.1657          |
| 1.4594        | 1.92  | 1200 | 3.1695          |


### Framework versions

- Transformers 4.57.6
- Pytorch 2.10.0+cu128
- Datasets 4.8.4
- Tokenizers 0.22.2
