初始化项目,由ModelHub XC社区提供模型
Model: 5ivatej/qwen2.5-1.5B-india-finetuned Source: Original Platform
This commit is contained in:
69
README.md
Normal file
69
README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
- kn
|
||||
- hi
|
||||
- ta
|
||||
- te
|
||||
- mr
|
||||
- gu
|
||||
tags:
|
||||
- lora
|
||||
- sft
|
||||
- indic
|
||||
- qwen2.5
|
||||
license: other
|
||||
base_model: Qwen/Qwen2.5-1.5B
|
||||
datasets:
|
||||
- ai4bharat/indic-align
|
||||
library_name: mlx
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# qwen1.5B-india-finetuned
|
||||
|
||||
## Overview
|
||||
This repository contains **Qwen2.5-1.5B** fine-tuned with **LoRA** on small Indic instruction-following datasets.
|
||||
The LoRA adapters were merged into the base weights, producing a **standalone checkpoint** that can be used directly with `mlx_lm`.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
- The base model [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) is released under the [Qwen License](https://huggingface.co/Qwen/Qwen2.5-1.5B/blob/main/LICENSE).
|
||||
- This fine-tuned checkpoint is subject to the same license. Please review the terms before use, especially for commercial scenarios.
|
||||
- Marked here as `license: other` to follow Hugging Face conventions.
|
||||
|
||||
---
|
||||
|
||||
## Training Configuration
|
||||
- **Method:** LoRA-SFT (attention + MLP)
|
||||
- **LoRA hyperparams:** `r=16, alpha=32, dropout=0.05`
|
||||
- **Max sequence length:** 1024
|
||||
- **Steps:** 1500
|
||||
- **Batch size:** 1
|
||||
- **Optimizer:** AdamW (default in `mlx_lm`)
|
||||
- **Hardware:** Apple Silicon (MacBook Pro M4)
|
||||
- **Framework:** [`mlx_lm`](https://github.com/ml-explore/mlx-examples/tree/main/llms/mlx_lm)
|
||||
|
||||
The training configuration YAML used can be found at:
|
||||
[`configs/qwen2.5-3b_lora.yaml`](./configs/qwen2.5-3b_lora.yaml)
|
||||
|
||||
---
|
||||
|
||||
## Data
|
||||
- Subsets from [ai4bharat/indic-align](https://huggingface.co/datasets/ai4bharat/indic-align) were used: **Dolly_T** and **Anudesh**.
|
||||
- Converted into completion-style prompts (`prompt`/`completion` pairs).
|
||||
- The focus is on **Indic languages** (Kannada, Hindi, Tamil, Telugu, Marathi, Gujarati) with some English instructions.
|
||||
- Preprocessed into `train.jsonl` / `valid.jsonl` (not included here).
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
Run with `mlx_lm`:
|
||||
|
||||
```bash
|
||||
python -m mlx_lm generate \
|
||||
--model 5ivatej/qwen2.5-1.5B-india-finetuned \
|
||||
--max-tokens 200 \
|
||||
--prompt "Reply ONLY in Kannada written in English letters. Question: kannada dalli mathadoo?\n\nAnswer:"
|
||||
Reference in New Issue
Block a user