114 lines
3.0 KiB
Markdown
114 lines
3.0 KiB
Markdown
---
|
|
base_model: meta-llama/Llama-3.2-3B
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- text-to-sql
|
|
- sql
|
|
- llama
|
|
- llama-3.2
|
|
- qlora
|
|
- lora
|
|
- unsloth
|
|
- gguf
|
|
language:
|
|
- en
|
|
datasets:
|
|
- gretelai/synthetic_text_to_sql
|
|
---
|
|
|
|
# Llama 3.2 3B — Text-to-SQL
|
|
|
|
A fine-tuned version of **Meta Llama 3.2 3B** for converting natural-language questions into SQL queries.
|
|
|
|
The model was fine-tuned using **QLoRA** with a 4-bit quantized base model and LoRA adapters. After training, the adapter was merged with the base model to produce this standalone model.
|
|
|
|
## Model Repositories
|
|
|
|
### Merged Model
|
|
|
|
The base model and trained LoRA adapter have been merged into a standalone model:
|
|
|
|
https://huggingface.co/farehaheha/llama3.2-3B-text-to-sql
|
|
|
|
### Quantized GGUF Model
|
|
|
|
A `Q4_K_M` GGUF quantized version is available for efficient local inference with llama.cpp and other GGUF-compatible runtimes:
|
|
|
|
https://huggingface.co/farehaheha/llama3.2-3B-text-to-sql-Q4_K_M-GGUF
|
|
|
|
## Training Details
|
|
|
|
| Parameter | Value |
|
|
|---|---|
|
|
| Base Model | `meta-llama/Llama-3.2-3B` |
|
|
| Fine-tuning Method | QLoRA |
|
|
| Base Model Quantization During Training | 4-bit |
|
|
| LoRA Rank | 16 |
|
|
| Training Dataset | `gretelai/synthetic_text_to_sql` |
|
|
| Dataset Split Used | Test split |
|
|
| Training Samples | ~5.85K |
|
|
| Epochs | 1 |
|
|
| Batch Size | 8 |
|
|
| Training Time | ~30 minutes |
|
|
|
|
> **Note:** This experiment used the dataset's test split for fine-tuning rather than the training split. Therefore, the original test split should not be used to report an unbiased evaluation score for this model.
|
|
|
|
## Dataset
|
|
|
|
The model was fine-tuned on:
|
|
|
|
**Gretel AI Synthetic Text-to-SQL**
|
|
|
|
https://huggingface.co/datasets/gretelai/synthetic_text_to_sql
|
|
|
|
The dataset contains natural-language questions paired with SQL queries and database context.
|
|
|
|
Approximately **5,850 examples from the test split** were used for this fine-tuning experiment.
|
|
|
|
## Fine-Tuning Approach
|
|
|
|
The training process used QLoRA:
|
|
|
|
1. The Llama 3.2 3B base model was loaded in 4-bit precision.
|
|
2. The original base model weights remained frozen.
|
|
3. LoRA adapters with rank 16 were trained on the Text-to-SQL dataset.
|
|
4. Training was performed for one epoch.
|
|
5. The trained LoRA adapter was merged with the base model.
|
|
6. The merged model was also converted to Q4_K_M GGUF for local inference.
|
|
|
|
This approach significantly reduces the memory required for fine-tuning compared with full-parameter training.
|
|
## Usage
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
model_id = "farehaheha/llama3.2-3B-text-to-sql"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_id,
|
|
device_map="auto",
|
|
torch_dtype="auto",
|
|
)
|
|
|
|
prompt = """### Database Schema:
|
|
{your_database_schema}
|
|
|
|
### Request:
|
|
{your_natural_language_request}
|
|
|
|
### SQL Query:
|
|
"""
|
|
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
|
|
|
outputs = model.generate(
|
|
**inputs,
|
|
max_new_tokens=256,
|
|
do_sample=False,
|
|
)
|
|
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
print(response) |