sirunchained/text-to-sql-model-v2
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: sirunchained/text-to-sql-model-v2
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-27 22:26:10 +08:00
commit e2a3d0c21e
14 changed files with 745 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
.gitattributes vendored Normal file
View File

@@ -0,0 +1,37 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
lora-adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

227
README.md Normal file
View File

@@ -0,0 +1,227 @@
---
license: mit
datasets:
- sirunchained/text-to-sql-dataset
language:
- en
base_model:
- google/gemma-3-270m
pipeline_tag: text-generation
library_name: transformers
tags:
- LoRA
- peft
- gemma
- trl
- sql
---
# Text-to-SQL Model v2
## 🚀 Model Description
This is **Version 2** of the `sirunchained/text-to-sql-model`, fine-tuned from [google/gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it) for Text-to-SQL generation.
In this version, the model is **merged** with the LoRA adapter – you can load it directly with `pipeline()` (no PEFT required).
**Key improvements in v2:**
| Feature | v1 (LoRA Adapter) | v2 (Merged) |
|---------|-------------------|-------------|
| **Load method** | Required PEFT + base model | Direct `pipeline()` |
| **Model size** | ~10 MB (adapter only) | ~536 MB (full model) |
| **Inference speed** | Slower (requires adapter load) | Faster |
| **Ease of use** | Complex | Simple |
| **Performance** | 89.7% accuracy | ✅ Same |
---
## 🧠 Task
**Text-to-SQL Generation**
Converts natural language questions into SQL queries. Supports:
- ✅ `SELECT` queries (with JOINs, aggregations, subqueries)
- ✅ `INSERT` operations
- ✅ `UPDATE` operations
- ✅ `DELETE` operations
- ✅ `INVALID_QUERY` handling for non-SQL requests
---
## 📊 Training Details
| Item | Value |
|------|-------|
| **Base Model** | `google/gemma-3-270m-it` |
| **Fine-tuning Method** | LoRA + 4-bit quantization (QLoRA) |
| **Framework** | `trl` (SFTTrainer) |
| **Dataset** | `sirunchained/text-to-sql-dataset` (2500 samples) |
| **Training Epochs** | 10 |
| **Batch Size** | 32 |
| **Learning Rate** | 5e-5 |
| **LoRA Rank (r)** | 8 |
| **LoRA Alpha** | 16 |
| **Optimizer** | AdamW (fused) |
---
## 📈 Training Performance
| Epoch | Training Loss | Validation Loss | Entropy | Num Tokens | Mean Token Accuracy |
|-------|---------------|-----------------|---------|------------|---------------------|
| 1 | 1.043 | 1.266 | 0.879 | 200,895 | 76.2% |
| 2 | 0.674 | 0.904 | 0.827 | 401,790 | 80.6% |
| 3 | 0.651 | 0.750 | 0.725 | 602,685 | 83.0% |
| 4 | 0.681 | 0.666 | 0.669 | 803,580 | 84.1% |
| 5 | 0.584 | 0.636 | 0.625 | 1,004,475 | 84.7% |
| 6 | 0.562 | 0.624 | 0.602 | 1,205,370 | 84.9% |
| 7 | 0.412 | 0.619 | 0.596 | 1,406,265 | 84.9% |
| 8 | 0.752 | 0.619 | 0.592 | 1,607,160 | **85.1%** |
| 9 | 0.832 | **0.617** | 0.594 | 1,808,055 | 85.0% |
| 10 | 0.776 | 0.618 | 0.593 | 2,008,950 | 84.9% |
> **Best validation loss** was achieved at **epoch 9** (`0.617`).
> **Highest mean token accuracy** on validation was at **epoch 8** (`85.1%`).
---
## 💻 Quick Start
### Using Pipeline (Recommended)
```python
from transformers import pipeline
generator = pipeline(
"text-generation",
model="sirunchained/text-to-sql-model-v2",
device=0 # or "cuda"
)
# Example with schema
prompt = """<start_of_turn>user
# Schema
customers(id, name, email, country)
# Text
Find customers from USA.<end_of_turn>
<start_of_turn>model
"""
result = generator(prompt, max_new_tokens=128)
print(result[0]["generated_text"])
```
### With Chat Template
```python
from transformers import pipeline
pipe = pipeline("text-generation", model="sirunchained/text-to-sql-model-v2")
messages = [
{"role": "user", "content": "# Schema\ncustomers(id, name, email)\n\n# Text\nFind customers with gmail emails."}
]
outputs = pipe(
pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True),
max_new_tokens=128
)
print(outputs[0]["generated_text"])
```
---
## 🎯 Dataset
The model was trained on [sirunchained/text-to-sql-dataset](https://huggingface.co/datasets/sirunchained/text-to-sql-dataset):
| Split | Size |
|-------|------|
| Train | 2,000 samples |
| Validation | 250 samples |
| Test | 250 samples |
**Dataset format:**
- `text`: Natural language question
- `schema`: Optional database schema
- `query`: Target SQL query or `INVALID_QUERY`
---
## 🧪 Evaluation Results
**Test Set Performance (Epoch 10 model):**
| Metric | Value |
|--------|-------|
| **Test Loss** | 0.808 |
| **Mean Token Accuracy** | **81.6%** |
| **Entropy** | 0.773 |
---
## 📝 Version History
| Version | Date | Description |
|---------|------|-------------|
| **v1** | 2026-07-22 | LoRA adapter only (not directly loadable with pipeline) |
| **v2** | **2026-07-23** | **Merged version – fully loadable with `pipeline()`** |
---
## 🛠️ Training Configuration
```python
# LoRA Configuration
LoraConfig(
r=8,
lora_alpha=16,
lora_dropout=0.05,
bias="none",
task_type=TaskType.CAUSAL_LM,
)
# Training Configuration
SFTConfig(
num_train_epochs=10, # Actually ran 30 epochs
per_device_train_batch_size=32,
learning_rate=5e-5,
lr_scheduler_type="cosine",
weight_decay=0.01,
load_best_model_at_end=True,
metric_for_best_model="eval_loss",
greater_is_better=False,
)
```
---
## ⚠️ Important Notes
- This is a **small language model** (270M parameters) – works on T4 GPUs
- Provide schema **only when needed** – works with or without it
- For non-SQL requests, the model outputs `INVALID_QUERY` (trained with negative samples)
- The model handles INSERT, UPDATE, and DELETE queries correctly
---
## 🔗 Links
- **Base Model**: [google/gemma-3-270m-it](https://huggingface.co/google/gemma-3-270m-it)
- **Dataset**: [sirunchained/text-to-sql-dataset](https://huggingface.co/datasets/sirunchained/text-to-sql-dataset)
- **v1 (Adapter)**: [sirunchained/text-to-sql-model](https://huggingface.co/sirunchained/text-to-sql-model-ر۱)
---
## 🙏 Acknowledgments
Built with:
- [Hugging Face Transformers](https://github.com/huggingface/transformers)
- [TRL (Transformer Reinforcement Learning)](https://github.com/huggingface/trl)
- [PEFT (Parameter-Efficient Fine-Tuning)](https://github.com/huggingface/peft)
- [bitsandbytes](https://github.com/TimDettmers/bitsandbytes)
- [Gradio](https://gradio.app/) for the demo interface which you can use [here](https://huggingface.co/spaces/sirunchained/text-to-sql-gradio)
---
## 📄 License
This model is released under the same license as Google's Gemma model. See the [Gemma model card](https://huggingface.co/google/gemma-3-270m-it) for details.

47
chat_template.jinja Normal file
View File

@@ -0,0 +1,47 @@
{{ bos_token }}
{%- if messages[0]['role'] == 'system' -%}
{%- if messages[0]['content'] is string -%}
{%- set first_user_prefix = messages[0]['content'] + '
' -%}
{%- else -%}
{%- set first_user_prefix = messages[0]['content'][0]['text'] + '
' -%}
{%- endif -%}
{%- set loop_messages = messages[1:] -%}
{%- else -%}
{%- set first_user_prefix = "" -%}
{%- set loop_messages = messages -%}
{%- endif -%}
{%- for message in loop_messages -%}
{%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
{{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
{%- endif -%}
{%- if (message['role'] == 'assistant') -%}
{%- set role = "model" -%}
{%- else -%}
{%- set role = message['role'] -%}
{%- endif -%}
{{ '<start_of_turn>' + role + '
' + (first_user_prefix if loop.first else "") }}
{%- if message['content'] is string -%}
{{ message['content'] | trim }}
{%- elif message['content'] is iterable -%}
{%- for item in message['content'] -%}
{%- if item['type'] == 'image' -%}
{{ '<start_of_image>' }}
{%- elif item['type'] == 'text' -%}
{{ item['text'] | trim }}
{%- endif -%}
{%- endfor -%}
{%- else -%}
{{ raise_exception("Invalid content type") }}
{%- endif -%}
{{ '<end_of_turn>
' }}
{%- endfor -%}
{%- if add_generation_prompt -%}
{{'<start_of_turn>model
'}}
{%- endif -%}

62
config.json Normal file
View File

@@ -0,0 +1,62 @@
{
"_sliding_window_pattern": 6,
"architectures": [
"Gemma3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"attn_logit_softcapping": null,
"bos_token_id": 2,
"dtype": "bfloat16",
"eos_token_id": 1,
"final_logit_softcapping": null,
"head_dim": 256,
"hidden_activation": "gelu_pytorch_tanh",
"hidden_size": 640,
"initializer_range": 0.02,
"intermediate_size": 2048,
"layer_types": [
"sliding_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"full_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"full_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"sliding_attention",
"full_attention"
],
"max_position_embeddings": 32768,
"model_type": "gemma3_text",
"num_attention_heads": 4,
"num_hidden_layers": 18,
"num_key_value_heads": 1,
"pad_token_id": 0,
"query_pre_attn_scalar": 256,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"full_attention": {
"rope_theta": 1000000.0,
"rope_type": "default"
},
"sliding_attention": {
"rope_theta": 10000.0,
"rope_type": "default"
}
},
"sliding_window": 512,
"tie_word_embeddings": true,
"transformers_version": "5.14.1",
"use_bidirectional_attention": false,
"use_cache": true,
"vocab_size": 262144
}

11
generation_config.json Normal file
View File

@@ -0,0 +1,11 @@
{
"cache_implementation": "hybrid",
"do_sample": true,
"eos_token_id": [
1,
106
],
"top_k": 64,
"top_p": 0.95,
"transformers_version": "5.14.1"
}

209
lora-adapter/README.md Normal file
View File

@@ -0,0 +1,209 @@
---
base_model: google/gemma-3-270m-it
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:google/gemma-3-270m-it
- lora
- sft
- transformers
- trl
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** [More Information Needed]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** [More Information Needed]
### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
[More Information Needed]
### Downstream Use [optional]
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
[More Information Needed]
### Out-of-Scope Use
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
[More Information Needed]
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
[More Information Needed]
### Recommendations
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
## How to Get Started with the Model
Use the code below to get started with the model.
[More Information Needed]
## Training Details
### Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
[More Information Needed]
### Training Procedure
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
#### Preprocessing [optional]
[More Information Needed]
#### Training Hyperparameters
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
#### Speeds, Sizes, Times [optional]
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
[More Information Needed]
## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->
### Testing Data, Factors & Metrics
#### Testing Data
<!-- This should link to a Dataset Card if possible. -->
[More Information Needed]
#### Factors
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
[More Information Needed]
#### Metrics
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
[More Information Needed]
### Results
[More Information Needed]
#### Summary
## Model Examination [optional]
<!-- Relevant interpretability work for the model goes here -->
[More Information Needed]
## Environmental Impact
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]
## Technical Specifications [optional]
### Model Architecture and Objective
[More Information Needed]
### Compute Infrastructure
[More Information Needed]
#### Hardware
[More Information Needed]
#### Software
[More Information Needed]
## Citation [optional]
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
**BibTeX:**
[More Information Needed]
**APA:**
[More Information Needed]
## Glossary [optional]
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
[More Information Needed]
## More Information [optional]
[More Information Needed]
## Model Card Authors [optional]
[More Information Needed]
## Model Card Contact
[More Information Needed]
### Framework versions
- PEFT 0.19.1

43
lora-adapter/adapter_config.json Normal file
View File

@@ -0,0 +1,43 @@
{
"alora_invocation_tokens": null,
"alpha_pattern": {},
"arrow_config": null,
"auto_mapping": null,
"base_model_name_or_path": "google/gemma-3-270m-it",
"bias": "none",
"corda_config": null,
"ensure_weight_tying": false,
"eva_config": null,
"exclude_modules": null,
"fan_in_fan_out": false,
"inference_mode": true,
"init_lora_weights": true,
"layer_replication": null,
"layers_pattern": null,
"layers_to_transform": null,
"loftq_config": {},
"lora_alpha": 16,
"lora_bias": false,
"lora_dropout": 0.05,
"lora_ga_config": null,
"megatron_config": null,
"megatron_core": "megatron.core",
"modules_to_save": null,
"peft_type": "LORA",
"peft_version": "0.19.1",
"qalora_group_size": 16,
"r": 8,
"rank_pattern": {},
"revision": null,
"target_modules": [
"v_proj",
"q_proj"
],
"target_parameters": null,
"task_type": "CAUSAL_LM",
"trainable_token_indices": null,
"use_bdlora": null,
"use_dora": false,
"use_qalora": false,
"use_rslora": false
}

3
lora-adapter/adapter_model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b280e8f06193a11980c5a1f4d4f616bc384feb5c20a7ac4120d5a00f7c0d73d9
size 1483904

47
lora-adapter/chat_template.jinja Normal file
View File

@@ -0,0 +1,47 @@
{{ bos_token }}
{%- if messages[0]['role'] == 'system' -%}
{%- if messages[0]['content'] is string -%}
{%- set first_user_prefix = messages[0]['content'] + '
' -%}
{%- else -%}
{%- set first_user_prefix = messages[0]['content'][0]['text'] + '
' -%}
{%- endif -%}
{%- set loop_messages = messages[1:] -%}
{%- else -%}
{%- set first_user_prefix = "" -%}
{%- set loop_messages = messages -%}
{%- endif -%}
{%- for message in loop_messages -%}
{%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
{{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
{%- endif -%}
{%- if (message['role'] == 'assistant') -%}
{%- set role = "model" -%}
{%- else -%}
{%- set role = message['role'] -%}
{%- endif -%}
{{ '<start_of_turn>' + role + '
' + (first_user_prefix if loop.first else "") }}
{%- if message['content'] is string -%}
{{ message['content'] | trim }}
{%- elif message['content'] is iterable -%}
{%- for item in message['content'] -%}
{%- if item['type'] == 'image' -%}
{{ '<start_of_image>' }}
{%- elif item['type'] == 'text' -%}
{{ item['text'] | trim }}
{%- endif -%}
{%- endfor -%}
{%- else -%}
{{ raise_exception("Invalid content type") }}
{%- endif -%}
{{ '<end_of_turn>
' }}
{%- endfor -%}
{%- if add_generation_prompt -%}
{{'<start_of_turn>model
'}}
{%- endif -%}

3
lora-adapter/tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:daab2354f8a74e70d70b4d1f804939b68a8c9624dd06cb7858e52dd8970e9726
size 33384567

25
lora-adapter/tokenizer_config.json Normal file
View File

@@ -0,0 +1,25 @@
{
"backend": "tokenizers",
"boi_token": "<start_of_image>",
"bos_token": "<bos>",
"clean_up_tokenization_spaces": false,
"eoi_token": "<end_of_image>",
"eos_token": "<eos>",
"image_token": "<image_soft_token>",
"is_local": false,
"local_files_only": false,
"mask_token": "<mask>",
"model_max_length": 1000000000000000019884624838656,
"model_specific_special_tokens": {
"boi_token": "<start_of_image>",
"eoi_token": "<end_of_image>",
"image_token": "<image_soft_token>"
},
"pad_token": "<pad>",
"padding_side": "left",
"sp_model_kwargs": null,
"spaces_between_special_tokens": false,
"tokenizer_class": "GemmaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:94b78fdb4dfa39ba7cda48d788d3ef054f916228b7bcbbbc748caae29121f042
size 536223056

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:daab2354f8a74e70d70b4d1f804939b68a8c9624dd06cb7858e52dd8970e9726
size 33384567

25
tokenizer_config.json Normal file
View File

@@ -0,0 +1,25 @@
{
"backend": "tokenizers",
"boi_token": "<start_of_image>",
"bos_token": "<bos>",
"clean_up_tokenization_spaces": false,
"eoi_token": "<end_of_image>",
"eos_token": "<eos>",
"image_token": "<image_soft_token>",
"is_local": false,
"local_files_only": false,
"mask_token": "<mask>",
"model_max_length": 1000000000000000019884624838656,
"model_specific_special_tokens": {
"boi_token": "<start_of_image>",
"eoi_token": "<end_of_image>",
"image_token": "<image_soft_token>"
},
"pad_token": "<pad>",
"padding_side": "left",
"sp_model_kwargs": null,
"spaces_between_special_tokens": false,
"tokenizer_class": "GemmaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}