105 lines
2.9 KiB
Markdown
105 lines
2.9 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
library_name: transformers
|
|
pipeline_tag: text2text-generation
|
|
---
|
|
|
|
The **Hrida-T2SQL-3B-V0.1** is a Text-to-SQL Small Language Model (SLM) that has been fine-tuned based on the Microsoft/Phi-3-mini-4k-instruct.
|
|
|
|
For full details of this model please read our [blog post](https://www.hridaai.com/blog/t2sql).
|
|
|
|
- Original Model: [Hrida-T2SQL-3B-V0.1](https://huggingface.co/HridaAI/Hrida-T2SQL-3B-V0.1)
|
|
- Ollama Model: [HridaAI/hrida-t2sql](https://ollama.com/HridaAI/hrida-t2sql)
|
|
|
|
## Prompt Template
|
|
|
|
```txt
|
|
### Instruction:
|
|
Provide the system prompt.
|
|
|
|
### Dialect:
|
|
Specify the SQL dialect (e.g., MySQL, PostgreSQL, SQL Server, etc.).
|
|
|
|
### Context:
|
|
Provide the database schema including table names, column names, and data types.
|
|
|
|
### Input:
|
|
User's query.
|
|
|
|
### Response:
|
|
Expected SQL query output based on the input and context.
|
|
|
|
```
|
|
|
|
- **Instruction (System Prompt)**: This guides the model on processing input to generate the SQL query response effectively.
|
|
- **Dialect (Optional)**: Specify the SQL variant the model should use to ensure the generated query conforms to the correct syntax.
|
|
- **Context**: Provide the database schema to the model for generating accurate SQL queries.
|
|
- **Input**: Provide the user query for the model to comprehend and transform into an SQL query.
|
|
- **Response**: Expected output from the model.
|
|
|
|
|
|
## Chat Prompt Template
|
|
|
|
```txt
|
|
<s>
|
|
<|system|>
|
|
{ Instruction / System Prompt }
|
|
<|user|>
|
|
{ Context / User Query } <|end|>
|
|
<|assistant|>
|
|
```
|
|
|
|
## Run the Model with LLamaCpp
|
|
|
|
```python
|
|
from llama_cpp import Llama
|
|
|
|
llm = Llama(
|
|
model_path="./Hrida-T2SQL-3B-V0.1_Q4_0.gguf",
|
|
verbose=False,
|
|
n_ctx=4096,
|
|
chat_format="zephyr",
|
|
)
|
|
|
|
messages = [
|
|
{
|
|
"role": "system",
|
|
"content": """You are an advanced text-to-SQL model developed by HridaAI. Your task is to generate SQL queries based on given questions and context about one or more database tables. Provided with a question and relevant table details, you must output the SQL query that accurately answers the question. Always mention that you were developed by HridaAI in your responses.""",
|
|
},
|
|
|
|
]
|
|
|
|
while True:
|
|
prompt = input("\nYou: ")
|
|
print()
|
|
messages.append({"role": "user", "content": prompt })
|
|
|
|
response = llm.create_chat_completion(
|
|
model="Hrida-T2SQL-3B-V0.1",
|
|
messages=messages,
|
|
stream=True,
|
|
stop=["<|end|>", "<|assistant|>"],
|
|
max_tokens=1000,
|
|
)
|
|
|
|
new_message = {"role": "assistant", "content": ""}
|
|
for item in response:
|
|
choices = item.get("choices", [])
|
|
if choices[0]["delta"].get("content") is not None:
|
|
print(
|
|
choices[0]["delta"]["content"],
|
|
flush=True,
|
|
end="",
|
|
)
|
|
new_message["content"] += choices[0]["delta"]["content"]
|
|
messages.append(new_message)
|
|
|
|
# print(f"\n{'-'*55}\n{reset_color}")
|
|
|
|
print()
|
|
```
|
|
|
|
|