初始化项目，由ModelHub XC社区提供模型

Model: ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL-GGUF-q4
Source: Original Platform

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+unsloth.F16.gguf filter=lfs diff=lfs merge=lfs -text
+unsloth.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,160 @@
+---
+base_model: ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL
+language:
+- en
+license: apache-2.0
+tags:
+- text-generation-inference
+- transformers
+- ruslanmv
+- llama
+- gguf
+---
+# Meta-Llama-3.1-8B-Text-to-SQL-GGUF-q4
+
+This model is a fine-tuned version of [ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL](https://huggingface.co/ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL) for Text-to-SQL generation. It is designed to convert natural language queries into SQL commands, optimized for efficient inference using GGUF (Grouped Quantization for Uniform Format).
+
+## Model Details
+
+- **Base Model**: [ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL](https://huggingface.co/ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL)
+- **Task**: Text-to-SQL generation
+- **Quantization**: GGUF (Q4, 4-bit quantization)
+- **License**: Apache-2.0
+
+## Installation
+
+To use this model, you need to install `llama-cpp-python` and `huggingface_hub` for downloading and running the quantized model.
+
+### Step 1: Install Required Packages
+
+```bash
+# Install llama-cpp-python from the appropriate repository
+!pip install llama-cpp-python \
+  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/12.1 \
+  --force-reinstall --upgrade --no-cache-dir --verbose
+
+# Install huggingface_hub to download models from Hugging Face
+!pip install huggingface_hub hf_transfer
+```
+
+### Step 2: Set up Hugging Face Hub and Download the Model
+
+Ensure that Hugging Face's transfer feature is enabled and download the quantized model from Hugging Face using the `huggingface-cli`.
+
+```python
+import os
+os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"
+
+!huggingface-cli download \
+ ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL-GGUF-q4 \
+ unsloth.Q4_K_M.gguf \
+ --local-dir . \
+ --local-dir-use-symlinks False
+```
+
+Make sure the downloaded model is stored in the local directory. Set the model path as follows:
+
+```python
+MODEL_PATH = "/content/unsloth.Q4_K_M.gguf"
+```
+
+## Usage Example
+
+Here is an example that demonstrates how to generate an SQL query from a natural language prompt using the quantized GGUF model and the `llama_cpp` library.
+
+### Step 1: Define the User Query and Prompt
+
+The user provides a natural language query, and we format the prompt using an Alpaca-style template.
+
+```python
+user_query = "Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"
+
+alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+
+### Instruction:
+{}
+
+### Input:
+{}
+
+### Response:
+"""
+
+prompt = alpaca_prompt.format(
+    "Provide the SQL query",
+    user_query
+)
+```
+
+### Step 2: Load the Model and Generate SQL Query
+
+To load the quantized model and perform inference, you will need the `llama_cpp` library.
+
+```python
+from llama_cpp import Llama
+import os
+
+# Get the current directory
+current_directory = os.getcwd()
+
+# Construct the full model path
+MODEL_PATH = os.path.join(current_directory, "unsloth.Q4_K_M.gguf")
+
+# Ensure the model path exists
+assert os.path.exists(MODEL_PATH), f"Model path {MODEL_PATH} does not exist."
+
+# Create the prompt for SQL query generation
+B_INST, E_INST = "<s>[INST]", "[/INST]"
+B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
+DEFAULT_SYSTEM_PROMPT = """\
+Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+"""
+SYSTEM_PROMPT = B_SYS + DEFAULT_SYSTEM_PROMPT + E_SYS
+
+def create_prompt(user_query):
+    instruction = f"Provide the SQL query. User asks: {user_query}\n"
+    prompt = B_INST + SYSTEM_PROMPT + instruction + E_INST
+    return prompt.strip()
+
+# Define user query
+user_query = "Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"  
+prompt = create_prompt(user_query)
+print(f"Prompt created:\n{prompt}")
+
+# Load the model
+try:
+    llm = Llama(model_path=MODEL_PATH, n_gpu_layers=1)  # Adjust GPU layers as per your hardware
+except AssertionError as e:
+    raise RuntimeError(f"Failed to load the model. Check that the model is in the correct format: {e}")
+
+# Perform inference
+try:
+    result = llm(
+        prompt=prompt,
+        max_tokens=200,
+        echo=False
+    )
+    print(result['choices'][0]['text'])
+except Exception as e:
+    print(f"Error during inference: {e}")
+```
+
+### Expected Output
+
+The model will return the following SQL query:
+
+```sql
+SELECT * FROM table1 WHERE anni = 2020
+```
+
+### Additional Notes
+
+- **Quantization**: The model is quantized using GGUF to enable efficient inference, especially on systems with limited memory.
+- **Prompt**: The prompt follows an Alpaca instruction style, which helps guide the model in generating SQL queries based on user input.
+- **Inference**: The `llama_cpp` library is used to perform inference with this GGUF model. Adjust `n_gpu_layers` and `max_tokens` based on your hardware capabilities and the complexity of the SQL query.
+
+## License
+
+This model is released under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) license.
+
+For more detailed information, visit the [model card on Hugging Face](https://huggingface.co/ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL).

config.json

@@ -0,0 +1,3 @@
+{
+    "model_type": "llama"
+}

unsloth.F16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e6ac362ac3e4081191245e9b81406b9b240b8fd0262a90d7b2357c1e3d79692
+size 16068891008

unsloth.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba1a48de32b0de97847c6469480e722c14c6eee429d59557e64db5a2c213ce06
+size 4920734080