45 lines
1.6 KiB
Markdown
45 lines
1.6 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: Qwen/Qwen3-0.6B-Base
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- lora
|
||
|
|
- qlora
|
||
|
|
- fine-tuned
|
||
|
|
- text-generation
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# qwen3-0.6b-promql
|
||
|
|
|
||
|
|
A small language model fine-tuned with QLoRA SFT on top of `Qwen/Qwen3-0.6B-Base` for an instruction -> rewrite task.
|
||
|
|
|
||
|
|
Training code, data, and benchmarks: [https://github.com/lajosbencz/lfm-train](https://github.com/lajosbencz/lfm-train)
|
||
|
|
|
||
|
|
## Prompt format
|
||
|
|
|
||
|
|
- System prompt: `You are a PromQL optimization expert. Given a PromQL expression and an instruction, rewrite the expression according to the instruction. Output only the improved PromQL expression, no explanation.`
|
||
|
|
- Input label: `Query`
|
||
|
|
|
||
|
|
## Usage (transformers)
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model_id = "lazos/qwen3-0.6b-promql"
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
|
||
|
|
|
||
|
|
messages = [
|
||
|
|
{"role": "system", "content": "You are a PromQL optimization expert. Given a PromQL expression and an instruction, rewrite the expression according to the instruction. Output only the improved PromQL expression, no explanation."},
|
||
|
|
{"role": "user", "content": "<your instruction>\n\nQuery:\n<your input>"},
|
||
|
|
]
|
||
|
|
inputs = tokenizer.apply_chat_template(
|
||
|
|
messages, add_generation_prompt=True, return_tensors="pt"
|
||
|
|
).to(model.device)
|
||
|
|
out = model.generate(inputs, max_new_tokens=256)
|
||
|
|
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
A Q4_K_M GGUF quantization is included for use with llama.cpp / Ollama.
|