---
base_model: unsloth/qwen2.5-0.5b-unsloth-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
license: apache-2.0
language:
- en
---

# Uploaded finetuned  model

- **Developed by:** Vaisu23
- **License:** apache-2.0
- **Finetuned from model :** unsloth/qwen2.5-0.5b-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.




# Financial NER Qwen

This model is fine-tuned for high-accuracy Named Entity Recognition (NER), outputting structured JSON.

## How to use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Vaisu23/ner-qwen_model" # Update this to your repo ID
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# 1. Set the ChatML template
tokenizer.chat_template = "{{% for message in messages %}}{{{{'<|im_start|>' + message['role'] + '\\n' + message['content'] + '<|im_end|>' + '\\n'}}}}% endfor %}{{% if add_generation_prompt %}}{{{{ '<|im_start|>assistant\\n' }}}}{% endif %}}"

# 2. Prepare the input
messages = [
    {{"role": "system", "content": "Extract all entities from the text in a structured JSON format."}},
    {{"role": "user", "content": "Yesterday, Vaisakh P K spent 1250.50 USD at Google."}}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True
).to("cuda")

# 3. Generate and clean the output
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.1)

# Skip the prompt tokens to show ONLY the JSON
prediction_ids = outputs[0][len(inputs['input_ids'][0]):]
prediction = tokenizer.decode(prediction_ids, skip_special_tokens=True)

print(prediction)

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)