base_model, tags, license, language

base_model	tags	license	language
unsloth/qwen2.5-0.5b-unsloth-bnb-4bit	text-generation-inference transformers unsloth qwen2	apache-2.0	en

Uploaded finetuned model

• Developed by: Vaisu23
• License: apache-2.0
• Finetuned from model : unsloth/qwen2.5-0.5b-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Financial NER Qwen

This model is fine-tuned for high-accuracy Named Entity Recognition (NER), outputting structured JSON.

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Vaisu23/ner-qwen_model" # Update this to your repo ID
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# 1. Set the ChatML template
tokenizer.chat_template = "{{% for message in messages %}}{{{{'<|im_start|>' + message['role'] + '\\n' + message['content'] + '<|im_end|>' + '\\n'}}}}% endfor %}{{% if add_generation_prompt %}}{{{{ '<|im_start|>assistant\\n' }}}}{% endif %}}"

# 2. Prepare the input
messages = [
    {{"role": "system", "content": "Extract all entities from the text in a structured JSON format."}},
    {{"role": "user", "content": "Yesterday, Vaisakh P K spent 1250.50 USD at Google."}}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True
).to("cuda")

# 3. Generate and clean the output
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.1)

# Skip the prompt tokens to show ONLY the JSON
prediction_ids = outputs[0][len(inputs['input_ids'][0]):]
prediction = tokenizer.decode(prediction_ids, skip_special_tokens=True)

print(prediction)

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)