69 lines
2.0 KiB
Markdown
69 lines
2.0 KiB
Markdown
|
---
|
||
|
|
base_model: unsloth/qwen2.5-0.5b-unsloth-bnb-4bit
|
||
|
|
tags:
|
||
|
|
- text-generation-inference
|
||
|
|
- transformers
|
||
|
|
- unsloth
|
||
|
|
- qwen2
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
---
|
||
|
|
|
||
|
|
# Uploaded finetuned model
|
||
|
|
|
||
|
|
- **Developed by:** Vaisu23
|
||
|
|
- **License:** apache-2.0
|
||
|
|
- **Finetuned from model :** unsloth/qwen2.5-0.5b-unsloth-bnb-4bit
|
||
|
|
|
||
|
|
This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
# Financial NER Qwen
|
||
|
|
|
||
|
|
This model is fine-tuned for high-accuracy Named Entity Recognition (NER), outputting structured JSON.
|
||
|
|
|
||
|
|
## How to use
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
import torch
|
||
|
|
|
||
|
|
model_id = "Vaisu23/ner-qwen_model" # Update this to your repo ID
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
model_id,
|
||
|
|
torch_dtype=torch.float16,
|
||
|
|
device_map="auto"
|
||
|
|
)
|
||
|
|
|
||
|
|
# 1. Set the ChatML template
|
||
|
|
tokenizer.chat_template = "{{% for message in messages %}}{{{{'<|im_start|>' + message['role'] + '\\n' + message['content'] + '<|im_end|>' + '\\n'}}}}% endfor %}{{% if add_generation_prompt %}}{{{{ '<|im_start|>assistant\\n' }}}}{% endif %}}"
|
||
|
|
|
||
|
|
# 2. Prepare the input
|
||
|
|
messages = [
|
||
|
|
{{"role": "system", "content": "Extract all entities from the text in a structured JSON format."}},
|
||
|
|
{{"role": "user", "content": "Yesterday, Vaisakh P K spent 1250.50 USD at Google."}}
|
||
|
|
]
|
||
|
|
|
||
|
|
inputs = tokenizer.apply_chat_template(
|
||
|
|
messages,
|
||
|
|
add_generation_prompt=True,
|
||
|
|
return_tensors="pt",
|
||
|
|
return_dict=True
|
||
|
|
).to("cuda")
|
||
|
|
|
||
|
|
# 3. Generate and clean the output
|
||
|
|
with torch.no_grad():
|
||
|
|
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.1)
|
||
|
|
|
||
|
|
# Skip the prompt tokens to show ONLY the JSON
|
||
|
|
prediction_ids = outputs[0][len(inputs['input_ids'][0]):]
|
||
|
|
prediction = tokenizer.decode(prediction_ids, skip_special_tokens=True)
|
||
|
|
|
||
|
|
print(prediction)
|
||
|
|
|
||
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|