初始化项目,由ModelHub XC社区提供模型
Model: Devildarker6789/qwen2vl-chartqa Source: Original Platform
This commit is contained in:
68
README.md
Normal file
68
README.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
language: en
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen2-VL-2B-Instruct
|
||||
datasets:
|
||||
- HuggingFaceM4/ChartQA
|
||||
tags:
|
||||
- vision-language
|
||||
- chart-qa
|
||||
- lora
|
||||
- multimodal
|
||||
---
|
||||
|
||||
# Qwen2-VL-2B Fine-tuned on ChartQA
|
||||
|
||||
Fine-tuned version of Qwen2-VL-2B-Instruct on the ChartQA dataset for visual question answering on charts.
|
||||
|
||||
## Model Details
|
||||
- **Base model:** Qwen/Qwen2-VL-2B-Instruct
|
||||
- **Fine-tuning:** LoRA (r=16, alpha=32, target: q/k/v/o projections)
|
||||
- **Dataset:** HuggingFaceM4/ChartQA (2000 train, 200 val samples)
|
||||
- **Training:** 1 epoch, lr=2e-4, T4 GPU on Kaggle
|
||||
- **Results:** Train Loss: 0.5040 | Val Loss: 0.6956
|
||||
|
||||
## How to Use
|
||||
```python
|
||||
from transformers import AutoProcessor, Qwen2VLForConditionalGeneration
|
||||
from PIL import Image
|
||||
import torch
|
||||
|
||||
model_id = "Devildarker6789/qwen2vl-chartqa"
|
||||
|
||||
processor = AutoProcessor.from_pretrained(model_id, min_pixels=256*28*28, max_pixels=512*28*28)
|
||||
model = Qwen2VLForConditionalGeneration.from_pretrained(
|
||||
model_id,
|
||||
torch_dtype=torch.float16,
|
||||
device_map="auto"
|
||||
)
|
||||
model.eval()
|
||||
|
||||
image = Image.open("your_chart.png").convert("RGB")
|
||||
question = "What is the highest value in this chart?"
|
||||
|
||||
messages = [{"role": "user", "content": [
|
||||
{"type": "image", "image": image},
|
||||
{"type": "text", "text": question}
|
||||
]}]
|
||||
|
||||
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
|
||||
|
||||
with torch.no_grad():
|
||||
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
|
||||
|
||||
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Training Details
|
||||
- Quantization: 8-bit during training
|
||||
- LoRA rank: 16, alpha: 32, dropout: 0.05
|
||||
- Optimizer: AdamW, lr=2e-4, cosine scheduler
|
||||
- Batch size: 1 × 16 grad accumulation = 16 effective
|
||||
- Hardware: T4 GPU (Kaggle)
|
||||
```
|
||||
|
||||
Then click **Save** — and your HF link is:
|
||||
```
|
||||
https://huggingface.co/Devildarker6789/qwen2vl-chartqa
|
||||
Reference in New Issue
Block a user