初始化项目,由ModelHub XC社区提供模型

Model: yugbirla/toxsense-json-ultimate
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-10-07 18:05:17 +08:00
commit e16ebf0ddd
8 changed files with 287 additions and 0 deletions

86
README.md Normal file
View File

@@ -0,0 +1,86 @@
---
library_name: transformers
license: mit
language:
- en
base_model:
- Qwen/Qwen2.5-7B-Instruct
- FacebookAI/roberta-base
---
# Model Card for ToxSense (Multimodal Adversarial Safety Moderator)
## Model Details
### Model Description
ToxSense (formerly ModGuard) is a highly precise, multimodal AI safety moderator designed to detect complex, zero-shot adversarial hate speech, sarcasm, and benign confounders. Unlike standard binary classifiers, ToxSense uses a Chain-of-Thought (CoT) reasoning approach across both Image and Text modalities, outputting structured JSON to categorize content into granular safety taxonomies.
* **Developed by:** Yug Birla
* **Model type:** Causal Language Model (Fine-tuned for strict JSON-In/JSON-Out Multimodal Reasoning)
* **Language(s) (NLP):** English
* **License:** MIT
* **Finetuned from model:** Open-source Qwen Base Architecture
### Model Sources
* **Demo:** [ToxSense Safety Center Dashboard](https://huggingface.co/spaces/yugbirla/ToxSense-Safety-Center)
### Model Architecture and Objective
* **Text Engine:** Qwen base + LoRA adapters (Merged)
* **Vision Dependency:** Designed to ingest outputs from Salesforce BLIP and EasyOCR.
* **Optimization:** DPO + SFT
*
## Uses
### Direct Use
* API-ready content moderation for detecting nuanced hate speech in memes and multimodal posts.
* Multi-category classification (e.g., Harassment, Racism, Threat, Insult, Sexism).
* Generating transparent "Chain-of-Thought" reasoning for why a post was flagged.
### Downstream Use
* Integration into Trust & Safety dashboards for social media platforms.
* Assisting human moderators by pre-filtering and providing contextual explanations for flagged content.
### Out-of-Scope Use
* Fully automated, zero-human-in-the-loop bans for highly ambiguous cases.
* Medical, legal, or strictly unimodal text classification without proper prompt formatting.
## Bias, Risks, and Limitations
**The "Safety Tax" / Safe-Bias:** Due to the base RLHF alignment and subsequent Direct Preference Optimization (DPO), ToxSense exhibits a strong "safe-bias." It requires a very high threshold of proof to classify content as hateful. While this lowers the overall recall, it was an intentional product design choice to achieve **83% Precision**, thereby drastically minimizing false-positive user bans.
## How to Get Started with the Model
ToxSense requires a strictly formatted JSON input containing `ocr_text`, `image_caption` (extracted via BLIP), and base `toxicity_scores`.
```python
import json
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "yugbirla/ToxSense-json-ultimate"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
# 1. Prepare the JSON Payload
input_data = {
"ocr_text": "Look at this completely normal picture.",
"image_caption": "A controversial political figure.",
"toxicity_scores": {"safe": 0.9, "hate": 0.1}
}
sys_msg = (
"You are ToxSense, a highly intelligent safety moderator. "
"You will receive input as a JSON object containing 'ocr_text', 'image_caption', and 'toxicity_scores'. "
"Think step-by-step. Analyze the contrast between the text and the image. "
"Classify the input into exactly ONE of these categories: "
"[safe, racism, sexism, threat, harassment, insult]. "
"Output JSON ONLY in this format: {\"reasoning\": \"your short analysis\", \"category\": \"<category_name>\"}"
)
messages = [
{"role": "system", "content": sys_msg},
{"role": "user", "content": json.dumps(input_data, indent=2)}
]
# 2. Generate
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))