初始化项目,由ModelHub XC社区提供模型
Model: parth-1/metaguard-policy-agent-v1 Source: Original Platform
This commit is contained in:
98
README.md
Normal file
98
README.md
Normal file
@@ -0,0 +1,98 @@
|
||||
---
|
||||
base_model: parth-1/metaguard-policy-agent-v1
|
||||
tags:
|
||||
- text-generation-inference
|
||||
- transformers
|
||||
- unsloth
|
||||
- llama
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
---
|
||||
|
||||
# Uploaded finetuned model
|
||||
---
|
||||
base_model: parth-1/metaguard-policy-agent-v1
|
||||
tags:
|
||||
- text-generation-inference
|
||||
- transformers
|
||||
- unsloth
|
||||
- llama
|
||||
- grpo
|
||||
- reinforcement-learning
|
||||
- agents
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
---
|
||||
|
||||
# MetaGuard Ad Policy Agent (v1)
|
||||
|
||||
- **Developed by:** parth-1
|
||||
- **License:** apache-2.0
|
||||
- **Finetuned from model:** parth-1/metaguard-policy-agent-v1
|
||||
|
||||
This is an experimental ad policy compliance agent fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face's TRL library. It is specifically designed to navigate the **MetaGuard Ad Policy Sandbox**, a multi-agent reinforcement learning environment where the model must gather signals and make terminal enforcement decisions.
|
||||
|
||||
|
||||
### Required Parser Patch
|
||||
To successfully step the environment without crashing the API, you **must** sanitize the model's output before passing it to the server. Use the following Python drop-in replacement for your parsing logic. This guarantees the required reasoning field is present, enforces strict enums, and strips out hallucinated keys to perfectly satisfy the environment schema.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
def parse_attempt(text):
|
||||
candidates = []
|
||||
if "```" in text:
|
||||
for p in text.split("```"):
|
||||
p = p.strip().lstrip("json").strip()
|
||||
candidates.append(p)
|
||||
candidates.append(text.strip())
|
||||
s, e = text.find("{"), text.rfind("}") + 1
|
||||
if s != -1 and e > s:
|
||||
candidates.append(text[s:e])
|
||||
|
||||
for c in candidates:
|
||||
try:
|
||||
r = json.loads(c)
|
||||
if isinstance(r, dict) and "action_type" in r:
|
||||
# 1. Guarantee required reasoning
|
||||
r.setdefault("reasoning", "No reasoning provided.")
|
||||
|
||||
# 2. Enforce strict enums for violation_category
|
||||
valid_cat = {"HEALTHCARE", "FINANCIAL", "NONE", None}
|
||||
if r.get("violation_category") not in valid_cat:
|
||||
r.pop("violation_category", None)
|
||||
|
||||
# 3. Nuke hallucinated extra fields
|
||||
allowed = {"action_type", "reasoning", "violation_category", "metadata"}
|
||||
return {k: v for k, v in r.items() if k in allowed}
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
```
|
||||
|
||||
## Evaluation Results
|
||||
The model was evaluated across a 4-task suite testing healthcare policies, financial policies, multimodal analysis, and targeting constraints (10 steps per episode).
|
||||
|
||||
**Baseline (Pre-GRPO)**
|
||||
* Success Rate: 0/4 (0%)
|
||||
* Mean Reward per Step: -0.185
|
||||
|
||||
**Final (Post-GRPO)**
|
||||
* Success Rate: 0/4 (0%)
|
||||
* Mean Reward per Step: -0.050
|
||||
|
||||
*Note: The GRPO fine-tuning successfully improved the agent's intermediate signal gathering phase, significantly pulling the mean reward up from the baseline.*
|
||||
|
||||
## Limitations
|
||||
While the model shows strong capability in the information-gathering phase (e.g., calling `query_regulations` and `analyze_image`), it currently struggles with strict terminal sequence chaining. Specifically, the agent frequently fails to explicitly call the `submit_audit` action before attempting to issue a final `approve` or `reject` decision, resulting in an incomplete task execution.
|
||||
|
||||
|
||||
- **Developed by:** parth-1
|
||||
- **License:** apache-2.0
|
||||
- **Finetuned from model :** parth-1/metaguard-policy-agent-v1
|
||||
|
||||
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
||||
|
||||
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
||||
Reference in New Issue
Block a user