99 lines
3.7 KiB
Markdown
99 lines
3.7 KiB
Markdown
---
|
|
base_model: parth-1/metaguard-policy-agent-v1
|
|
tags:
|
|
- text-generation-inference
|
|
- transformers
|
|
- unsloth
|
|
- llama
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
---
|
|
|
|
# Uploaded finetuned model
|
|
---
|
|
base_model: parth-1/metaguard-policy-agent-v1
|
|
tags:
|
|
- text-generation-inference
|
|
- transformers
|
|
- unsloth
|
|
- llama
|
|
- grpo
|
|
- reinforcement-learning
|
|
- agents
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
---
|
|
|
|
# MetaGuard Ad Policy Agent (v1)
|
|
|
|
- **Developed by:** parth-1
|
|
- **License:** apache-2.0
|
|
- **Finetuned from model:** parth-1/metaguard-policy-agent-v1
|
|
|
|
This is an experimental ad policy compliance agent fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face's TRL library. It is specifically designed to navigate the **MetaGuard Ad Policy Sandbox**, a multi-agent reinforcement learning environment where the model must gather signals and make terminal enforcement decisions.
|
|
|
|
|
|
### Required Parser Patch
|
|
To successfully step the environment without crashing the API, you **must** sanitize the model's output before passing it to the server. Use the following Python drop-in replacement for your parsing logic. This guarantees the required reasoning field is present, enforces strict enums, and strips out hallucinated keys to perfectly satisfy the environment schema.
|
|
|
|
```python
|
|
import json
|
|
|
|
def parse_attempt(text):
|
|
candidates = []
|
|
if "```" in text:
|
|
for p in text.split("```"):
|
|
p = p.strip().lstrip("json").strip()
|
|
candidates.append(p)
|
|
candidates.append(text.strip())
|
|
s, e = text.find("{"), text.rfind("}") + 1
|
|
if s != -1 and e > s:
|
|
candidates.append(text[s:e])
|
|
|
|
for c in candidates:
|
|
try:
|
|
r = json.loads(c)
|
|
if isinstance(r, dict) and "action_type" in r:
|
|
# 1. Guarantee required reasoning
|
|
r.setdefault("reasoning", "No reasoning provided.")
|
|
|
|
# 2. Enforce strict enums for violation_category
|
|
valid_cat = {"HEALTHCARE", "FINANCIAL", "NONE", None}
|
|
if r.get("violation_category") not in valid_cat:
|
|
r.pop("violation_category", None)
|
|
|
|
# 3. Nuke hallucinated extra fields
|
|
allowed = {"action_type", "reasoning", "violation_category", "metadata"}
|
|
return {k: v for k, v in r.items() if k in allowed}
|
|
except Exception:
|
|
continue
|
|
return None
|
|
```
|
|
|
|
## Evaluation Results
|
|
The model was evaluated across a 4-task suite testing healthcare policies, financial policies, multimodal analysis, and targeting constraints (10 steps per episode).
|
|
|
|
**Baseline (Pre-GRPO)**
|
|
* Success Rate: 0/4 (0%)
|
|
* Mean Reward per Step: -0.185
|
|
|
|
**Final (Post-GRPO)**
|
|
* Success Rate: 0/4 (0%)
|
|
* Mean Reward per Step: -0.050
|
|
|
|
*Note: The GRPO fine-tuning successfully improved the agent's intermediate signal gathering phase, significantly pulling the mean reward up from the baseline.*
|
|
|
|
## Limitations
|
|
While the model shows strong capability in the information-gathering phase (e.g., calling `query_regulations` and `analyze_image`), it currently struggles with strict terminal sequence chaining. Specifically, the agent frequently fails to explicitly call the `submit_audit` action before attempting to issue a final `approve` or `reject` decision, resulting in an incomplete task execution.
|
|
|
|
|
|
- **Developed by:** parth-1
|
|
- **License:** apache-2.0
|
|
- **Finetuned from model :** parth-1/metaguard-policy-agent-v1
|
|
|
|
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
|
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|