---
base_model: parth-1/metaguard-policy-agent-v1
tags:
- text-generation-inference
- transformers
- unsloth
- llama
license: apache-2.0
language:
- en
---

# Uploaded finetuned  model
---
base_model: parth-1/metaguard-policy-agent-v1
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- grpo
- reinforcement-learning
- agents
license: apache-2.0
language:
- en
---

# MetaGuard Ad Policy Agent (v1)

- **Developed by:** parth-1
- **License:** apache-2.0
- **Finetuned from model:** parth-1/metaguard-policy-agent-v1

This is an experimental ad policy compliance agent fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face's TRL library. It is specifically designed to navigate the **MetaGuard Ad Policy Sandbox**, a multi-agent reinforcement learning environment where the model must gather signals and make terminal enforcement decisions.


### Required Parser Patch
To successfully step the environment without crashing the API, you **must** sanitize the model's output before passing it to the server. Use the following Python drop-in replacement for your parsing logic. This guarantees the required reasoning field is present, enforces strict enums, and strips out hallucinated keys to perfectly satisfy the environment schema.

```python
import json

def parse_attempt(text):
    candidates = []
    if "```" in text:
        for p in text.split("```"):
            p = p.strip().lstrip("json").strip()
            candidates.append(p)
    candidates.append(text.strip())
    s, e = text.find("{"), text.rfind("}") + 1
    if s != -1 and e > s:
        candidates.append(text[s:e])
        
    for c in candidates:
        try:
            r = json.loads(c)
            if isinstance(r, dict) and "action_type" in r:
                # 1. Guarantee required reasoning
                r.setdefault("reasoning", "No reasoning provided.")
                
                # 2. Enforce strict enums for violation_category
                valid_cat = {"HEALTHCARE", "FINANCIAL", "NONE", None}
                if r.get("violation_category") not in valid_cat:
                    r.pop("violation_category", None)
                    
                # 3. Nuke hallucinated extra fields
                allowed = {"action_type", "reasoning", "violation_category", "metadata"}
                return {k: v for k, v in r.items() if k in allowed}
        except Exception:
            continue
    return None
```

## Evaluation Results
The model was evaluated across a 4-task suite testing healthcare policies, financial policies, multimodal analysis, and targeting constraints (10 steps per episode).

**Baseline (Pre-GRPO)**
* Success Rate: 0/4 (0%)
* Mean Reward per Step: -0.185

**Final (Post-GRPO)**
* Success Rate: 0/4 (0%)
* Mean Reward per Step: -0.050

*Note: The GRPO fine-tuning successfully improved the agent's intermediate signal gathering phase, significantly pulling the mean reward up from the baseline.*

## Limitations
While the model shows strong capability in the information-gathering phase (e.g., calling `query_regulations` and `analyze_image`), it currently struggles with strict terminal sequence chaining. Specifically, the agent frequently fails to explicitly call the `submit_audit` action before attempting to issue a final `approve` or `reject` decision, resulting in an incomplete task execution. 


- **Developed by:** parth-1
- **License:** apache-2.0
- **Finetuned from model :** parth-1/metaguard-policy-agent-v1

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)