Files

ModelHub XC 06f303562a 初始化项目，由ModelHub XC社区提供模型

Model: kartikraut09/ecocloud-grpo-qwen
Source: Original Platform

2026-06-16 05:32:16 +08:00

3.7 KiB

Raw Blame History

license, language, tags, base_model, pipeline_tag

license

language

⚡ CloudEdge GRPO Controller

A Qwen2.5-0.5B model fine-tuned with Group Relative Policy Optimization (GRPO) to manage cloud infrastructure crises.

Built for the Meta PyTorch OpenEnv Hackathon Grand Finale.

Model Description

This model is a reinforcement-learning-trained controller for the CloudEdge cloud crisis simulator. It learns to select optimal infrastructure actions (scale_up, scale_down, optimize_energy, migrate_region) by balancing three competing objectives:

Objective	Target	Agent
Latency	< 150ms	ResourceAgent
Cost	< $400/hr	CostAgent
Carbon	< 220 units	SustainabilityAgent

Training Method

Algorithm: GRPO (Group Relative Policy Optimization) via TRL
Base Model: Qwen/Qwen2.5-0.5B-Instruct
Training Steps: 512
Generations per prompt: 4
Reward Function: Shaped multi-objective reward with gap closure + worst-metric bonus

Shaped Reward Function

reward = Σ (gap_closure × weight) + worst_metric_bonus

Action	Reward (crisis state)	Model Learns
`optimize_energy`	+7.5	"Best action — addresses cost + carbon simultaneously"
`scale_down`	+5.75	"Good — reduces cost effectively"
`migrate_region`	+3.75	"Moderate — helps carbon but hurts cost"
`scale_up`	+1.5	"Worst — increases cost and carbon"

Training Results

The model converged to selecting optimize_energy as the dominant policy when all metrics are above target — which is the mathematically optimal action given the shaped reward function.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("kartikraut09/ecocloud-grpo-qwen")
tokenizer = AutoTokenizer.from_pretrained("kartikraut09/ecocloud-grpo-qwen")

prompt = """<|im_start|>system
You are the CloudEdge controller managing a cloud platform in crisis.
Pick the BEST single action for the current state. Respond with ONLY the action name.

Actions:
  scale_up        → latency -40, cost +30, carbon +20
  scale_down      → latency +25, cost -35, carbon -15
  optimize_energy → latency +10, cost -20, carbon -40
  migrate_region  → latency +15, cost +10, carbon -50

Targets: latency<150ms, cost<$400, carbon<220<|im_end|>
<|im_start|>user
Cloud state: latency=280ms, cost=$620/hr, carbon=380, load=critical. Best action?<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=16, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# Output: optimize_energy

Technical Details

Architecture: Qwen2 (0.5B parameters)
Framework: PyTorch + HuggingFace Transformers + TRL
Environment: OpenEnv-compatible Gymnasium-style simulator
Training Hardware: Google Colab T4 GPU
Training Time: ~15 minutes (512 steps)

Project Links

GitHub: KartikRaut09/ecocloud-war-room
Hackathon: Meta PyTorch OpenEnv Hackathon Grand Finale
Themes: Multi-Agent Interactions · Long-Horizon Planning · World Modeling

Citation

@misc{cloudedge2026,
  title={CloudEdge: Multi-Agent LLM Simulator for Sustainable Cloud Crisis Management},
  author={Kartik Raut},
  year={2026},
  url={https://github.com/KartikRaut09/ecocloud-war-room}
}

License

MIT License

3.7 KiB Raw Blame History Unescape Escape