初始化项目,由ModelHub XC社区提供模型
Model: Sinestro38/qwen3-1p7b-sgd-polaris-step300-best-val Source: Original Platform
This commit is contained in:
33
reward_score_box_or_minerva.py
Normal file
33
reward_score_box_or_minerva.py
Normal file
@@ -0,0 +1,33 @@
|
||||
"""
|
||||
Reward function that tries `\boxed{}` extraction first, then Minerva-style "Answer: X".
|
||||
|
||||
Why: verl's default math_dapo.compute_score uses Minerva regex (looks for literal
|
||||
"Answer: X" in the last 300 chars). That works for base/raw models that ramble in
|
||||
prose, but FAILS for distilled CoT models like DS-R1-Distill that emit
|
||||
`<think>...</think>\\boxed{X}` with no "Answer:" preamble. Matching both styles
|
||||
unblocks the SGD-on-DS-R1-Distill experiment without regressing the 1.7B-Base run.
|
||||
|
||||
Selected via verl's custom_reward_function config:
|
||||
custom_reward_function.path=/path/to/this/file.py
|
||||
custom_reward_function.name=compute_score
|
||||
|
||||
Returns a bare float (1.0 or -1.0) — NOT the {score, acc, pred} dict that
|
||||
math_dapo returns natively. verl's validation metric aggregator can't mean a
|
||||
field of None values (which `pred` becomes when no box is found), so the dict
|
||||
form crashes process_validation_metrics. The bare-float form sidesteps this.
|
||||
"""
|
||||
|
||||
from verl.utils.reward_score import math_dapo
|
||||
|
||||
|
||||
def compute_score(data_source, solution_str, ground_truth, extra_info=None, **kwargs):
|
||||
"""Box-first, Minerva-fallback reward. Returns 1.0 (correct) or -1.0 (wrong)."""
|
||||
boxed = math_dapo.compute_score(solution_str, ground_truth, strict_box_verify=True)
|
||||
if boxed["score"] == 1.0:
|
||||
return 1.0
|
||||
|
||||
minerva = math_dapo.compute_score(solution_str, ground_truth, strict_box_verify=False)
|
||||
if minerva["score"] == 1.0:
|
||||
return 1.0
|
||||
|
||||
return -1.0
|
||||
Reference in New Issue
Block a user