qwen3-0.6B-interleaved-thinking/THOUGHT_USE_PROBE.md

Phase 4B Causal Thought-Use Probe Results

Date: 2026-04-27

Direct Conclusion

This is the core mechanistic appendix experiment to use for the final report. The probe shows that thought text is behaviorally causal for suffix reward, but the model's own generated thoughts are not the best available scaffold in this small run.

The cleanest signal is from thought swapping: replacing a prefix's own thought with another prefix's model-generated thought sharply reduces reward across all arms. That means the thought channel carries prefix-specific control information. However, blank or generic thoughts often match or outperform normal generated thoughts, so this does not support a strong claim that RLMT made the model's own thoughts optimally useful.

Important scale context: this is a deliberately small 0.6B-model proxy with only 200 RLMT update steps. The right interpretation is not that RLMT failed in general. It is that, at this scale and training budget, the model learned an emerging thought-conditioned continuation interface but not a mature policy for generating reliably high-utility thoughts.

Source Artifacts

• Probe script: /Users/jarrodbarnes/projects/synthetic-pretrain/scripts/phase4b_thought_use_probe.py
• Remote output dir: spark-f7e2:/home/jarrodbarnes/synthetic-pretrain/outputs/phase4b_mech_interp_thought_use/phase4b-thought-use-core-20260427
• Remote summary: spark-f7e2:/home/jarrodbarnes/synthetic-pretrain/outputs/phase4b_mech_interp_thought_use/phase4b-thought-use-core-20260427/summary.json
• Remote log: spark-f7e2:/home/jarrodbarnes/synthetic-pretrain/outputs/phase4b_mech_interp_thought_use/phase4b-thought-use-core-20260427/probe.log
• Smoke output: spark-f7e2:/home/jarrodbarnes/synthetic-pretrain/outputs/phase4b_mech_interp_thought_use/smoke-thought-use-20260427/summary.json
• Judge server used for final run: spark-cfd0:http://100.113.207.120:30000, served model qwen-judge

Method

Paper-aligned reward object:

prefix -> intervened thought -> predicted suffix -> judge(predicted suffix, true suffix)

The judge only sees and scores the predicted suffix against the true suffix. It does not directly score thought quality.

Arms:

Arm	Checkpoint
think_base	Base+Think SFT
think_phase3	Phase3+Think SFT
think_base_rlmt	Base+Think+RLMT
think_phase3_rlmt	Phase3+Think+RLMT

Conditions:

Condition	Intervention
normal_model_thought	model samples a thought, then suffix is sampled after external </think>
blank_thought	suffix is sampled after an empty thought and external </think>
generic_thought	suffix is sampled after a short neutral scaffold thought
same_arm_swapped_thought	suffix is sampled after another prefix's model-generated thought from the same arm

Run scale:

• 32 held-out prefixes.
• 4 samples per prefix per condition.
• 128 judged suffixes per arm-condition.
• 2,048 total judged suffixes.
• Invalid judge rate: 0.0 for every arm-condition.

Main Metrics

Arm	Condition	Reward mean	Any-success prefix rate	Avg thought words	Avg suffix words
think_base	normal_model_thought	0.1172	0.3438	34.7	89.8
think_base	blank_thought	0.1406	0.4062	0.0	88.2
think_base	generic_thought	0.2031	0.5938	11.5	92.0
think_base	same_arm_swapped_thought	0.0156	0.0625	34.7	87.1
think_phase3	normal_model_thought	0.1250	0.4062	35.7	87.0
think_phase3	blank_thought	0.1172	0.4062	0.0	84.7
think_phase3	generic_thought	0.1953	0.5000	11.5	93.4
think_phase3	same_arm_swapped_thought	0.0234	0.0938	35.7	91.9
think_base_rlmt	normal_model_thought	0.0859	0.2812	33.9	87.1
think_base_rlmt	blank_thought	0.1094	0.3750	0.0	89.4
think_base_rlmt	generic_thought	0.1797	0.4062	11.5	90.5
think_base_rlmt	same_arm_swapped_thought	0.0156	0.0625	33.9	85.3
think_phase3_rlmt	normal_model_thought	0.1094	0.3125	34.1	89.5
think_phase3_rlmt	blank_thought	0.1562	0.4375	0.0	93.5
think_phase3_rlmt	generic_thought	0.1797	0.4375	11.5	92.3
think_phase3_rlmt	same_arm_swapped_thought	0.0156	0.0625	34.1	83.8

Causal Sensitivity

Sensitivity is prefix-level normal_model_thought - intervened_condition. Positive means the model's own thought helped relative to the intervention.

Arm	Normal - blank	Normal - generic	Normal - swapped
think_base	-0.0234	-0.0859	+0.1016
think_phase3	+0.0078	-0.0703	+0.1016
think_base_rlmt	-0.0234	-0.0938	+0.0703
think_phase3_rlmt	-0.0469	-0.0703	+0.0938

Prefix-level win rates for normal_model_thought over swapped:

Arm	Normal better	Swapped better	Tie
think_base	0.3438	0.0625	0.5938
think_phase3	0.4062	0.0312	0.5625
think_base_rlmt	0.2500	0.0312	0.7188
think_phase3_rlmt	0.2500	0.0000	0.7500

Interpretation

The strongest result is that swapped thoughts are consistently harmful. Across all four arms, replacing the local thought with another prefix's thought drops reward mean to 0.0156-0.0234 and any-success prefix rate to 0.0625-0.0938. This is direct behavioral evidence that the thought channel is not merely decorative: mismatched thought content can causally steer suffix generation away from rewarded continuations.

The second result is more sobering. Blank and generic thoughts often perform as well as or better than the model's own sampled thoughts. Generic thoughts are strongest in this run, with reward means 0.1797-0.2031. This suggests that a clean generic scaffold may stabilize suffix generation better than the small model's sampled thoughts, which are noisy and sometimes harmful.

Phase3 has a small pre-RLMT advantage under normal model thoughts: think_phase3 reward mean 0.1250 vs think_base 0.1172, and any-success prefix rate 0.4062 vs 0.3438. After RLMT, think_phase3_rlmt also beats think_base_rlmt under normal model thoughts: 0.1094 vs 0.0859 reward mean. However, RLMT did not increase causal sensitivity to the model's own thoughts. The normal-vs-swapped sensitivity is lower after RLMT in the base lineage and only slightly lower in the Phase3 lineage.

This should be framed against the experimental scale. The paper's thinking-mid-training results use larger models, far more mid-training, and downstream RL post-training. Our run uses Qwen3-0.6B and 200 RLMT steps, so generic scaffolds beating sampled thoughts is better read as evidence of an immature thought policy than as a broad negative result. The positive applied-interp signal is that semantic mismatch in the thought channel reliably damages behavior even at this small scale.

Safe Report Claim

Use this as the applied-mech-interp appendix claim:

In a deliberately small 0.6B setting with only 200 RLMT updates, a causal thought-use probe shows that interleaved thought text is already a real behavioral control surface: swapping in an unrelated thought sharply reduces suffix reward across all Phase 4B arms. However, the model's own sampled thoughts are not yet reliably better than blank or generic scaffolds, and RLMT does not clearly increase dependence on model-generated thoughts at this scale. This supports a cautious interpretation: Phase 4B learned an emerging thought-conditioned continuation interface, but not a fully optimized internal reasoning policy.

Short version for the final report:

Thought use is present but immature. Even a 0.6B model after short RLMT is sensitive to thought content, but it has not yet learned to reliably generate the best thoughts itself.

Claims Not Supported

• Do not claim RLMT made thoughts uniformly more useful.
• Do not claim the model's sampled thoughts are better than generic scaffolding.
• Do not claim a discovered circuit or representation.
• Do not claim causal use of a specific internal activation; this is a causal behavioral intervention on text.
• Do not frame this as a general null result for thinking RLMT; the model scale and 200-step RLMT budget are too small for that claim.

Validation

• Local syntax: python3 -m py_compile scripts/phase4b_thought_use_probe.py
• Local lint: ruff check scripts/phase4b_thought_use_probe.py
• Spark smoke: think_base, 4 prefixes, 2 samples per prefix, 3 conditions; initial run caught judge outage, rerun passed with 0 invalid responses.
• Full Spark run: 4 arms, 32 prefixes, 4 samples per prefix, 4 conditions; completed with 0 invalid judge responses.

Report Placement

This should replace the linear reward-decodability probe in the final research report. The decodability result can be omitted: it was weaker, less causal, and less directly useful to research engineers.