---
language:
  - en
license: other
base_model: Qwen/Qwen2.5-1.5B-Instruct
library_name: transformers
pipeline_tag: text-generation
tags:
  - crca
  - causal-reasoning
  - qwen2
  - 1.5b
  - finetuned
---

# CRCA 1.5B Full Finetune

## Overview
CR-CA (Causal Reasoning and Counterfactual Analysis) is a reasoning-focused stack
that targets structured causal analysis, counterfactuals, and multi-step reasoning.
This 1.5B model is a CR-CA reasoning-optimized causal language model based on the
Qwen2 architecture (`Qwen2ForCausalLM`).

## Model Details
- **Model type:** `qwen2`
- **Architecture:** `Qwen2ForCausalLM`
- **Hidden size:** `1536`
- **Layers:** `28`
- **Attention heads:** `12` (KV heads: `2`)
- **Max position embeddings:** `32768`
- **Vocab size:** `151936`
- **Dtype:** `float16`

## Training Summary
This model was produced via full finetuning for CR-CA reasoning. Training metadata
is stored in `training_args.bin`.

Key training parameters:
- **Per-device batch size:** 8
- **Gradient accumulation:** 16
- **Epochs:** 2
- **Learning rate:** 5e-4
- **Precision:** FP16
- **DeepSpeed config:** `training/deepspeed_zero2_1_5b.json`
- **Scheduler:** cosine
- **Warmup steps:** 100
- **Save steps:** 200

## Training Data
The training data uses a prompt/response JSONL format:

```
{"prompt": "...", "response": "..."}
```

The dataset includes public reasoning data (e.g., GSM8K-style math word problems).
This is used to strengthen multi-step reasoning, structured derivations, and final
answer formatting.

## Evaluation Report (Real-World Causal Tasks)
Evaluation was run on 2026-02-01 using GPT-4o-mini over 6 real-world causal tasks.
Overall score: **48.3%**.

Per-task scores:
- Monetary Policy Counterfactual (US Macro 2025): **55/100**
- Tariff Pass-Through and Pricing (Beige Book + Firm Data): **55/100**
- Supply Chain Reroute Counterfactual (Port Disruption): **45/100**
- Inventory & Stockout Causal Impact (Retail): **25/100**
- Inflation Drivers (World Bank CPI Data): **65/100**
- Workforce Training Program (Labor Market Causal Impact): **45/100**

Key strengths observed:
- Clear task framing and attempt at counterfactual reasoning.
- Some identification of confounders and causal factors.

Key limitations observed:
- Inconsistent causal graphs and directional effects.
- Weak counterfactual grounding and numerical reasoning errors.
- Limited depth and rigor on confounder adjustment strategies.

## Intended Use
For causal reasoning, counterfactual analysis, structured CR-CA reasoning prompts,
and multi-step reasoning tasks.

## Generation Settings
Default generation parameters are stored in `generation_config.json`:
- `do_sample`: `true`
- `temperature`: `0.7`
- `top_p`: `0.8`
- `top_k`: `20`
- `repetition_penalty`: `1.1`

## Limitations
- Outputs should be validated for factual correctness.
- The model may hallucinate causal claims without evidence.

## License
Follow the base model and dataset licenses used for training. Add your explicit
license here if required.