---
language:
  - en
license: other
base_model: Qwen/Qwen2.5-1.5B-Instruct
library_name: transformers
pipeline_tag: text-generation
tags:
  - crca
  - causal-reasoning
  - qwen2
  - 1.5b
  - finetuned
---

# CRCA 1.5B Full Finetune

## Overview
CR-CA (Causal Reasoning and Counterfactual Analysis) is a reasoning-focused stack
that targets structured causal analysis, counterfactuals, and multi-step reasoning.
This 1.5B model is a CR-CA reasoning-optimized causal language model based on the
Qwen2 architecture (`Qwen2ForCausalLM`).

## Model Details
- **Model type:** `qwen2`
- **Architecture:** `Qwen2ForCausalLM`
- **Hidden size:** `1536`
- **Layers:** `28`
- **Attention heads:** `12` (KV heads: `2`)
- **Max position embeddings:** `32768`
- **Vocab size:** `151936`
- **Dtype:** `float16`

## Training Summary
This model was produced via full finetuning for CR-CA reasoning. Training metadata
is stored in `training_args.bin`.

Key training parameters:
- **Per-device batch size:** 8
- **Gradient accumulation:** 16
- **Epochs:** 2
- **Learning rate:** 5e-4
- **Precision:** FP16
- **DeepSpeed config:** `training/deepspeed_zero2_1_5b.json`
- **Scheduler:** cosine
- **Warmup steps:** 100
- **Save steps:** 200

## Training Data
The training data uses a prompt/response JSONL format:

```
{"prompt": "...", "response": "..."}
```

The dataset includes public reasoning data (e.g., GSM8K-style math word problems).
This is used to strengthen multi-step reasoning, structured derivations, and final
answer formatting.

## Intended Use
For causal reasoning, counterfactual analysis, structured CR-CA reasoning prompts,
and multi-step reasoning tasks.

## Generation Settings
Default generation parameters are stored in `generation_config.json`:
- `do_sample`: `true`
- `temperature`: `0.7`
- `top_p`: `0.8`
- `top_k`: `20`
- `repetition_penalty`: `1.1`

## Limitations
- Outputs should be validated for factual correctness.
- The model may hallucinate causal claims without evidence.

## License
Follow the base model and dataset licenses used for training. Add your explicit
license here if required.