DoodDood-TOMAGPT-GGUF/README.md

---
license: apache-2.0
base_model: DoodDood/TOMAGPT
datasets:
  - DoodDood/HearsayGRPOTrainingData2
tags:
  - legal
  - hearsay
  - classification
  - grpo
  - reinforcement-learning
  - legalbench
  - lora
pipeline_tag: text-generation
model-index:
  - name: TOMAGPT
    results:
      - task:
          type: text-classification
          name: Hearsay Classification
        dataset:
          name: LegalBench Hearsay
          type: nguha/legalbench
          split: test
        metrics:
          - type: accuracy
            value: 77.7
            name: Decomposed Accuracy
---

# GGUF Files for TOMAGPT

These are the GGUF files for [DoodDood/TOMAGPT](https://huggingface.co/DoodDood/TOMAGPT).

## Downloads

| GGUF Link | Quantization | Description |
| ---- | ----- | ----------- |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q2_K.gguf) | Q2_K | Lowest quality |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q3_K_S.gguf) | Q3_K_S |  |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.IQ3_S.gguf) | IQ3_S | Integer quant, preferable over Q3_K_S |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.IQ3_M.gguf) | IQ3_M | Integer quant |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q3_K_M.gguf) | Q3_K_M |  |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q3_K_L.gguf) | Q3_K_L |  |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.IQ4_XS.gguf) | IQ4_XS | Integer quant |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q4_K_S.gguf) | Q4_K_S | Fast with good performance |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q4_K_M.gguf) | Q4_K_M | **Recommended:** Perfect mix of speed and performance |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q5_K_S.gguf) | Q5_K_S |  |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q5_K_M.gguf) | Q5_K_M |  |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q6_K.gguf) | Q6_K | Very good quality |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.Q8_0.gguf) | Q8_0 | Best quality |
| [Download](https://huggingface.co/Flexan/DoodDood-TOMAGPT-GGUF/resolve/main/TOMAGPT.f16.gguf) | f16 | Full precision, don't bother; use a quant |

## Note from Flexan

I provide GGUFs and quantizations of publicly available models that do not have a GGUF equivalent available yet.
This process is not yet automated and I download, convert, quantize, and upload them **by hand**, usually for models **I deem interesting and wish to try out**.

If there are some quants missing that you'd like me to add, you may request one in the community tab.
If you want to request a public model to be converted, you can also request that in the community tab.
If you have questions regarding the model, please refer to the original model repo.

# TOMAGPT

A **Qwen3-4B-Instruct-2507** model fine-tuned with GRPO (Group Relative Policy Optimization) to classify legal hearsay by decomposing it into three sub-elements under the U.S. Federal Rules of Evidence.

## What It Does

TOMAGPT classifies whether a statement is hearsay by analyzing three sub-elements:

1. **Assertion** -- Is the statement an assertion?
2. **Out-of-court** -- Was the statement made out of court?
3. **TOMA** -- Is the statement offered to prove the truth of the matter asserted?

Hearsay = YES only if all three sub-elements are YES.

## Results

Evaluated on the [LegalBench hearsay test set](https://huggingface.co/datasets/nguha/legalbench) (94 examples):

| Metric | Base Model | TOMAGPT | Delta |
|--------|-----------|---------|-------|
| **Overall accuracy** | 71.3% | **77.7%** | +6.4% |
| **TOMA sub-element** | 78.0% | **95.1%** | +17.1% |
| Assertion sub-element | 90.2% | 95.1% | +4.9% |
| Non-verbal hearsay | 33.3% | 83.3% | +50.0% |
| Standard hearsay | 93.1% | 100.0% | +6.9% |
| Non-assertive conduct | 89.5% | 100.0% | +10.5% |

## Training Details

- **Method**: GRPO (Group Relative Policy Optimization)
- **Platform**: [Prime Intellect Lab](https://lab.primeintellect.ai)
- **Environment**: `smolclaims/TOMAGPT` (v0.3.0)
- **Base model**: Qwen/Qwen3-4B-Instruct-2507
- **Training data**: [DoodDood/HearsayGRPOTrainingData2](https://huggingface.co/datasets/DoodDood/HearsayGRPOTrainingData2) (3,140 examples)
- **Steps**: 500
- **Learning rate**: 1e-5
- **Batch size**: 128
- **Rollouts per example**: 16

### LoRA Configuration

- **Rank (r)**: 16
- **Alpha**: 32
- **Dropout**: 0.0
- **Target modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

### Reward Functions

| Function | Weight | Description |
|----------|--------|-------------|
| assertion_reward | 1.5 | +1/-1 on assertion accuracy |
| out_of_court_reward | 1.0 | +1/-1 on out-of-court accuracy |
| toma_reward | 2.0 | +1/-1 on TOMA accuracy |
| consistency_penalty | 1.0 | -0.5 for contradictory outputs |
| format_compliance | 1.0 | -0.25 per missing field |
| constraint_penalty | 1.0 | -0.5 for logical violations |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "DoodDood/TOMAGPT", torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("DoodDood/TOMAGPT")

system_prompt = (
    "You are a legal assistant identifying hearsay. Hearsay is defined as "
    "an out-of-court statement introduced to prove the truth of the matter "
    "asserted.\n\n"
    "Respond in EXACTLY this format (semicolon-separated):\n"
    "is_hearsay: YES/NO; an_assertion: YES/NO; made_out_of_court: YES/NO; "
    "is_for_toma: YES/NO"
)

scenario = "At trial, the prosecution presents testimony from a police officer who states that a bystander at the scene told him, 'The defendant ran the red light.'"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": scenario}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)

response = tokenizer.decode(output[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)
# Expected: is_hearsay: YES; an_assertion: YES; made_out_of_court: YES; is_for_toma: YES
```

## Links

- **Training data**: [DoodDood/HearsayGRPOTrainingData2](https://huggingface.co/datasets/DoodDood/HearsayGRPOTrainingData2)
- **GRPO environment**: `smolclaims/TOMAGPT` on [Prime Intellect](https://lab.primeintellect.ai)
- **Eval benchmark**: [nguha/legalbench](https://huggingface.co/datasets/nguha/legalbench) (hearsay subset)