FrameLLaMA-3.1-8B-Instruct-FullFN17/README.md

---
base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
license: apache-2.0
language:
- en
---

# Uploaded  model

- **License:** apache-2.0
- **Finetuned from model :** unsloth/meta-llama-3.1-8b-instruct-bnb-4bit

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
from unsloth import is_bfloat16_supported

# Precision
dtype = torch.bfloat16 if is_bfloat16_supported() else torch.float16

# Models
BASE_MODEL = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit"
LORA_MODEL = "sag-uniroma2/FrameLLaMA-3.1-8B-Instruct-FullFN17"

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=dtype,
    device_map="auto"
)

# Load LoRA
model = PeftModel.from_pretrained(base_model, LORA_MODEL)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(LORA_MODEL, use_fast=True)

# Device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

# 🔥 ===== YOUR SAMPLE HERE =====
premise = "John drowned Martha."
hypothesis = "Martha died."

# Prompt
input_text = f"""
        Judge if the hypothesis necessarily follows from the premise.
        Consider the truth value of the premise. If the premise is true, does it necessarily mean that the hypothesis must also be true?
        Output E if the hypothesis must always be true.
        Output C if the hypothesis must always be false.
        Output N if the hypothesis may be either true or false.
        Do not output anything other than letters E, C, or N.

        Premise: {premise}
        Hypothesis: {hypothesis}
        # Output:"""

# Tokenize
inputs = tokenizer(input_text, return_tensors="pt").to(device)

# Generate
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=10,
        do_sample=False
    )

# Decode only generated part
input_len = inputs["input_ids"].shape[1]
generated = output_ids[0][input_len:]
response = tokenizer.decode(generated, skip_special_tokens=True).strip()

# Print result
print("Premise:", premise)
print("Hypothesis:", hypothesis)
print("Prediction:", response)
```
## Description

**FrameLLaMA-3.1-8B-Instruct-FullFN17** is a frame-aware language model designed to improve event-level semantic reasoning in Large Language Models (LLMs). The model injects structured knowledge from FrameNet 1.7 into Llama-3.1-8B-Instruct using parameter-efficient LoRA fine-tuning. 

Unlike standard instruction tuning, this model leverages **principle-oriented supervision**, where frame definitions, participant roles, semantic types, lexical senses, and frame-to-frame relations are converted into structured question–answer tasks. This enables the model to learn reusable semantic constraints rather than isolated facts.

The model is optimized for tasks where meaning depends on **event structure, participant roles, and lexical disambiguation**, such as Natural Language Inference (NLI) and Semantic Role Labeling (SRL).

---

## Model Details

- **Base Model**: Llama-3.1-8B-Instruct  
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)  
- **Training Data**: FrameNet 1.7 (full inventory, 1,200+ frames)  
- **Supervision Type**: Principle-oriented QA-style prompts  
- **Tasks**: NLI, SRL (evaluation), semantic reasoning  
- **Model Type**: Instruction-tuned causal language model  

---

## Key Features

- ✅ **Full FrameNet Coverage**: Trained on 1,200+ frames.  
- ✅ **Principle-Oriented Learning**: Encodes role constraints, semantic types, and frame relations  
- ✅ **Event-Level Reasoning**: Improves understanding of causality, entailment, and contradiction  
- ✅ **Frame-Aware Inference**: Better handling of lexical ambiguity and role compatibility  
- ✅ **Parameter-Efficient Training**: Uses LoRA for scalable adaptation  
- ✅ **Generalization Beyond SRL**: Transfers to NLI and semantic inference tasks  

---

## Performance

- Evaluated on:
  - **SNLI (diagnostic subset)** for event-level inference  
  - **CONLL-style FrameNet SRL dataset** (via OpenSesame preprocessing)  

- Observed improvements:
  - Strong gains in **entailment and contradiction detection**  
  - Improved **frame identification and role-span alignment** in SRL  
  - Reduced reliance on surface-level lexical cues  

---

## Use Cases

- **Natural Language Inference (NLI)**: Event-based reasoning and entailment detection  
- **Semantic Role Labeling (SRL)**: Frame and role prediction  
- **Event Understanding**: Modeling causality and participant structure  
- **Linguistically-Informed AI**: Applications requiring structured semantic interpretation  
- **Research on LLM Interpretability**: Studying structured knowledge injection  

---

## Output Format

- Single token or short response:
  - **E** → Entailment  
  - **C** → Contradiction  
  - **N** → Neutral  

- Outputs are concise and reflect event-level semantic reasoning.

---

## Training Details

- FrameNet structures (definitions, roles, relations) are **linearized into QA-style templates**
- Supervision includes:
  - Frame definitions  
  - Role constraints  
  - Semantic types  
  - Lexical unit disambiguation  
  - Frame-to-frame relations  
- Negative samples generated via similarity-based filtering  
- Fine-tuned using LoRA for efficiency and scalability  

---

## GitHub

For training scripts, datasets, and evaluation:

👉 https://github.com/crux82/FrameLLaMA

## Citation

If you use this model, please cite: