50 lines
1013 B
Markdown
50 lines
1013 B
Markdown
|
# Modifications
|
||
|
|
|
||
|
|
This repository contains a modified derivative of Google's Gemma model.
|
||
|
|
|
||
|
|
## Base Model
|
||
|
|
|
||
|
|
- google/gemma-3-1b-it
|
||
|
|
|
||
|
|
## Training Distribution
|
||
|
|
|
||
|
|
- unsloth/gemma-3-1b-it
|
||
|
|
|
||
|
|
## Modification Type
|
||
|
|
|
||
|
|
Full-parameter supervised fine-tuning (Full FT)
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
|
||
|
|
The model was adapted for Retrieval-Augmented Generation (RAG) as part of
|
||
|
|
the TSFT-RAG research project.
|
||
|
|
|
||
|
|
The fine-tuning objective emphasizes:
|
||
|
|
|
||
|
|
- context-grounded question answering
|
||
|
|
- explicit handling of unsupported questions
|
||
|
|
- structured JSON generation
|
||
|
|
- topic extraction
|
||
|
|
- key-message extraction
|
||
|
|
- argument extraction
|
||
|
|
- citation-aware responses
|
||
|
|
|
||
|
|
## Technical Changes
|
||
|
|
|
||
|
|
The following components were modified:
|
||
|
|
|
||
|
|
- model weights
|
||
|
|
|
||
|
|
The following components remain derived from the original Gemma release:
|
||
|
|
|
||
|
|
- tokenizer
|
||
|
|
- tokenizer configuration
|
||
|
|
- model architecture
|
||
|
|
- vocabulary
|
||
|
|
- chat template
|
||
|
|
|
||
|
|
No architectural changes were made to the original Gemma model.
|
||
|
|
|
||
|
|
The resulting model remains fully compatible with the Hugging Face
|
||
|
|
Transformers ecosystem.
|