---
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation
- metadata-localization
- chat
- without-metadata
- sft
- lora-merged
---

# combined_without_metadata_chat

## Summary

This repo contains the merged chat model for the combined without metadata branch of the metadata localization project. It was produced by supervised fine-tuning on the project QA benchmark after project pretraining.

## Variant Metadata

- Stage: `sft_chat`
- Family: `chat`
- Metadata condition: `without_metadata`
- Base model lineage: `combined_without_metadata_1b`

## Weights & Biases Provenance

- No matching W&B run was resolved automatically.

## SFT Notes

- Fine-tuning method: `PEFT / LoRA`
- Optimizer: `adamw_bnb_8bit`
- `bf16=True`, `gradient_checkpointing=True`, `use_liger_kernel=True`
- `per_device_train_batch_size=2`, `gradient_accumulation_steps=8`
- LoRA targets: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`

## Project Context

This model is part of the metadata localization release. Related checkpoints and variants are grouped in the public Hugging Face collection [Metadata Conditioned LLMs](https://huggingface.co/collections/iamshnoo/metadata-conditioned-llms).
- Training data source: [News on the Web (NOW) Corpus](https://www.english-corpora.org/now/)
- Project repository: [https://github.com/iamshnoo/metadata_localization](https://github.com/iamshnoo/metadata_localization)
- Paper: [https://arxiv.org/abs/2601.15236](https://arxiv.org/abs/2601.15236)

Last synced: `2026-04-02 14:48:17 UTC`