qwen-1.7b-gpt-oss-20b-pt-BR-distilled/README.md

library_name, license, pretty_name, base_model, tags

library_name	license	pretty_name	base_model	tags
transformers	apache-2.0	Qwen3-1.7B GPT-OSS-20B Distilled	Qwen/Qwen3-1.7B	qwen distillation sft text-generation transformers peft

Qwen3-1.7B-GPT-OSS-20B-Distilled

This model is a fine-tuned version of Qwen/Qwen3-1.7B, distilled from openai/gpt-oss-20b using LoRA and SFTTrainer.

Model Description

This is a student model created through knowledge distillation from a larger, more capable teacher model. The goal is to transfer the knowledge and capabilities of the openai/gpt-oss-20b (teacher) to a smaller, more efficient Qwen/Qwen3-1.7B (student) model.

Training Details

• Teacher Model: openai/gpt-oss-20b
• Dataset: Synthetic dataset of 1000 samples in Brazilian Portuguese generated by the teacher using random samples from dominguesm/alpaca-data-pt-br.
• LoRA Config: r=16, alpha=32, dropout=0.05
• Training Hyperparams: 3 epochs, learning rate=0.0002, batch size=2

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")
model = PeftModel.from_pretrained(model, "wandgibaut/qwen-1.7b-gpt-oss-20b-distilled")