mnoukhov/pythia410m-sft-tldr
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mnoukhov/pythia410m-sft-tldr
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-30 14:21:21 +08:00
commit 4bb3a5553f
215 changed files with 118927 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
code/configs/newsft_pythia410m_tldr.yml Normal file
View File

@@ -0,0 +1,20 @@
task_type: tldr
model_name: EleutherAI/pythia-410m-deduped
hub_model_id: mnoukhov/pythia410m-sft-tldr
push_to_hub: True
dataset_name: vwxyzjn/summarize_from_feedback_tldr_3_filtered_oai_preprocessing_1706381144
dataset_eval_split: validation
report_to: "wandb"
learning_rate: 3e-6
lr_scheduler_type: cosine
bf16: True
gradient_accumulation_steps: 4
per_device_train_batch_size: 32
per_device_eval_batch_size: 8
num_train_epochs: 1
max_seq_length: 580
gradient_checkpointing: False
evaluation_strategy: "steps"
eval_steps: 0.2
logging_steps: 100
ddp_find_unused_parameters: False