初始化项目，由ModelHub XC社区提供模型

Model: Harvard-DCML/boomerang-pythia-3.8B
Source: Original Platform

README.md

@@ -0,0 +1,59 @@
+---
+license: apache-2.0
+datasets:
+- EleutherAI/the_pile_deduplicated
+---
+
+## Model Description
+
+Boomerang distillation is a phenomenon in LLMs where we can distill a teacher model into a student and reincorporate teacher layers to create intermediate-sized models with no additional training. This is the student model distilled from Pythia-6.9B from [our paper](https://arxiv.org/abs/2510.05064).
+
+## Training Procedure
+
+This model was initialized from Pythia-6.9B by copying every other layer and the last 2 layers. It was distilled on 2.1B tokens of The Pile deduplicated with cross entropy, KL, and cosine loss to match the activations of Pythia-6.9B. We used the following hyperparameters:
+
+- Learning rate: 3e-4
+- Learning rate scheduler: cosine
+- Warmup ratio: 0.01
+- Optimizer: AdamW
+- Adam betas: (0.9, 0.95)
+- Adam epsilon: 1e-8
+- Weight decay: 0.1
+- Max. gradient norm: 1.0
+- Number of training steps: 500
+- Max. sequence length: 2048
+- Effective batch size: 2048
+- Mixed precision: bf16
+- KLDiv weight: 0.1
+- Cosine distance weight per layer: 0.11
+
+## Use
+
+To interpolate between this model and Pythia-6.9B, please use the `build_intermediate_model` function from [our github repository](https://github.com/dcml-lab/boomerang-distillation):
+```python3
+import torch
+from patching.patch import build_intermediate_model
+
+intermediate_model = build_intermediate_model(
+  teacher_name_or_path = "EleutherAI/pythia-6.9b",
+  student_name_or_path = "Harvard-DCML/boomerang-pythia-3.8B",
+  num_layers_to_patch = 2,
+  patch_first_k_layers = False,
+  dtype = torch.bfloat16,
+)
+```
+Notes:
+1. Changing `num_layers_to_patch` changes the size of the intermediate model by patching different numbers of student layers.
+2. `patch_first_k_layers` should be set to False for this model for optimal interpolation performance.
+
+## Citation
+
+```
+@article{kangaslahti2025boomerang,
+  title={Boomerang Distillation Enables Zero-Shot Model Size Interpolation},
+  author={Kangaslahti, Sara and Nayak, Nihal V and Geuter, Jonathan and Fumero, Marco and Locatello, Francesco and Alvarez-Melis, David},
+  journal={arXiv preprint arXiv:2510.05064},
+  year={2025},
+  url={https://arxiv.org/abs/2510.05064}
+}
+```