SylReg-LM-7B/README.md

---
library_name: transformers
datasets:
- ryota-komatsu/SylReg
language:
- en
base_model:
- Qwen/Qwen2.5-7B
license: cc-by-nc-sa-4.0
---

# SylReg-LM 7B

<!-- Provide a quick summary of what the model is/does. -->

![](performance.png)

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Model type:** Qwen2ForCausalLM
- **Language(s) (NLP):** English
- **License:** CC BY-NC-SA 4.0
- **Finetuned from model:** [Qwen/Qwen2.5-7B](https://huggingface.co/Qwen/Qwen2.5-7B)

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** [Code](https://github.com/ryota-komatsu/speaker_disentangled_hubert)
- **Paper:** [arXiv:2607.04064](https://arxiv.org/abs/2607.04064)
- **Demo:** [Project page](https://ryota-komatsu.github.io/speaker_disentangled_hubert)

## How to Get Started with the Model

Use the code below to get started with the model.

```sh
git clone https://github.com/ryota-komatsu/speaker_disentangled_hubert.git
cd speaker_disentangled_hubert

sudo apt install git-lfs  # for UTMOS

conda create -y -n py310 -c pytorch -c nvidia -c conda-forge python=3.10 pip=24.0 setuptools=81.0.0 faiss-gpu=1.13.2
conda activate py310
pip install -r requirements/requirements.txt

sh scripts/setup.sh
```

```python
import re

import torch
import torchaudio
from transformers import AutoModelForCausalLM, AutoTokenizer

from src.flow_matching import FlowMatchingWithBigVGan
from src.s5hubert import SylRegForSyllableDiscovery

wav_path = "/path/to/wav"

# download pretrained models from hugging face hub
encoder = SylRegForSyllableDiscovery.from_pretrained("ryota-komatsu/SylReg-Distill", device_map="cuda", dtype="auto")
decoder = FlowMatchingWithBigVGan.from_pretrained("ryota-komatsu/SylReg-Decoder", device_map="cuda", dtype="auto")
speechlm = AutoModelForCausalLM.from_pretrained("ryota-komatsu/SylReg-LM-7B", device_map="cuda", dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("ryota-komatsu/SylReg-LM-7B")

# load a waveform
waveform, sr = torchaudio.load(wav_path)
waveform = torchaudio.functional.resample(waveform, sr, 16000)

# encode a waveform into syllabic units
outputs = encoder(waveform.to(encoder.device))
units = outputs[0]["units"]  # [3950, 67, ..., 503]

# speech language modeling
text = "".join(f"<{unit}>" for unit in units)
input_ids = tokenizer(text, padding=True, return_tensors="pt").input_ids.to(speechlm.device)
generated_ids = speechlm.generate(input_ids=input_ids, do_sample=True, temperature=0.8)[0]
units = tokenizer.decode(generated_ids)
units = torch.tensor([int(unit) for unit in re.findall(r"<(\d+)>", units)], device=decoder.device)

# unit-to-speech synthesis
generated_speech = decoder(units.unsqueeze(0)).waveform.cpu()
```

## Training Details

### Training Data

|  | Hours | License | Provider |
| --- | --- | --- | --- |
| [LibriSpeech](https://huggingface.co/datasets/openslr/librispeech_asr) | 960 | CC BY 4.0 | V. Panayotov *et al.* |
| [Libriheavy](https://huggingface.co/datasets/pkufool/libriheavy) | 50,978 | public domain | W. Kang *et al.* |
| [Emilia-Large](https://huggingface.co/datasets/amphion/Emilia-Dataset) | 4,447 | CC BY 4.0, CC BY-NC 4.0 | H. He *et al.* |
| [People's Speech (clean, clean_sa)](https://huggingface.co/datasets/MLCommons/peoples_speech) | 5,640 | CC-BY, CC-BY-SA | D. Galvez *et al.* |
| [VoxPopuli](https://huggingface.co/datasets/facebook/voxpopuli) | 543 | CC0-1.0 | C. Wang *et al.* |
| [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) | 27,810 | cdla-sharing-1.0 | R. Eldan *et al.* |
| [Cosmopedia-v2](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) | 38,986 | odc-by | L. B. Allal *et al.* |
| Total | 129,364 |  |  |

### Training Hyperparameters

- **Training regime:** bf16 mixed precision
- **Training steps:** 15k
- **Batch size:** 2,097,152 (=2<sup>21</sup>) tokens
- **Optimizer:** AdamW(lr=0.0003, betas=(0.9, 0.95), weight_decay=0.01)
- **Scheduler:** warmup_stable_decay(num_warmup_steps=100, num_decay_steps=5000, min_lr_ratio=0.1)

## Hardware

32 NVIDIA H100 GPUs

## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

```bibtex
@article{Komatsu_SylReg_2026,
  author    = {Komatsu, Ryota and Kawakita, Kota and Okamoto, Takuma and Shinozaki, Takahiro},
  title     = {Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization},
  year      = {2026},
  volume    = {7},
  journal   = {IEEE Open Journal of Signal Processing},
  pages     = {},
}
```