初始化项目，由ModelHub XC社区提供模型
Model: reazon-research/japanese-hubert-base-k2-rs35kh Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
91
README.md
Normal file
91
README.md
Normal file
@@ -0,0 +1,91 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags:
|
||||
- automatic-speech-recognition
|
||||
- hubert
|
||||
- k2ssl
|
||||
datasets:
|
||||
- reazon-research/reazonspeech
|
||||
language:
|
||||
- ja
|
||||
metrics:
|
||||
- cer
|
||||
base_model:
|
||||
- reazon-research/japanese-hubert-base-k2
|
||||
license: apache-2.0
|
||||
pipeline_tag: automatic-speech-recognition
|
||||
---
|
||||
|
||||
# `japanese-hubert-base-k2-rs35kh`
|
||||
|
||||
This model is a [Hubert Base](https://huggingface.co/reazon-research/japanese-hubert-base-k2) fine-tuned on the large-scale Japanese ASR corpus [ReazonSpeech v2.0](https://huggingface.co/datasets/reazon-research/reazonspeech) using the k2 framework.
|
||||
|
||||
## Usage
|
||||
|
||||
You can use this model through `transformers` library:
|
||||
```python
|
||||
import librosa
|
||||
import numpy as np
|
||||
from transformers import AutoProcessor, HubertForCTC
|
||||
|
||||
model = HubertForCTC.from_pretrained(
|
||||
"reazon-research/japanese-hubert-base-k2-rs35kh",
|
||||
torch_dtype=torch.bfloat16,
|
||||
attn_implementation="flash_attention_2",
|
||||
).to("cuda")
|
||||
processor = AutoProcessor.from_pretrained("reazon-research/japanese-hubert-base-k2-rs35kh")
|
||||
|
||||
audio, _ = librosa.load(audio_filepath, sr=16_000)
|
||||
audio = np.pad(audio, pad_width=int(0.5 * 16_000)) # Recommend to pad audio before inference
|
||||
input_values = processor(
|
||||
audio,
|
||||
return_tensors="pt",
|
||||
sampling_rate=16_000
|
||||
).input_values.to("cuda").to(torch.bfloat16)
|
||||
|
||||
with torch.inference_mode():
|
||||
logits = model(input_values).logits.cpu()
|
||||
predicted_ids = torch.argmax(logits, dim=-1)[0]
|
||||
transcription = processor.decode(predicted_ids, skip_special_tokens=True)
|
||||
```
|
||||
|
||||
## Test Results
|
||||
|
||||
We report the Character Error Rate (CER) of our model and the other wav2vec2 families.
|
||||
| Model | #Prameters⬇ | AVERAGE⬇ | JSUT-BASIC5000⬇ | Common Voice⬇ | TEDxJP-10K⬇ |
|
||||
| :------------------------------------------------- | :---------: | :--------: | :-------------: | :-----------: | :---------: |
|
||||
| reazon-research/japanese-wav2vec2-large-rs35kh | 319M | 16.25% | 11.00% | 18.23% | 19.53% |
|
||||
| reazon-research/japanese-wav2vec2-base-rs35kh | 96.7M | 20.40% | 13.22% | 23.76% | 24.23% |
|
||||
| reazon-research/japanese-hubert-base-k2-rs35kh | 98.4M | 11.23% | 9.94% | 11.59% | 12.18% |
|
||||
| reazon-research/japanese-hubert-base-k2-rs35kh-bpe | 98.4M | **11.07%** | **9.76%** | **11.36%** | **12.10%** |
|
||||
|
||||
We also report the CER for long-form speech.
|
||||
| Model | #Prameters⬇ | JSUT-BOOK⬇ |
|
||||
| :------------------------------------------------------ | :---------: | :--------: |
|
||||
| reazon-research/japanese-wav2vec2-large-rs35kh | 319M | 30.98% |
|
||||
| reazon-research/japanese-wav2vec2-base-rs35kh | 96.7M | 82.84% |
|
||||
| reazon-research/japanese-hubert-base-k2-rs35kh | 98.4M | **27.05%** |
|
||||
| + [Silero VAD](https://github.com/snakers4/silero-vad) | | **19.59%** |
|
||||
| reazon-research/japanese-hubert-base-k2-rs35kh-bpe | 98.4M | 84.55% |
|
||||
| + [Silero VAD](https://github.com/snakers4/silero-vad) | | **19.34%** |
|
||||
|
||||
## Citation
|
||||
```bibtex
|
||||
@misc{japanese-hubert-base-k2-rs35kh,
|
||||
title={japanese-hubert-base-k2-rs35kh},
|
||||
author={Sasaki, Yuta},
|
||||
url = {https://huggingface.co/reazon-research/japanese-hubert-base-k2-rs35kh},
|
||||
year = {2025}
|
||||
}
|
||||
|
||||
@article{yang2024k2ssl,
|
||||
title={k2SSL: A faster and better framework for self-supervised speech representation learning},
|
||||
author={Yang, Yifan and Zhuo, Jianheng and Jin, Zengrui and Ma, Ziyang and Yang, Xiaoyu and Yao, Zengwei and Guo, Liyong and Kang, Wei and Kuang, Fangjun and Lin, Long and others},
|
||||
journal={arXiv preprint arXiv:2411.17100},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
[Apache Licence 2.0](https://choosealicense.com/licenses/apache-2.0/)
|
||||
72
config.json
Normal file
72
config.json
Normal file
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"activation_dropout": 0.1,
|
||||
"apply_spec_augment": true,
|
||||
"architectures": [
|
||||
"HubertForCTC"
|
||||
],
|
||||
"attention_dropout": 0.1,
|
||||
"bos_token_id": 1,
|
||||
"classifier_proj_size": 256,
|
||||
"conv_bias": false,
|
||||
"conv_dim": [
|
||||
512,
|
||||
512,
|
||||
512,
|
||||
512,
|
||||
512,
|
||||
512,
|
||||
512
|
||||
],
|
||||
"conv_kernel": [
|
||||
10,
|
||||
3,
|
||||
3,
|
||||
3,
|
||||
3,
|
||||
2,
|
||||
2
|
||||
],
|
||||
"conv_pos_batch_norm": false,
|
||||
"conv_stride": [
|
||||
5,
|
||||
2,
|
||||
2,
|
||||
2,
|
||||
2,
|
||||
2,
|
||||
2
|
||||
],
|
||||
"ctc_loss_reduction": "sum",
|
||||
"ctc_zero_infinity": false,
|
||||
"do_stable_layer_norm": false,
|
||||
"dtype": "float32",
|
||||
"eos_token_id": 2,
|
||||
"feat_extract_activation": "gelu",
|
||||
"feat_extract_norm": "group",
|
||||
"feat_proj_dropout": 0.0,
|
||||
"feat_proj_layer_norm": true,
|
||||
"final_dropout": 0.1,
|
||||
"hidden_act": "gelu",
|
||||
"hidden_dropout": 0.1,
|
||||
"hidden_size": 768,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 3072,
|
||||
"layer_norm_eps": 1e-05,
|
||||
"layerdrop": 0.1,
|
||||
"mask_feature_length": 10,
|
||||
"mask_feature_min_masks": 0,
|
||||
"mask_feature_prob": 0.0,
|
||||
"mask_time_length": 10,
|
||||
"mask_time_min_masks": 2,
|
||||
"mask_time_prob": 0.05,
|
||||
"model_type": "hubert",
|
||||
"num_attention_heads": 12,
|
||||
"num_conv_pos_embedding_groups": 16,
|
||||
"num_conv_pos_embeddings": 128,
|
||||
"num_feat_extract_layers": 7,
|
||||
"num_hidden_layers": 12,
|
||||
"pad_token_id": 0,
|
||||
"transformers_version": "4.56.0",
|
||||
"use_weighted_layer_sum": false,
|
||||
"vocab_size": 5230
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:221e373f22a7007ca911d88774382fdf44c82eb609ba12c210086238df176a46
|
||||
size 393599760
|
||||
10
preprocessor_config.json
Normal file
10
preprocessor_config.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"do_normalize": true,
|
||||
"feature_extractor_type": "Wav2Vec2FeatureExtractor",
|
||||
"feature_size": 1,
|
||||
"padding_side": "right",
|
||||
"padding_value": 0.0,
|
||||
"processor_class": "Wav2Vec2Processor",
|
||||
"return_attention_mask": false,
|
||||
"sampling_rate": 16000
|
||||
}
|
||||
6
special_tokens_map.json
Normal file
6
special_tokens_map.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"bos_token": "<sos/eos>",
|
||||
"eos_token": "<sos/eos>",
|
||||
"pad_token": "<blk>",
|
||||
"unk_token": "<unk>"
|
||||
}
|
||||
41
tokenizer_config.json
Normal file
41
tokenizer_config.json
Normal file
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"added_tokens_decoder": {
|
||||
"0": {
|
||||
"content": "<blk>",
|
||||
"lstrip": true,
|
||||
"normalized": false,
|
||||
"rstrip": true,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"5228": {
|
||||
"content": "<unk>",
|
||||
"lstrip": true,
|
||||
"normalized": false,
|
||||
"rstrip": true,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"5229": {
|
||||
"content": "<sos/eos>",
|
||||
"lstrip": true,
|
||||
"normalized": false,
|
||||
"rstrip": true,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"bos_token": "<sos/eos>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"do_lower_case": false,
|
||||
"eos_token": "<sos/eos>",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<blk>",
|
||||
"processor_class": "Wav2Vec2Processor",
|
||||
"replace_word_delimiter_char": " ",
|
||||
"target_lang": null,
|
||||
"tokenizer_class": "Wav2Vec2CTCTokenizer",
|
||||
"unk_token": "<unk>",
|
||||
"word_delimiter_token": "|"
|
||||
}
|
||||
5232
vocab.json
Normal file
5232
vocab.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user