初始化项目，由ModelHub XC社区提供模型
Model: artyomboyko/whisper-small-ru-v2 Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
116
README.md
Normal file
116
README.md
Normal file
@@ -0,0 +1,116 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: openai/whisper-small
|
||||
tags:
|
||||
- generated_from_trainer
|
||||
metrics:
|
||||
- wer
|
||||
- cer
|
||||
model-index:
|
||||
- name: whisper-small
|
||||
results:
|
||||
- task:
|
||||
name: Speech Recognition
|
||||
type: automatic-speech-recognition
|
||||
dataset:
|
||||
name: Common Voice 15
|
||||
type: artyomboyko/common_voice_15_0_RU
|
||||
args: ru
|
||||
metrics:
|
||||
- name: Test WER
|
||||
type: wer
|
||||
value: 12.675
|
||||
- name: Test CER
|
||||
type: cer
|
||||
value: 3.7305
|
||||
language:
|
||||
- ru
|
||||
datasets:
|
||||
- artyomboyko/common_voice_15_0_RU
|
||||
---
|
||||
|
||||
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
||||
should probably proofread and complete it, then remove this comment. -->
|
||||
|
||||
# Whisper-small-ru-v2
|
||||
|
||||
This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on an Russian part of the Common Voice 15 dataset.
|
||||
It achieves the following results on the evaluation set:
|
||||
- Loss: 0.1329
|
||||
- Wer: 12.6750
|
||||
- Cer: 3.7305
|
||||
- Learning Rate: 0.0000
|
||||
|
||||
## Model description
|
||||
|
||||
Same as [openai/whisper-small](https://huggingface.co/openai/whisper-small).
|
||||
|
||||
## Intended uses & limitations
|
||||
|
||||
Same as [openai/whisper-small](https://huggingface.co/openai/whisper-small)
|
||||
|
||||
## Training and evaluation data
|
||||
|
||||
Fine-tunned on an [Russian part of the Common Voice 15 dataset](https://huggingface.co/datasets/artyomboyko/common_voice_15_0_RU).
|
||||
|
||||
## Training procedure
|
||||
|
||||
According to the article ["Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers"](https://huggingface.co/blog/fine-tune-whisper)
|
||||
|
||||
### Training hyperparameters
|
||||
|
||||
The following hyperparameters were used during training:
|
||||
- learning_rate: 5e-08
|
||||
- train_batch_size: 16
|
||||
- eval_batch_size: 8
|
||||
- seed: 42
|
||||
- gradient_accumulation_steps: 2
|
||||
- total_train_batch_size: 32
|
||||
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
||||
- lr_scheduler_type: linear
|
||||
- lr_scheduler_warmup_steps: 250
|
||||
- training_steps: 15000
|
||||
- mixed_precision_training: Native AMP
|
||||
|
||||
### Training results
|
||||
|
||||
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer | Rate |
|
||||
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:------:|:------:|
|
||||
| 0.0661 | 0.09 | 500 | 0.1358 | 12.9097 | 3.8217 | 0.0000 |
|
||||
| 0.0616 | 0.17 | 1000 | 0.1357 | 12.9620 | 3.8949 | 0.0000 |
|
||||
| 0.0601 | 0.26 | 1500 | 0.1357 | 12.8795 | 3.8225 | 0.0000 |
|
||||
| 0.0666 | 0.35 | 2000 | 0.1353 | 12.9481 | 3.8871 | 0.0000 |
|
||||
| 0.0669 | 0.43 | 2500 | 0.1352 | 12.8284 | 3.8283 | 0.0000 |
|
||||
| 0.0665 | 0.52 | 3000 | 0.1351 | 12.8203 | 3.7833 | 0.0000 |
|
||||
| 0.0649 | 0.61 | 3500 | 0.1349 | 12.8098 | 3.7824 | 0.0000 |
|
||||
| 0.0607 | 0.69 | 4000 | 0.1347 | 12.8110 | 3.8105 | 0.0000 |
|
||||
| 0.0636 | 0.78 | 4500 | 0.1345 | 12.7994 | 3.7893 | 0.0000 |
|
||||
| 0.063 | 0.87 | 5000 | 0.1342 | 12.8319 | 3.8084 | 0.0000 |
|
||||
| 0.0589 | 0.95 | 5500 | 0.1341 | 12.8807 | 3.8551 | 0.0000 |
|
||||
| 0.0734 | 1.04 | 6000 | 0.1341 | 12.7691 | 3.7604 | 0.0000 |
|
||||
| 0.0577 | 1.13 | 6500 | 0.1340 | 12.7645 | 3.7602 | 0.0000 |
|
||||
| 0.052 | 1.21 | 7000 | 0.1340 | 12.7610 | 3.7655 | 0.0000 |
|
||||
| 0.0626 | 1.3 | 7500 | 0.1339 | 12.7657 | 3.7593 | 0.0000 |
|
||||
| 0.0617 | 1.39 | 8000 | 0.1338 | 12.7912 | 3.8268 | 0.0000 |
|
||||
| 0.063 | 1.47 | 8500 | 0.1337 | 12.7343 | 3.7573 | 0.0000 |
|
||||
| 0.0668 | 1.56 | 9000 | 0.1336 | 12.7308 | 3.7198 | 0.0000 |
|
||||
| 0.0634 | 1.65 | 9500 | 0.1335 | 12.7215 | 3.7400 | 0.0000 |
|
||||
| 0.0604 | 1.73 | 10000 | 0.1333 | 12.7192 | 3.7515 | 0.0000 |
|
||||
| 0.0707 | 1.82 | 10500 | 0.1333 | 12.7052 | 3.7568 | 0.0000 |
|
||||
| 0.0639 | 1.91 | 11000 | 0.1332 | 12.6983 | 3.7617 | 0.0000 |
|
||||
| 0.0617 | 1.99 | 11500 | 0.1331 | 12.6936 | 3.7402 | 0.0000 |
|
||||
| 0.0601 | 2.08 | 12000 | 0.1330 | 12.6901 | 3.7586 | 0.0000 |
|
||||
| 0.0632 | 2.17 | 12500 | 0.1330 | 12.6785 | 3.7279 | 0.0000 |
|
||||
| 0.0626 | 2.25 | 13000 | 0.1330 | 12.6808 | 3.7333 | 0.0000 |
|
||||
| 0.066 | 2.34 | 13500 | 0.1329 | 12.6704 | 3.7512 | 0.0000 |
|
||||
| 0.0674 | 2.42 | 14000 | 0.1329 | 12.6599 | 3.7384 | 0.0000 |
|
||||
| 0.0637 | 2.51 | 14500 | 0.1329 | 12.6797 | 3.7428 | 0.0000 |
|
||||
| 0.0641 | 2.6 | 15000 | 0.1329 | 12.6750 | 3.7305 | 0.0000 |
|
||||
|
||||
|
||||
### Framework versions
|
||||
|
||||
- Transformers 4.36.0.dev0
|
||||
- Pytorch 2.1.1+cu121
|
||||
- Datasets 2.15.0
|
||||
- Tokenizers 0.15.0
|
||||
108
added_tokens.json
Normal file
108
added_tokens.json
Normal file
@@ -0,0 +1,108 @@
|
||||
{
|
||||
"<|af|>": 50327,
|
||||
"<|am|>": 50334,
|
||||
"<|ar|>": 50272,
|
||||
"<|as|>": 50350,
|
||||
"<|az|>": 50304,
|
||||
"<|ba|>": 50355,
|
||||
"<|be|>": 50330,
|
||||
"<|bg|>": 50292,
|
||||
"<|bn|>": 50302,
|
||||
"<|bo|>": 50347,
|
||||
"<|br|>": 50309,
|
||||
"<|bs|>": 50315,
|
||||
"<|ca|>": 50270,
|
||||
"<|cs|>": 50283,
|
||||
"<|cy|>": 50297,
|
||||
"<|da|>": 50285,
|
||||
"<|de|>": 50261,
|
||||
"<|el|>": 50281,
|
||||
"<|en|>": 50259,
|
||||
"<|es|>": 50262,
|
||||
"<|et|>": 50307,
|
||||
"<|eu|>": 50310,
|
||||
"<|fa|>": 50300,
|
||||
"<|fi|>": 50277,
|
||||
"<|fo|>": 50338,
|
||||
"<|fr|>": 50265,
|
||||
"<|gl|>": 50319,
|
||||
"<|gu|>": 50333,
|
||||
"<|haw|>": 50352,
|
||||
"<|ha|>": 50354,
|
||||
"<|he|>": 50279,
|
||||
"<|hi|>": 50276,
|
||||
"<|hr|>": 50291,
|
||||
"<|ht|>": 50339,
|
||||
"<|hu|>": 50286,
|
||||
"<|hy|>": 50312,
|
||||
"<|id|>": 50275,
|
||||
"<|is|>": 50311,
|
||||
"<|it|>": 50274,
|
||||
"<|ja|>": 50266,
|
||||
"<|jw|>": 50356,
|
||||
"<|ka|>": 50329,
|
||||
"<|kk|>": 50316,
|
||||
"<|km|>": 50323,
|
||||
"<|kn|>": 50306,
|
||||
"<|ko|>": 50264,
|
||||
"<|la|>": 50294,
|
||||
"<|lb|>": 50345,
|
||||
"<|ln|>": 50353,
|
||||
"<|lo|>": 50336,
|
||||
"<|lt|>": 50293,
|
||||
"<|lv|>": 50301,
|
||||
"<|mg|>": 50349,
|
||||
"<|mi|>": 50295,
|
||||
"<|mk|>": 50308,
|
||||
"<|ml|>": 50296,
|
||||
"<|mn|>": 50314,
|
||||
"<|mr|>": 50320,
|
||||
"<|ms|>": 50282,
|
||||
"<|mt|>": 50343,
|
||||
"<|my|>": 50346,
|
||||
"<|ne|>": 50313,
|
||||
"<|nl|>": 50271,
|
||||
"<|nn|>": 50342,
|
||||
"<|nocaptions|>": 50362,
|
||||
"<|notimestamps|>": 50363,
|
||||
"<|no|>": 50288,
|
||||
"<|oc|>": 50328,
|
||||
"<|pa|>": 50321,
|
||||
"<|pl|>": 50269,
|
||||
"<|ps|>": 50340,
|
||||
"<|pt|>": 50267,
|
||||
"<|ro|>": 50284,
|
||||
"<|ru|>": 50263,
|
||||
"<|sa|>": 50344,
|
||||
"<|sd|>": 50332,
|
||||
"<|si|>": 50322,
|
||||
"<|sk|>": 50298,
|
||||
"<|sl|>": 50305,
|
||||
"<|sn|>": 50324,
|
||||
"<|so|>": 50326,
|
||||
"<|sq|>": 50317,
|
||||
"<|sr|>": 50303,
|
||||
"<|startoflm|>": 50360,
|
||||
"<|startofprev|>": 50361,
|
||||
"<|startoftranscript|>": 50258,
|
||||
"<|su|>": 50357,
|
||||
"<|sv|>": 50273,
|
||||
"<|sw|>": 50318,
|
||||
"<|ta|>": 50287,
|
||||
"<|te|>": 50299,
|
||||
"<|tg|>": 50331,
|
||||
"<|th|>": 50289,
|
||||
"<|tk|>": 50341,
|
||||
"<|tl|>": 50348,
|
||||
"<|transcribe|>": 50359,
|
||||
"<|translate|>": 50358,
|
||||
"<|tr|>": 50268,
|
||||
"<|tt|>": 50351,
|
||||
"<|uk|>": 50280,
|
||||
"<|ur|>": 50290,
|
||||
"<|uz|>": 50337,
|
||||
"<|vi|>": 50278,
|
||||
"<|yi|>": 50335,
|
||||
"<|yo|>": 50325,
|
||||
"<|zh|>": 50260
|
||||
}
|
||||
52
config.json
Normal file
52
config.json
Normal file
@@ -0,0 +1,52 @@
|
||||
{
|
||||
"_name_or_path": "artyomboyko/whisper-small-fine_tuned-ru-v2",
|
||||
"activation_dropout": 0.0,
|
||||
"activation_function": "gelu",
|
||||
"apply_spec_augment": false,
|
||||
"architectures": [
|
||||
"WhisperForConditionalGeneration"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"begin_suppress_tokens": [
|
||||
220,
|
||||
50257
|
||||
],
|
||||
"bos_token_id": 50257,
|
||||
"classifier_proj_size": 256,
|
||||
"d_model": 768,
|
||||
"decoder_attention_heads": 12,
|
||||
"decoder_ffn_dim": 3072,
|
||||
"decoder_layerdrop": 0.0,
|
||||
"decoder_layers": 12,
|
||||
"decoder_start_token_id": 50258,
|
||||
"dropout": 0.0,
|
||||
"encoder_attention_heads": 12,
|
||||
"encoder_ffn_dim": 3072,
|
||||
"encoder_layerdrop": 0.0,
|
||||
"encoder_layers": 12,
|
||||
"eos_token_id": 50257,
|
||||
"forced_decoder_ids": null,
|
||||
"init_std": 0.02,
|
||||
"is_encoder_decoder": true,
|
||||
"mask_feature_length": 10,
|
||||
"mask_feature_min_masks": 0,
|
||||
"mask_feature_prob": 0.0,
|
||||
"mask_time_length": 10,
|
||||
"mask_time_min_masks": 2,
|
||||
"mask_time_prob": 0.05,
|
||||
"max_length": 448,
|
||||
"max_source_positions": 1500,
|
||||
"max_target_positions": 448,
|
||||
"median_filter_width": 7,
|
||||
"model_type": "whisper",
|
||||
"num_hidden_layers": 12,
|
||||
"num_mel_bins": 80,
|
||||
"pad_token_id": 50257,
|
||||
"scale_embedding": false,
|
||||
"suppress_tokens": [],
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.36.0.dev0",
|
||||
"use_cache": true,
|
||||
"use_weighted_layer_sum": false,
|
||||
"vocab_size": 51865
|
||||
}
|
||||
263
generation_config.json
Normal file
263
generation_config.json
Normal file
@@ -0,0 +1,263 @@
|
||||
{
|
||||
"alignment_heads": [
|
||||
[
|
||||
5,
|
||||
3
|
||||
],
|
||||
[
|
||||
5,
|
||||
9
|
||||
],
|
||||
[
|
||||
8,
|
||||
0
|
||||
],
|
||||
[
|
||||
8,
|
||||
4
|
||||
],
|
||||
[
|
||||
8,
|
||||
7
|
||||
],
|
||||
[
|
||||
8,
|
||||
8
|
||||
],
|
||||
[
|
||||
9,
|
||||
0
|
||||
],
|
||||
[
|
||||
9,
|
||||
7
|
||||
],
|
||||
[
|
||||
9,
|
||||
9
|
||||
],
|
||||
[
|
||||
10,
|
||||
5
|
||||
]
|
||||
],
|
||||
"begin_suppress_tokens": [
|
||||
220,
|
||||
50257
|
||||
],
|
||||
"bos_token_id": 50257,
|
||||
"decoder_start_token_id": 50258,
|
||||
"eos_token_id": 50257,
|
||||
"forced_decoder_ids": [
|
||||
[
|
||||
1,
|
||||
null
|
||||
],
|
||||
[
|
||||
2,
|
||||
50359
|
||||
]
|
||||
],
|
||||
"is_multilingual": true,
|
||||
"lang_to_id": {
|
||||
"<|af|>": 50327,
|
||||
"<|am|>": 50334,
|
||||
"<|ar|>": 50272,
|
||||
"<|as|>": 50350,
|
||||
"<|az|>": 50304,
|
||||
"<|ba|>": 50355,
|
||||
"<|be|>": 50330,
|
||||
"<|bg|>": 50292,
|
||||
"<|bn|>": 50302,
|
||||
"<|bo|>": 50347,
|
||||
"<|br|>": 50309,
|
||||
"<|bs|>": 50315,
|
||||
"<|ca|>": 50270,
|
||||
"<|cs|>": 50283,
|
||||
"<|cy|>": 50297,
|
||||
"<|da|>": 50285,
|
||||
"<|de|>": 50261,
|
||||
"<|el|>": 50281,
|
||||
"<|en|>": 50259,
|
||||
"<|es|>": 50262,
|
||||
"<|et|>": 50307,
|
||||
"<|eu|>": 50310,
|
||||
"<|fa|>": 50300,
|
||||
"<|fi|>": 50277,
|
||||
"<|fo|>": 50338,
|
||||
"<|fr|>": 50265,
|
||||
"<|gl|>": 50319,
|
||||
"<|gu|>": 50333,
|
||||
"<|haw|>": 50352,
|
||||
"<|ha|>": 50354,
|
||||
"<|he|>": 50279,
|
||||
"<|hi|>": 50276,
|
||||
"<|hr|>": 50291,
|
||||
"<|ht|>": 50339,
|
||||
"<|hu|>": 50286,
|
||||
"<|hy|>": 50312,
|
||||
"<|id|>": 50275,
|
||||
"<|is|>": 50311,
|
||||
"<|it|>": 50274,
|
||||
"<|ja|>": 50266,
|
||||
"<|jw|>": 50356,
|
||||
"<|ka|>": 50329,
|
||||
"<|kk|>": 50316,
|
||||
"<|km|>": 50323,
|
||||
"<|kn|>": 50306,
|
||||
"<|ko|>": 50264,
|
||||
"<|la|>": 50294,
|
||||
"<|lb|>": 50345,
|
||||
"<|ln|>": 50353,
|
||||
"<|lo|>": 50336,
|
||||
"<|lt|>": 50293,
|
||||
"<|lv|>": 50301,
|
||||
"<|mg|>": 50349,
|
||||
"<|mi|>": 50295,
|
||||
"<|mk|>": 50308,
|
||||
"<|ml|>": 50296,
|
||||
"<|mn|>": 50314,
|
||||
"<|mr|>": 50320,
|
||||
"<|ms|>": 50282,
|
||||
"<|mt|>": 50343,
|
||||
"<|my|>": 50346,
|
||||
"<|ne|>": 50313,
|
||||
"<|nl|>": 50271,
|
||||
"<|nn|>": 50342,
|
||||
"<|no|>": 50288,
|
||||
"<|oc|>": 50328,
|
||||
"<|pa|>": 50321,
|
||||
"<|pl|>": 50269,
|
||||
"<|ps|>": 50340,
|
||||
"<|pt|>": 50267,
|
||||
"<|ro|>": 50284,
|
||||
"<|ru|>": 50263,
|
||||
"<|sa|>": 50344,
|
||||
"<|sd|>": 50332,
|
||||
"<|si|>": 50322,
|
||||
"<|sk|>": 50298,
|
||||
"<|sl|>": 50305,
|
||||
"<|sn|>": 50324,
|
||||
"<|so|>": 50326,
|
||||
"<|sq|>": 50317,
|
||||
"<|sr|>": 50303,
|
||||
"<|su|>": 50357,
|
||||
"<|sv|>": 50273,
|
||||
"<|sw|>": 50318,
|
||||
"<|ta|>": 50287,
|
||||
"<|te|>": 50299,
|
||||
"<|tg|>": 50331,
|
||||
"<|th|>": 50289,
|
||||
"<|tk|>": 50341,
|
||||
"<|tl|>": 50348,
|
||||
"<|tr|>": 50268,
|
||||
"<|tt|>": 50351,
|
||||
"<|uk|>": 50280,
|
||||
"<|ur|>": 50290,
|
||||
"<|uz|>": 50337,
|
||||
"<|vi|>": 50278,
|
||||
"<|yi|>": 50335,
|
||||
"<|yo|>": 50325,
|
||||
"<|zh|>": 50260
|
||||
},
|
||||
"max_initial_timestamp_index": 1,
|
||||
"max_length": 448,
|
||||
"no_timestamps_token_id": 50363,
|
||||
"pad_token_id": 50257,
|
||||
"return_timestamps": false,
|
||||
"suppress_tokens": [
|
||||
1,
|
||||
2,
|
||||
7,
|
||||
8,
|
||||
9,
|
||||
10,
|
||||
14,
|
||||
25,
|
||||
26,
|
||||
27,
|
||||
28,
|
||||
29,
|
||||
31,
|
||||
58,
|
||||
59,
|
||||
60,
|
||||
61,
|
||||
62,
|
||||
63,
|
||||
90,
|
||||
91,
|
||||
92,
|
||||
93,
|
||||
359,
|
||||
503,
|
||||
522,
|
||||
542,
|
||||
873,
|
||||
893,
|
||||
902,
|
||||
918,
|
||||
922,
|
||||
931,
|
||||
1350,
|
||||
1853,
|
||||
1982,
|
||||
2460,
|
||||
2627,
|
||||
3246,
|
||||
3253,
|
||||
3268,
|
||||
3536,
|
||||
3846,
|
||||
3961,
|
||||
4183,
|
||||
4667,
|
||||
6585,
|
||||
6647,
|
||||
7273,
|
||||
9061,
|
||||
9383,
|
||||
10428,
|
||||
10929,
|
||||
11938,
|
||||
12033,
|
||||
12331,
|
||||
12562,
|
||||
13793,
|
||||
14157,
|
||||
14635,
|
||||
15265,
|
||||
15618,
|
||||
16553,
|
||||
16604,
|
||||
18362,
|
||||
18956,
|
||||
20075,
|
||||
21675,
|
||||
22520,
|
||||
26130,
|
||||
26161,
|
||||
26435,
|
||||
28279,
|
||||
29464,
|
||||
31650,
|
||||
32302,
|
||||
32470,
|
||||
36865,
|
||||
42863,
|
||||
47425,
|
||||
49870,
|
||||
50254,
|
||||
50258,
|
||||
50358,
|
||||
50359,
|
||||
50360,
|
||||
50361,
|
||||
50362
|
||||
],
|
||||
"task_to_id": {
|
||||
"transcribe": 50359,
|
||||
"translate": 50358
|
||||
},
|
||||
"transformers_version": "4.36.0.dev0"
|
||||
}
|
||||
50000
merges.txt
Normal file
50000
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:330a26fc4b6d70eb1d1e3948e3de2c002d6d10ef2fd94bec793e5690651c9c75
|
||||
size 966995080
|
||||
1742
normalizer.json
Normal file
1742
normalizer.json
Normal file
File diff suppressed because it is too large
Load Diff
14
preprocessor_config.json
Normal file
14
preprocessor_config.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"chunk_length": 30,
|
||||
"feature_extractor_type": "WhisperFeatureExtractor",
|
||||
"feature_size": 80,
|
||||
"hop_length": 160,
|
||||
"n_fft": 400,
|
||||
"n_samples": 480000,
|
||||
"nb_max_frames": 3000,
|
||||
"padding_side": "right",
|
||||
"padding_value": 0.0,
|
||||
"processor_class": "WhisperProcessor",
|
||||
"return_attention_mask": false,
|
||||
"sampling_rate": 16000
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:02cf540038f6edcea0d16fcdb487b79fb474c77ca4ac7d5e8680e26976999cc1
|
||||
size 112472
|
||||
139
special_tokens_map.json
Normal file
139
special_tokens_map.json
Normal file
@@ -0,0 +1,139 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|endoftext|>",
|
||||
"<|startoftranscript|>",
|
||||
"<|en|>",
|
||||
"<|zh|>",
|
||||
"<|de|>",
|
||||
"<|es|>",
|
||||
"<|ru|>",
|
||||
"<|ko|>",
|
||||
"<|fr|>",
|
||||
"<|ja|>",
|
||||
"<|pt|>",
|
||||
"<|tr|>",
|
||||
"<|pl|>",
|
||||
"<|ca|>",
|
||||
"<|nl|>",
|
||||
"<|ar|>",
|
||||
"<|sv|>",
|
||||
"<|it|>",
|
||||
"<|id|>",
|
||||
"<|hi|>",
|
||||
"<|fi|>",
|
||||
"<|vi|>",
|
||||
"<|he|>",
|
||||
"<|uk|>",
|
||||
"<|el|>",
|
||||
"<|ms|>",
|
||||
"<|cs|>",
|
||||
"<|ro|>",
|
||||
"<|da|>",
|
||||
"<|hu|>",
|
||||
"<|ta|>",
|
||||
"<|no|>",
|
||||
"<|th|>",
|
||||
"<|ur|>",
|
||||
"<|hr|>",
|
||||
"<|bg|>",
|
||||
"<|lt|>",
|
||||
"<|la|>",
|
||||
"<|mi|>",
|
||||
"<|ml|>",
|
||||
"<|cy|>",
|
||||
"<|sk|>",
|
||||
"<|te|>",
|
||||
"<|fa|>",
|
||||
"<|lv|>",
|
||||
"<|bn|>",
|
||||
"<|sr|>",
|
||||
"<|az|>",
|
||||
"<|sl|>",
|
||||
"<|kn|>",
|
||||
"<|et|>",
|
||||
"<|mk|>",
|
||||
"<|br|>",
|
||||
"<|eu|>",
|
||||
"<|is|>",
|
||||
"<|hy|>",
|
||||
"<|ne|>",
|
||||
"<|mn|>",
|
||||
"<|bs|>",
|
||||
"<|kk|>",
|
||||
"<|sq|>",
|
||||
"<|sw|>",
|
||||
"<|gl|>",
|
||||
"<|mr|>",
|
||||
"<|pa|>",
|
||||
"<|si|>",
|
||||
"<|km|>",
|
||||
"<|sn|>",
|
||||
"<|yo|>",
|
||||
"<|so|>",
|
||||
"<|af|>",
|
||||
"<|oc|>",
|
||||
"<|ka|>",
|
||||
"<|be|>",
|
||||
"<|tg|>",
|
||||
"<|sd|>",
|
||||
"<|gu|>",
|
||||
"<|am|>",
|
||||
"<|yi|>",
|
||||
"<|lo|>",
|
||||
"<|uz|>",
|
||||
"<|fo|>",
|
||||
"<|ht|>",
|
||||
"<|ps|>",
|
||||
"<|tk|>",
|
||||
"<|nn|>",
|
||||
"<|mt|>",
|
||||
"<|sa|>",
|
||||
"<|lb|>",
|
||||
"<|my|>",
|
||||
"<|bo|>",
|
||||
"<|tl|>",
|
||||
"<|mg|>",
|
||||
"<|as|>",
|
||||
"<|tt|>",
|
||||
"<|haw|>",
|
||||
"<|ln|>",
|
||||
"<|ha|>",
|
||||
"<|ba|>",
|
||||
"<|jw|>",
|
||||
"<|su|>",
|
||||
"<|translate|>",
|
||||
"<|transcribe|>",
|
||||
"<|startoflm|>",
|
||||
"<|startofprev|>",
|
||||
"<|nocaptions|>",
|
||||
"<|notimestamps|>"
|
||||
],
|
||||
"bos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
981
tokenizer_config.json
Normal file
981
tokenizer_config.json
Normal file
@@ -0,0 +1,981 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"50257": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50258": {
|
||||
"content": "<|startoftranscript|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50259": {
|
||||
"content": "<|en|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50260": {
|
||||
"content": "<|zh|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50261": {
|
||||
"content": "<|de|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50262": {
|
||||
"content": "<|es|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50263": {
|
||||
"content": "<|ru|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50264": {
|
||||
"content": "<|ko|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50265": {
|
||||
"content": "<|fr|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50266": {
|
||||
"content": "<|ja|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50267": {
|
||||
"content": "<|pt|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50268": {
|
||||
"content": "<|tr|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50269": {
|
||||
"content": "<|pl|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50270": {
|
||||
"content": "<|ca|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50271": {
|
||||
"content": "<|nl|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50272": {
|
||||
"content": "<|ar|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50273": {
|
||||
"content": "<|sv|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50274": {
|
||||
"content": "<|it|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50275": {
|
||||
"content": "<|id|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50276": {
|
||||
"content": "<|hi|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50277": {
|
||||
"content": "<|fi|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50278": {
|
||||
"content": "<|vi|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50279": {
|
||||
"content": "<|he|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50280": {
|
||||
"content": "<|uk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50281": {
|
||||
"content": "<|el|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50282": {
|
||||
"content": "<|ms|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50283": {
|
||||
"content": "<|cs|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50284": {
|
||||
"content": "<|ro|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50285": {
|
||||
"content": "<|da|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50286": {
|
||||
"content": "<|hu|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50287": {
|
||||
"content": "<|ta|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50288": {
|
||||
"content": "<|no|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50289": {
|
||||
"content": "<|th|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50290": {
|
||||
"content": "<|ur|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50291": {
|
||||
"content": "<|hr|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50292": {
|
||||
"content": "<|bg|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50293": {
|
||||
"content": "<|lt|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50294": {
|
||||
"content": "<|la|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50295": {
|
||||
"content": "<|mi|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50296": {
|
||||
"content": "<|ml|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50297": {
|
||||
"content": "<|cy|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50298": {
|
||||
"content": "<|sk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50299": {
|
||||
"content": "<|te|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50300": {
|
||||
"content": "<|fa|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50301": {
|
||||
"content": "<|lv|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50302": {
|
||||
"content": "<|bn|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50303": {
|
||||
"content": "<|sr|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50304": {
|
||||
"content": "<|az|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50305": {
|
||||
"content": "<|sl|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50306": {
|
||||
"content": "<|kn|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50307": {
|
||||
"content": "<|et|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50308": {
|
||||
"content": "<|mk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50309": {
|
||||
"content": "<|br|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50310": {
|
||||
"content": "<|eu|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50311": {
|
||||
"content": "<|is|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50312": {
|
||||
"content": "<|hy|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50313": {
|
||||
"content": "<|ne|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50314": {
|
||||
"content": "<|mn|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50315": {
|
||||
"content": "<|bs|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50316": {
|
||||
"content": "<|kk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50317": {
|
||||
"content": "<|sq|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50318": {
|
||||
"content": "<|sw|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50319": {
|
||||
"content": "<|gl|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50320": {
|
||||
"content": "<|mr|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50321": {
|
||||
"content": "<|pa|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50322": {
|
||||
"content": "<|si|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50323": {
|
||||
"content": "<|km|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50324": {
|
||||
"content": "<|sn|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50325": {
|
||||
"content": "<|yo|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50326": {
|
||||
"content": "<|so|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50327": {
|
||||
"content": "<|af|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50328": {
|
||||
"content": "<|oc|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50329": {
|
||||
"content": "<|ka|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50330": {
|
||||
"content": "<|be|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50331": {
|
||||
"content": "<|tg|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50332": {
|
||||
"content": "<|sd|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50333": {
|
||||
"content": "<|gu|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50334": {
|
||||
"content": "<|am|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50335": {
|
||||
"content": "<|yi|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50336": {
|
||||
"content": "<|lo|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50337": {
|
||||
"content": "<|uz|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50338": {
|
||||
"content": "<|fo|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50339": {
|
||||
"content": "<|ht|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50340": {
|
||||
"content": "<|ps|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50341": {
|
||||
"content": "<|tk|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50342": {
|
||||
"content": "<|nn|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50343": {
|
||||
"content": "<|mt|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50344": {
|
||||
"content": "<|sa|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50345": {
|
||||
"content": "<|lb|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50346": {
|
||||
"content": "<|my|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50347": {
|
||||
"content": "<|bo|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50348": {
|
||||
"content": "<|tl|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50349": {
|
||||
"content": "<|mg|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50350": {
|
||||
"content": "<|as|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50351": {
|
||||
"content": "<|tt|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50352": {
|
||||
"content": "<|haw|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50353": {
|
||||
"content": "<|ln|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50354": {
|
||||
"content": "<|ha|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50355": {
|
||||
"content": "<|ba|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50356": {
|
||||
"content": "<|jw|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50357": {
|
||||
"content": "<|su|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50358": {
|
||||
"content": "<|translate|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50359": {
|
||||
"content": "<|transcribe|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50360": {
|
||||
"content": "<|startoflm|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50361": {
|
||||
"content": "<|startofprev|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50362": {
|
||||
"content": "<|nocaptions|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"50363": {
|
||||
"content": "<|notimestamps|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|endoftext|>",
|
||||
"<|startoftranscript|>",
|
||||
"<|en|>",
|
||||
"<|zh|>",
|
||||
"<|de|>",
|
||||
"<|es|>",
|
||||
"<|ru|>",
|
||||
"<|ko|>",
|
||||
"<|fr|>",
|
||||
"<|ja|>",
|
||||
"<|pt|>",
|
||||
"<|tr|>",
|
||||
"<|pl|>",
|
||||
"<|ca|>",
|
||||
"<|nl|>",
|
||||
"<|ar|>",
|
||||
"<|sv|>",
|
||||
"<|it|>",
|
||||
"<|id|>",
|
||||
"<|hi|>",
|
||||
"<|fi|>",
|
||||
"<|vi|>",
|
||||
"<|he|>",
|
||||
"<|uk|>",
|
||||
"<|el|>",
|
||||
"<|ms|>",
|
||||
"<|cs|>",
|
||||
"<|ro|>",
|
||||
"<|da|>",
|
||||
"<|hu|>",
|
||||
"<|ta|>",
|
||||
"<|no|>",
|
||||
"<|th|>",
|
||||
"<|ur|>",
|
||||
"<|hr|>",
|
||||
"<|bg|>",
|
||||
"<|lt|>",
|
||||
"<|la|>",
|
||||
"<|mi|>",
|
||||
"<|ml|>",
|
||||
"<|cy|>",
|
||||
"<|sk|>",
|
||||
"<|te|>",
|
||||
"<|fa|>",
|
||||
"<|lv|>",
|
||||
"<|bn|>",
|
||||
"<|sr|>",
|
||||
"<|az|>",
|
||||
"<|sl|>",
|
||||
"<|kn|>",
|
||||
"<|et|>",
|
||||
"<|mk|>",
|
||||
"<|br|>",
|
||||
"<|eu|>",
|
||||
"<|is|>",
|
||||
"<|hy|>",
|
||||
"<|ne|>",
|
||||
"<|mn|>",
|
||||
"<|bs|>",
|
||||
"<|kk|>",
|
||||
"<|sq|>",
|
||||
"<|sw|>",
|
||||
"<|gl|>",
|
||||
"<|mr|>",
|
||||
"<|pa|>",
|
||||
"<|si|>",
|
||||
"<|km|>",
|
||||
"<|sn|>",
|
||||
"<|yo|>",
|
||||
"<|so|>",
|
||||
"<|af|>",
|
||||
"<|oc|>",
|
||||
"<|ka|>",
|
||||
"<|be|>",
|
||||
"<|tg|>",
|
||||
"<|sd|>",
|
||||
"<|gu|>",
|
||||
"<|am|>",
|
||||
"<|yi|>",
|
||||
"<|lo|>",
|
||||
"<|uz|>",
|
||||
"<|fo|>",
|
||||
"<|ht|>",
|
||||
"<|ps|>",
|
||||
"<|tk|>",
|
||||
"<|nn|>",
|
||||
"<|mt|>",
|
||||
"<|sa|>",
|
||||
"<|lb|>",
|
||||
"<|my|>",
|
||||
"<|bo|>",
|
||||
"<|tl|>",
|
||||
"<|mg|>",
|
||||
"<|as|>",
|
||||
"<|tt|>",
|
||||
"<|haw|>",
|
||||
"<|ln|>",
|
||||
"<|ha|>",
|
||||
"<|ba|>",
|
||||
"<|jw|>",
|
||||
"<|su|>",
|
||||
"<|translate|>",
|
||||
"<|transcribe|>",
|
||||
"<|startoflm|>",
|
||||
"<|startofprev|>",
|
||||
"<|nocaptions|>",
|
||||
"<|notimestamps|>"
|
||||
],
|
||||
"bos_token": "<|endoftext|>",
|
||||
"clean_up_tokenization_spaces": true,
|
||||
"eos_token": "<|endoftext|>",
|
||||
"errors": "replace",
|
||||
"model_max_length": 1024,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"processor_class": "WhisperProcessor",
|
||||
"return_attention_mask": false,
|
||||
"tokenizer_class": "WhisperTokenizer",
|
||||
"unk_token": "<|endoftext|>"
|
||||
}
|
||||
3
training_args.bin
Normal file
3
training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8f678cc05c722cdd4a166528548fd4a1f013e280b8bc715ea7aa36c18d1edc89
|
||||
size 4792
|
||||
50260
vocab.json
Normal file
50260
vocab.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user