kotoba-tech/kotoba-whisper-v2.1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: kotoba-tech/kotoba-whisper-v2.1
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-15 00:23:31 +08:00
commit 3e8e1a4b75
17 changed files with 182706 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

138
README.md Normal file
View File

@@ -0,0 +1,138 @@
---
language: ja
library_name: transformers
license: apache-2.0
tags:
- audio
- automatic-speech-recognition
- hf-asr-leaderboard
widget:
- example_title: CommonVoice 8.0 (Test Split)
src: >-
https://huggingface.co/datasets/japanese-asr/ja_asr.common_voice_8_0/resolve/main/sample.flac
- example_title: JSUT Basic 5000
src: >-
https://huggingface.co/datasets/japanese-asr/ja_asr.jsut_basic5000/resolve/main/sample.flac
- example_title: ReazonSpeech (Test Split)
src: >-
https://huggingface.co/datasets/japanese-asr/ja_asr.reazonspeech_test/resolve/main/sample.flac
pipeline_tag: automatic-speech-recognition
datasets:
- japanese-asr/whisper_transcriptions.reazonspeech.all
- japanese-asr/whisper_transcriptions.reazonspeech.all.wer_10.0
- japanese-asr/whisper_transcriptions.reazonspeech.all.wer_10.0.vectorized
---
# Kotoba-Whisper-v2.1
_Kotoba-Whisper-v2.1_ is a Japanese ASR model based on [kotoba-tech/kotoba-whisper-v2.0](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.0), with
additional postprocessing stacks integrated as [`pipeline`](https://huggingface.co/docs/transformers/en/main_classes/pipelines). The new features includes
adding punctuation with [punctuators](https://github.com/1-800-BAD-CODE/punctuators/tree/main).
These libraries are merged into Kotoba-Whisper-v2.1 via pipeline and will be applied seamlessly to the predicted transcription from [kotoba-tech/kotoba-whisper-v2.0](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.0).
The pipeline has been developed through the collaboration between [Asahi Ushio](https://asahiushio.com) and [Kotoba Technologies](https://twitter.com/kotoba_tech)
Following table presents the raw CER (unlike usual CER where the punctuations are removed before computing the metrics, see the evaluation script [here](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.1/blob/main/run_short_form_eval.py))
along with the.
| model | [CommonVoice 8 (Japanese test set)](https://huggingface.co/datasets/japanese-asr/ja_asr.common_voice_8_0) | [JSUT Basic 5000](https://huggingface.co/datasets/japanese-asr/ja_asr.jsut_basic5000) | [ReazonSpeech (held out test set)](https://huggingface.co/datasets/japanese-asr/ja_asr.reazonspeech_test) |
|:--------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------:|----------------------------------------------------------------------------------------:|------------------------------------------------------------------------------------------------------------:|
| [kotoba-tech/kotoba-whisper-v2.0](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.0) | 17.6 | 15.4 | 17.4 |
| [kotoba-tech/kotoba-whisper-v2.1](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.1) | 17.7 | 15.4 | 17 | -->
| [kotoba-tech/kotoba-whisper-v1.0](https://huggingface.co/kotoba-tech/kotoba-whisper-v1.0) | 17.8 | 15.2 | 17.8 |
| [kotoba-tech/kotoba-whisper-v1.1](https://huggingface.co/kotoba-tech/kotoba-whisper-v1.1) | 17.9 | 15 | 17.8 |
| [openai/whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) | 15.3 | 13.4 | 20.5 |
| [openai/whisper-large-v2](https://huggingface.co/openai/whisper-large-v2) | 15.9 | 10.6 | 34.6 |
| [openai/whisper-large](https://huggingface.co/openai/whisper-large) | 16.6 | 11.3 | 40.7 |
| [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) | 17.9 | 13.1 | 39.3 |
| [openai/whisper-base](https://huggingface.co/openai/whisper-base) | 34.5 | 26.4 | 76 |
| [openai/whisper-small](https://huggingface.co/openai/whisper-small) | 21.5 | 18.9 | 48.1 |
| [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) | 58.8 | 38.3 | 153.3 |
Regarding to the normalized CER, since those update from v2.1 will be removed by the normalization, kotoba-tech/kotoba-whisper-v2.1 marks the same CER values as [kotoba-tech/kotoba-whisper-v2.0](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.0).
### Latency
Please refer to the section of the latency in the kotoba-whisper-v1.1 [here](https://huggingface.co/kotoba-tech/kotoba-whisper-v1.1#latency).
## Transformers Usage
Kotoba-Whisper-v2.1 is supported in the Hugging Face 🤗 Transformers library from version 4.39 onwards. To run the model, first
install the latest version of Transformers.
```bash
pip install --upgrade pip
pip install --upgrade transformers accelerate torchaudio
pip install stable-ts==2.16.0
pip install punctuators==0.0.5
```
### Transcription
The model can be used with the [`pipeline`](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.AutomaticSpeechRecognitionPipeline)
class to transcribe audio files as follows:
```python
import torch
from transformers import pipeline
from datasets import load_dataset
# config
model_id = "kotoba-tech/kotoba-whisper-v2.1"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model_kwargs = {"attn_implementation": "sdpa"} if torch.cuda.is_available() else {}
generate_kwargs = {"language": "ja", "task": "transcribe"}
# load model
pipe = pipeline(
model=model_id,
torch_dtype=torch_dtype,
device=device,
model_kwargs=model_kwargs,
batch_size=16,
trust_remote_code=True,
punctuator=True
)
# load sample audio
dataset = load_dataset("japanese-asr/ja_asr.reazonspeech_test", split="test")
sample = dataset[0]["audio"]
# run inference
result = pipe(sample, chunk_length_s=15, return_timestamps=True, generate_kwargs=generate_kwargs)
print(result)
```
- To transcribe a local audio file, simply pass the path to your audio file when you call the pipeline:
```diff
- result = pipe(sample, return_timestamps=True, generate_kwargs=generate_kwargs)
+ result = pipe("audio.mp3", return_timestamps=True, generate_kwargs=generate_kwargs)
```
- To deactivate punctuator:
```diff
- punctuator=True,
+ punctuator=False,
```
### Flash Attention 2
We recommend using [Flash-Attention 2](https://huggingface.co/docs/transformers/main/en/perf_infer_gpu_one#flashattention-2)
if your GPU allows for it. To do so, you first need to install [Flash Attention](https://github.com/Dao-AILab/flash-attention):
```
pip install flash-attn --no-build-isolation
```
Then pass `attn_implementation="flash_attention_2"` to `from_pretrained`:
```diff
- model_kwargs = {"attn_implementation": "sdpa"} if torch.cuda.is_available() else {}
+ model_kwargs = {"attn_implementation": "flash_attention_2"} if torch.cuda.is_available() else {}
```
## Acknowledgements
* [OpenAI](https://openai.com/) for the Whisper [model](https://huggingface.co/openai/whisper-large-v3).
* Hugging Face 🤗 [Transformers](https://github.com/huggingface/transformers) for the model integration.
* Hugging Face 🤗 for the [Distil-Whisper codebase](https://github.com/huggingface/distil-whisper).
* [Reazon Human Interaction Lab](https://research.reazon.jp/) for the [ReazonSpeech dataset](https://huggingface.co/datasets/reazon-research/reazonspeech).

1611
added_tokens.json Normal file
View File

File diff suppressed because it is too large Load Diff

61
config.json Normal file
View File

@@ -0,0 +1,61 @@
{
"_name_or_path": "kotoba-tech/kotoba-whisper-v2.0",
"activation_dropout": 0.0,
"activation_function": "gelu",
"apply_spec_augment": false,
"architectures": [
"WhisperForConditionalGeneration"
],
"attention_dropout": 0.0,
"begin_suppress_tokens": [
220,
50257
],
"bos_token_id": 50257,
"classifier_proj_size": 256,
"custom_pipelines": {
"kotoba-whisper": {
"impl": "kotoba_whisper.KotobaWhisperPipeline",
"pt": [
"WhisperForConditionalGeneration"
],
"tf": [
"TFWhisperForConditionalGeneration"
]
}
},
"d_model": 1280,
"decoder_attention_heads": 20,
"decoder_ffn_dim": 5120,
"decoder_layerdrop": 0.0,
"decoder_layers": 2,
"decoder_start_token_id": 50258,
"dropout": 0.0,
"encoder_attention_heads": 20,
"encoder_ffn_dim": 5120,
"encoder_layerdrop": 0.0,
"encoder_layers": 32,
"eos_token_id": 50257,
"init_std": 0.02,
"is_encoder_decoder": true,
"mask_feature_length": 10,
"mask_feature_min_masks": 0,
"mask_feature_prob": 0.0,
"mask_time_length": 10,
"mask_time_min_masks": 2,
"mask_time_prob": 0.05,
"max_length": 448,
"max_source_positions": 1500,
"max_target_positions": 448,
"median_filter_width": 7,
"model_type": "whisper",
"num_hidden_layers": 32,
"num_mel_bins": 128,
"pad_token_id": 50256,
"scale_embedding": false,
"torch_dtype": "float32",
"transformers_version": "4.41.0.dev0",
"use_cache": true,
"use_weighted_layer_sum": false,
"vocab_size": 51866
}

265
generation_config.json Normal file
View File

@@ -0,0 +1,265 @@
{
"alignment_heads": [
[
7,
0
],
[
10,
17
],
[
12,
18
],
[
13,
12
],
[
16,
1
],
[
17,
14
],
[
19,
11
],
[
21,
4
],
[
24,
1
],
[
25,
6
]
],
"begin_suppress_tokens": [
220,
50257
],
"bos_token_id": 50257,
"decoder_start_token_id": 50258,
"eos_token_id": 50257,
"forced_decoder_ids": [
[
1,
null
],
[
2,
50360
]
],
"is_multilingual": true,
"lang_to_id": {
"<|af|>": 50327,
"<|am|>": 50334,
"<|ar|>": 50272,
"<|as|>": 50350,
"<|az|>": 50304,
"<|ba|>": 50355,
"<|be|>": 50330,
"<|bg|>": 50292,
"<|bn|>": 50302,
"<|bo|>": 50347,
"<|br|>": 50309,
"<|bs|>": 50315,
"<|ca|>": 50270,
"<|cs|>": 50283,
"<|cy|>": 50297,
"<|da|>": 50285,
"<|de|>": 50261,
"<|el|>": 50281,
"<|en|>": 50259,
"<|es|>": 50262,
"<|et|>": 50307,
"<|eu|>": 50310,
"<|fa|>": 50300,
"<|fi|>": 50277,
"<|fo|>": 50338,
"<|fr|>": 50265,
"<|gl|>": 50319,
"<|gu|>": 50333,
"<|haw|>": 50352,
"<|ha|>": 50354,
"<|he|>": 50279,
"<|hi|>": 50276,
"<|hr|>": 50291,
"<|ht|>": 50339,
"<|hu|>": 50286,
"<|hy|>": 50312,
"<|id|>": 50275,
"<|is|>": 50311,
"<|it|>": 50274,
"<|ja|>": 50266,
"<|jw|>": 50356,
"<|ka|>": 50329,
"<|kk|>": 50316,
"<|km|>": 50323,
"<|kn|>": 50306,
"<|ko|>": 50264,
"<|la|>": 50294,
"<|lb|>": 50345,
"<|ln|>": 50353,
"<|lo|>": 50336,
"<|lt|>": 50293,
"<|lv|>": 50301,
"<|mg|>": 50349,
"<|mi|>": 50295,
"<|mk|>": 50308,
"<|ml|>": 50296,
"<|mn|>": 50314,
"<|mr|>": 50320,
"<|ms|>": 50282,
"<|mt|>": 50343,
"<|my|>": 50346,
"<|ne|>": 50313,
"<|nl|>": 50271,
"<|nn|>": 50342,
"<|no|>": 50288,
"<|oc|>": 50328,
"<|pa|>": 50321,
"<|pl|>": 50269,
"<|ps|>": 50340,
"<|pt|>": 50267,
"<|ro|>": 50284,
"<|ru|>": 50263,
"<|sa|>": 50344,
"<|sd|>": 50332,
"<|si|>": 50322,
"<|sk|>": 50298,
"<|sl|>": 50305,
"<|sn|>": 50324,
"<|so|>": 50326,
"<|sq|>": 50317,
"<|sr|>": 50303,
"<|su|>": 50357,
"<|sv|>": 50273,
"<|sw|>": 50318,
"<|ta|>": 50287,
"<|te|>": 50299,
"<|tg|>": 50331,
"<|th|>": 50289,
"<|tk|>": 50341,
"<|tl|>": 50348,
"<|tr|>": 50268,
"<|tt|>": 50351,
"<|uk|>": 50280,
"<|ur|>": 50290,
"<|uz|>": 50337,
"<|vi|>": 50278,
"<|yi|>": 50335,
"<|yo|>": 50325,
"<|yue|>": 50358,
"<|zh|>": 50260
},
"max_initial_timestamp_index": 50,
"max_length": 448,
"no_timestamps_token_id": 50364,
"pad_token_id": 50257,
"prev_sot_token_id": 50362,
"return_timestamps": false,
"suppress_tokens": [
1,
2,
7,
8,
9,
10,
14,
25,
26,
27,
28,
29,
31,
58,
59,
60,
61,
62,
63,
90,
91,
92,
93,
359,
503,
522,
542,
873,
893,
902,
918,
922,
931,
1350,
1853,
1982,
2460,
2627,
3246,
3253,
3268,
3536,
3846,
3961,
4183,
4667,
6585,
6647,
7273,
9061,
9383,
10428,
10929,
11938,
12033,
12331,
12562,
13793,
14157,
14635,
15265,
15618,
16553,
16604,
18362,
18956,
20075,
21675,
22520,
26130,
26161,
26435,
28279,
29464,
31650,
32302,
32470,
36865,
42863,
47425,
49870,
50254,
50258,
50359,
50360,
50361,
50362,
50363
],
"task_to_id": {
"transcribe": 50360,
"translate": 50359
},
"transformers_version": "4.41.0.dev0"
}

306
kotoba_whisper.py Normal file
View File

@@ -0,0 +1,306 @@
from typing import Union, Optional, Dict, List, Any
import requests
import torch
import numpy as np
from transformers.pipelines.audio_utils import ffmpeg_read
from transformers.pipelines.automatic_speech_recognition import AutomaticSpeechRecognitionPipeline, chunk_iter
from transformers.utils import is_torchaudio_available
from transformers.modeling_utils import PreTrainedModel
from transformers.tokenization_utils import PreTrainedTokenizer
from transformers.feature_extraction_sequence_utils import SequenceFeatureExtractor
from stable_whisper import WhisperResult
from punctuators.models import PunctCapSegModelONNX
class Punctuator:
ja_punctuations = ["!", "?", "、", "。"]
def __init__(self, model: str = "pcs_47lang"):
self.punctuation_model = PunctCapSegModelONNX.from_pretrained(model)
def punctuate(self, pipeline_chunk: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
def validate_punctuation(raw: str, punctuated: str):
if 'unk' in punctuated.lower() or any(p in raw for p in self.ja_punctuations):
return raw
if punctuated.count("。") > 1:
ind = punctuated.rfind("。")
punctuated = punctuated.replace("。", "")
punctuated = punctuated[:ind] + "。" + punctuated[ind:]
return punctuated
text_edit = self.punctuation_model.infer([c['text'] for c in pipeline_chunk])
return [
{
'timestamp': c['timestamp'],
'text': validate_punctuation(c['text'], "".join(e))
} for c, e in zip(pipeline_chunk, text_edit)
]
def _fix_timestamp(sample_rate: int, result: List[Dict[str, Any]], audio: np.ndarray) -> WhisperResult or None:
def replace_none_ts(parts):
total_dur = round(audio.shape[-1] / sample_rate, 3)
_medium_dur = _ts_nonzero_mask = None
def ts_nonzero_mask() -> np.ndarray:
nonlocal _ts_nonzero_mask
if _ts_nonzero_mask is None:
_ts_nonzero_mask = np.array([(p['end'] or p['start']) is not None for p in parts])
return _ts_nonzero_mask
def medium_dur() -> float:
nonlocal _medium_dur
if _medium_dur is None:
nonzero_dus = [p['end'] - p['start'] for p in parts if None not in (p['end'], p['start'])]
nonzero_durs = np.array(nonzero_dus)
_medium_dur = np.median(nonzero_durs) * 2 if len(nonzero_durs) else 2.0
return _medium_dur
def _curr_max_end(start: float, next_idx: float) -> float:
max_end = total_dur
if next_idx != len(parts):
mask = np.flatnonzero(ts_nonzero_mask()[next_idx:])
if len(mask):
_part = parts[mask[0]+next_idx]
max_end = _part['start'] or _part['end']
new_end = round(start + medium_dur(), 3)
if new_end > max_end:
return max_end
return new_end
for i, part in enumerate(parts, 1):
if part['start'] is None:
is_first = i == 1
if is_first:
new_start = round((part['end'] or 0) - medium_dur(), 3)
part['start'] = max(new_start, 0.0)
else:
part['start'] = parts[i - 2]['end']
if part['end'] is None:
no_next_start = i == len(parts) or parts[i]['start'] is None
part['end'] = _curr_max_end(part['start'], i) if no_next_start else parts[i]['start']
words = [dict(start=word['timestamp'][0], end=word['timestamp'][1], word=word['text']) for word in result]
replace_none_ts(words)
return WhisperResult([words], force_order=True, check_sorted=True)
def fix_timestamp(pipeline_output: List[Dict[str, Any]], audio: np.ndarray, sample_rate: int) -> List[Dict[str, Any]]:
result = _fix_timestamp(sample_rate=sample_rate, audio=audio, result=pipeline_output)
result.adjust_by_silence(
audio,
q_levels=20,
k_size=5,
sample_rate=sample_rate,
min_word_dur=None,
word_level=True,
verbose=True,
nonspeech_error=0.1,
use_word_position=True
)
if result.has_words:
result.regroup(True)
return [{"timestamp": [s.start, s.end], "text": s.text} for s in result.segments]
class KotobaWhisperPipeline(AutomaticSpeechRecognitionPipeline):
def __init__(self,
model: "PreTrainedModel",
feature_extractor: Union["SequenceFeatureExtractor", str] = None,
tokenizer: Optional[PreTrainedTokenizer] = None,
device: Union[int, "torch.device"] = None,
torch_dtype: Optional[Union[str, "torch.dtype"]] = None,
punctuator: bool = True,
stable_ts: bool = False,
**kwargs):
self.type = "seq2seq_whisper"
self.stable_ts = stable_ts
if punctuator:
self.punctuator = Punctuator()
else:
self.punctuator = None
super().__init__(
model=model,
feature_extractor=feature_extractor,
tokenizer=tokenizer,
device=device,
torch_dtype=torch_dtype,
**kwargs
)
def preprocess(self, inputs, chunk_length_s=0, stride_length_s=None):
if isinstance(inputs, str):
if inputs.startswith("http://") or inputs.startswith("https://"):
# We need to actually check for a real protocol, otherwise it's impossible to use a local file
# like http_huggingface_co.png
inputs = requests.get(inputs).content
else:
with open(inputs, "rb") as f:
inputs = f.read()
if isinstance(inputs, bytes):
inputs = ffmpeg_read(inputs, self.feature_extractor.sampling_rate)
stride = None
extra = {}
if isinstance(inputs, dict):
stride = inputs.pop("stride", None)
# Accepting `"array"` which is the key defined in `datasets` for
# better integration
if not ("sampling_rate" in inputs and ("raw" in inputs or "array" in inputs)):
raise ValueError(
"When passing a dictionary to AutomaticSpeechRecognitionPipeline, the dict needs to contain a "
'"raw" key containing the numpy array representing the audio and a "sampling_rate" key, '
"containing the sampling_rate associated with that array"
)
_inputs = inputs.pop("raw", None)
if _inputs is None:
# Remove path which will not be used from `datasets`.
inputs.pop("path", None)
_inputs = inputs.pop("array", None)
in_sampling_rate = inputs.pop("sampling_rate")
extra = inputs
inputs = _inputs
if in_sampling_rate != self.feature_extractor.sampling_rate:
if is_torchaudio_available():
from torchaudio import functional as F
else:
raise ImportError(
"torchaudio is required to resample audio samples in AutomaticSpeechRecognitionPipeline. "
"The torchaudio package can be installed through: `pip install torchaudio`."
)
inputs = F.resample(
torch.from_numpy(inputs), in_sampling_rate, self.feature_extractor.sampling_rate
).numpy()
ratio = self.feature_extractor.sampling_rate / in_sampling_rate
else:
ratio = 1
if stride is not None:
if stride[0] + stride[1] > inputs.shape[0]:
raise ValueError("Stride is too large for input")
# Stride needs to get the chunk length here, it's going to get
# swallowed by the `feature_extractor` later, and then batching
# can add extra data in the inputs, so we need to keep track
# of the original length in the stride so we can cut properly.
stride = (inputs.shape[0], int(round(stride[0] * ratio)), int(round(stride[1] * ratio)))
if not isinstance(inputs, np.ndarray):
raise ValueError(f"We expect a numpy ndarray as input, got `{type(inputs)}`")
if len(inputs.shape) != 1:
raise ValueError("We expect a single channel audio input for AutomaticSpeechRecognitionPipeline")
if chunk_length_s:
if stride_length_s is None:
stride_length_s = chunk_length_s / 6
if isinstance(stride_length_s, (int, float)):
stride_length_s = [stride_length_s, stride_length_s]
# XXX: Carefuly, this variable will not exist in `seq2seq` setting.
# Currently chunking is not possible at this level for `seq2seq` so
# it's ok.
align_to = getattr(self.model.config, "inputs_to_logits_ratio", 1)
chunk_len = int(round(chunk_length_s * self.feature_extractor.sampling_rate / align_to) * align_to)
stride_left = int(round(stride_length_s[0] * self.feature_extractor.sampling_rate / align_to) * align_to)
stride_right = int(round(stride_length_s[1] * self.feature_extractor.sampling_rate / align_to) * align_to)
if chunk_len < stride_left + stride_right:
raise ValueError("Chunk length must be superior to stride length")
for item in chunk_iter(
inputs, self.feature_extractor, chunk_len, stride_left, stride_right, self.torch_dtype
):
item["audio_array"] = inputs
yield item
else:
if inputs.shape[0] > self.feature_extractor.n_samples:
processed = self.feature_extractor(
inputs,
sampling_rate=self.feature_extractor.sampling_rate,
truncation=False,
padding="longest",
return_tensors="pt",
)
else:
processed = self.feature_extractor(
inputs, sampling_rate=self.feature_extractor.sampling_rate, return_tensors="pt"
)
if self.torch_dtype is not None:
processed = processed.to(dtype=self.torch_dtype)
if stride is not None:
processed["stride"] = stride
yield {"is_last": True, "audio_array": inputs, **processed, **extra}
def _forward(self, model_inputs, return_timestamps=False, **generate_kwargs):
attention_mask = model_inputs.pop("attention_mask", None)
stride = model_inputs.pop("stride", None)
is_last = model_inputs.pop("is_last")
audio_array = model_inputs.pop("audio_array")
encoder = self.model.get_encoder()
# Consume values so we can let extra information flow freely through
# the pipeline (important for `partial` in microphone)
if type(return_timestamps) is not bool:
raise ValueError("return_timestamps should be bool")
if "input_features" in model_inputs:
inputs = model_inputs.pop("input_features")
elif "input_values" in model_inputs:
inputs = model_inputs.pop("input_values")
else:
raise ValueError(
"Seq2Seq speech recognition model requires either a "
f"`input_features` or `input_values` key, but only has {model_inputs.keys()}"
)
# custom processing for Whisper timestamps and word-level timestamps
generate_kwargs["return_timestamps"] = True
if inputs.shape[-1] > self.feature_extractor.nb_max_frames:
generate_kwargs["input_features"] = inputs
else:
generate_kwargs["encoder_outputs"] = encoder(inputs, attention_mask=attention_mask)
tokens = self.model.generate(attention_mask=attention_mask, **generate_kwargs)
# whisper longform generation stores timestamps in "segments"
out = {"tokens": tokens}
if self.type == "seq2seq_whisper":
if stride is not None:
out["stride"] = stride
# Leftover
extra = model_inputs
return {"is_last": is_last, "audio_array": audio_array, **out, **extra}
def postprocess(self,
model_outputs,
decoder_kwargs: Optional[Dict] = None,
return_timestamps=None,
return_language=None):
assert len(model_outputs) > 0
for model_output in model_outputs:
audio_array = model_output.pop("audio_array")[0]
outputs = super().postprocess(
model_outputs=model_outputs,
decoder_kwargs=decoder_kwargs,
return_timestamps=True,
return_language=return_language
)
if self.stable_ts:
outputs["chunks"] = fix_timestamp(
pipeline_output=outputs["chunks"], audio=audio_array, sample_rate=self.feature_extractor.sampling_rate
)
if self.punctuator:
outputs["chunks"] = self.punctuator.punctuate(outputs["chunks"])
outputs["text"] = "".join([c["text"] for c in outputs["chunks"]])
if not return_timestamps:
outputs.pop("chunks")
return outputs

50001
merges.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0ef3e7b379515f0c35d0e7885638ddef0fa9f5c8e3e3f88cbc6da9b39edd1e9
size 3025686376

1742
normalizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

306
pipeline/kotoba_whisper.py Normal file
View File

@@ -0,0 +1,306 @@
from typing import Union, Optional, Dict, List, Any
import requests
import torch
import numpy as np
from transformers.pipelines.audio_utils import ffmpeg_read
from transformers.pipelines.automatic_speech_recognition import AutomaticSpeechRecognitionPipeline, chunk_iter
from transformers.utils import is_torchaudio_available
from transformers.modeling_utils import PreTrainedModel
from transformers.tokenization_utils import PreTrainedTokenizer
from transformers.feature_extraction_sequence_utils import SequenceFeatureExtractor
from stable_whisper import WhisperResult
from punctuators.models import PunctCapSegModelONNX
class Punctuator:
ja_punctuations = ["!", "?", "、", "。"]
def __init__(self, model: str = "pcs_47lang"):
self.punctuation_model = PunctCapSegModelONNX.from_pretrained(model)
def punctuate(self, pipeline_chunk: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
def validate_punctuation(raw: str, punctuated: str):
if 'unk' in punctuated.lower() or any(p in raw for p in self.ja_punctuations):
return raw
if punctuated.count("。") > 1:
ind = punctuated.rfind("。")
punctuated = punctuated.replace("。", "")
punctuated = punctuated[:ind] + "。" + punctuated[ind:]
return punctuated
text_edit = self.punctuation_model.infer([c['text'] for c in pipeline_chunk])
return [
{
'timestamp': c['timestamp'],
'text': validate_punctuation(c['text'], "".join(e))
} for c, e in zip(pipeline_chunk, text_edit)
]
def _fix_timestamp(sample_rate: int, result: List[Dict[str, Any]], audio: np.ndarray) -> WhisperResult or None:
def replace_none_ts(parts):
total_dur = round(audio.shape[-1] / sample_rate, 3)
_medium_dur = _ts_nonzero_mask = None
def ts_nonzero_mask() -> np.ndarray:
nonlocal _ts_nonzero_mask
if _ts_nonzero_mask is None:
_ts_nonzero_mask = np.array([(p['end'] or p['start']) is not None for p in parts])
return _ts_nonzero_mask
def medium_dur() -> float:
nonlocal _medium_dur
if _medium_dur is None:
nonzero_dus = [p['end'] - p['start'] for p in parts if None not in (p['end'], p['start'])]
nonzero_durs = np.array(nonzero_dus)
_medium_dur = np.median(nonzero_durs) * 2 if len(nonzero_durs) else 2.0
return _medium_dur
def _curr_max_end(start: float, next_idx: float) -> float:
max_end = total_dur
if next_idx != len(parts):
mask = np.flatnonzero(ts_nonzero_mask()[next_idx:])
if len(mask):
_part = parts[mask[0]+next_idx]
max_end = _part['start'] or _part['end']
new_end = round(start + medium_dur(), 3)
if new_end > max_end:
return max_end
return new_end
for i, part in enumerate(parts, 1):
if part['start'] is None:
is_first = i == 1
if is_first:
new_start = round((part['end'] or 0) - medium_dur(), 3)
part['start'] = max(new_start, 0.0)
else:
part['start'] = parts[i - 2]['end']
if part['end'] is None:
no_next_start = i == len(parts) or parts[i]['start'] is None
part['end'] = _curr_max_end(part['start'], i) if no_next_start else parts[i]['start']
words = [dict(start=word['timestamp'][0], end=word['timestamp'][1], word=word['text']) for word in result]
replace_none_ts(words)
return WhisperResult([words], force_order=True, check_sorted=True)
def fix_timestamp(pipeline_output: List[Dict[str, Any]], audio: np.ndarray, sample_rate: int) -> List[Dict[str, Any]]:
result = _fix_timestamp(sample_rate=sample_rate, audio=audio, result=pipeline_output)
result.adjust_by_silence(
audio,
q_levels=20,
k_size=5,
sample_rate=sample_rate,
min_word_dur=None,
word_level=True,
verbose=True,
nonspeech_error=0.1,
use_word_position=True
)
if result.has_words:
result.regroup(True)
return [{"timestamp": [s.start, s.end], "text": s.text} for s in result.segments]
class KotobaWhisperPipeline(AutomaticSpeechRecognitionPipeline):
def __init__(self,
model: "PreTrainedModel",
feature_extractor: Union["SequenceFeatureExtractor", str] = None,
tokenizer: Optional[PreTrainedTokenizer] = None,
device: Union[int, "torch.device"] = None,
torch_dtype: Optional[Union[str, "torch.dtype"]] = None,
punctuator: bool = True,
stable_ts: bool = False,
**kwargs):
self.type = "seq2seq_whisper"
self.stable_ts = stable_ts
if punctuator:
self.punctuator = Punctuator()
else:
self.punctuator = None
super().__init__(
model=model,
feature_extractor=feature_extractor,
tokenizer=tokenizer,
device=device,
torch_dtype=torch_dtype,
**kwargs
)
def preprocess(self, inputs, chunk_length_s=0, stride_length_s=None):
if isinstance(inputs, str):
if inputs.startswith("http://") or inputs.startswith("https://"):
# We need to actually check for a real protocol, otherwise it's impossible to use a local file
# like http_huggingface_co.png
inputs = requests.get(inputs).content
else:
with open(inputs, "rb") as f:
inputs = f.read()
if isinstance(inputs, bytes):
inputs = ffmpeg_read(inputs, self.feature_extractor.sampling_rate)
stride = None
extra = {}
if isinstance(inputs, dict):
stride = inputs.pop("stride", None)
# Accepting `"array"` which is the key defined in `datasets` for
# better integration
if not ("sampling_rate" in inputs and ("raw" in inputs or "array" in inputs)):
raise ValueError(
"When passing a dictionary to AutomaticSpeechRecognitionPipeline, the dict needs to contain a "
'"raw" key containing the numpy array representing the audio and a "sampling_rate" key, '
"containing the sampling_rate associated with that array"
)
_inputs = inputs.pop("raw", None)
if _inputs is None:
# Remove path which will not be used from `datasets`.
inputs.pop("path", None)
_inputs = inputs.pop("array", None)
in_sampling_rate = inputs.pop("sampling_rate")
extra = inputs
inputs = _inputs
if in_sampling_rate != self.feature_extractor.sampling_rate:
if is_torchaudio_available():
from torchaudio import functional as F
else:
raise ImportError(
"torchaudio is required to resample audio samples in AutomaticSpeechRecognitionPipeline. "
"The torchaudio package can be installed through: `pip install torchaudio`."
)
inputs = F.resample(
torch.from_numpy(inputs), in_sampling_rate, self.feature_extractor.sampling_rate
).numpy()
ratio = self.feature_extractor.sampling_rate / in_sampling_rate
else:
ratio = 1
if stride is not None:
if stride[0] + stride[1] > inputs.shape[0]:
raise ValueError("Stride is too large for input")
# Stride needs to get the chunk length here, it's going to get
# swallowed by the `feature_extractor` later, and then batching
# can add extra data in the inputs, so we need to keep track
# of the original length in the stride so we can cut properly.
stride = (inputs.shape[0], int(round(stride[0] * ratio)), int(round(stride[1] * ratio)))
if not isinstance(inputs, np.ndarray):
raise ValueError(f"We expect a numpy ndarray as input, got `{type(inputs)}`")
if len(inputs.shape) != 1:
raise ValueError("We expect a single channel audio input for AutomaticSpeechRecognitionPipeline")
if chunk_length_s:
if stride_length_s is None:
stride_length_s = chunk_length_s / 6
if isinstance(stride_length_s, (int, float)):
stride_length_s = [stride_length_s, stride_length_s]
# XXX: Carefuly, this variable will not exist in `seq2seq` setting.
# Currently chunking is not possible at this level for `seq2seq` so
# it's ok.
align_to = getattr(self.model.config, "inputs_to_logits_ratio", 1)
chunk_len = int(round(chunk_length_s * self.feature_extractor.sampling_rate / align_to) * align_to)
stride_left = int(round(stride_length_s[0] * self.feature_extractor.sampling_rate / align_to) * align_to)
stride_right = int(round(stride_length_s[1] * self.feature_extractor.sampling_rate / align_to) * align_to)
if chunk_len < stride_left + stride_right:
raise ValueError("Chunk length must be superior to stride length")
for item in chunk_iter(
inputs, self.feature_extractor, chunk_len, stride_left, stride_right, self.torch_dtype
):
item["audio_array"] = inputs
yield item
else:
if inputs.shape[0] > self.feature_extractor.n_samples:
processed = self.feature_extractor(
inputs,
sampling_rate=self.feature_extractor.sampling_rate,
truncation=False,
padding="longest",
return_tensors="pt",
)
else:
processed = self.feature_extractor(
inputs, sampling_rate=self.feature_extractor.sampling_rate, return_tensors="pt"
)
if self.torch_dtype is not None:
processed = processed.to(dtype=self.torch_dtype)
if stride is not None:
processed["stride"] = stride
yield {"is_last": True, "audio_array": inputs, **processed, **extra}
def _forward(self, model_inputs, return_timestamps=False, **generate_kwargs):
attention_mask = model_inputs.pop("attention_mask", None)
stride = model_inputs.pop("stride", None)
is_last = model_inputs.pop("is_last")
audio_array = model_inputs.pop("audio_array")
encoder = self.model.get_encoder()
# Consume values so we can let extra information flow freely through
# the pipeline (important for `partial` in microphone)
if type(return_timestamps) is not bool:
raise ValueError("return_timestamps should be bool")
if "input_features" in model_inputs:
inputs = model_inputs.pop("input_features")
elif "input_values" in model_inputs:
inputs = model_inputs.pop("input_values")
else:
raise ValueError(
"Seq2Seq speech recognition model requires either a "
f"`input_features` or `input_values` key, but only has {model_inputs.keys()}"
)
# custom processing for Whisper timestamps and word-level timestamps
generate_kwargs["return_timestamps"] = True
if inputs.shape[-1] > self.feature_extractor.nb_max_frames:
generate_kwargs["input_features"] = inputs
else:
generate_kwargs["encoder_outputs"] = encoder(inputs, attention_mask=attention_mask)
tokens = self.model.generate(attention_mask=attention_mask, **generate_kwargs)
# whisper longform generation stores timestamps in "segments"
out = {"tokens": tokens}
if self.type == "seq2seq_whisper":
if stride is not None:
out["stride"] = stride
# Leftover
extra = model_inputs
return {"is_last": is_last, "audio_array": audio_array, **out, **extra}
def postprocess(self,
model_outputs,
decoder_kwargs: Optional[Dict] = None,
return_timestamps=None,
return_language=None):
assert len(model_outputs) > 0
for model_output in model_outputs:
audio_array = model_output.pop("audio_array")[0]
outputs = super().postprocess(
model_outputs=model_outputs,
decoder_kwargs=decoder_kwargs,
return_timestamps=True,
return_language=return_language
)
if self.stable_ts:
outputs["chunks"] = fix_timestamp(
pipeline_output=outputs["chunks"], audio=audio_array, sample_rate=self.feature_extractor.sampling_rate
)
if self.punctuator:
outputs["chunks"] = self.punctuator.punctuate(outputs["chunks"])
outputs["text"] = "".join([c["text"] for c in outputs["chunks"]])
if not return_timestamps:
outputs.pop("chunks")
return outputs

29
pipeline/push_pipeline.py Normal file
View File

@@ -0,0 +1,29 @@
from kotoba_whisper import KotobaWhisperPipeline
from transformers.pipelines import PIPELINE_REGISTRY, pipeline
from transformers import WhisperForConditionalGeneration, TFWhisperForConditionalGeneration
model_alias = "kotoba-tech/kotoba-whisper-v2.1"
PIPELINE_REGISTRY.register_pipeline(
"kotoba-whisper",
pipeline_class=KotobaWhisperPipeline,
pt_model=WhisperForConditionalGeneration,
tf_model=TFWhisperForConditionalGeneration
)
pipe = pipeline(
task="kotoba-whisper",
model="kotoba-tech/kotoba-whisper-v2.0",
chunk_length_s=15,
batch_size=16,
punctuator=True,
stable_ts=True,
)
pipe.push_to_hub(model_alias)
pipe = pipeline(model=model_alias,
punctuator=True,
stable_ts=True,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)

154
pipeline/test_pipeline.py Normal file
View File

@@ -0,0 +1,154 @@
from pprint import pprint
from datasets import load_dataset
from transformers.pipelines import pipeline
model_alias = "kotoba-tech/kotoba-whisper-v1.1"
print("""### P + S ###""")
pipe = pipeline(model=model_alias,
punctuator=True,
stable_ts=True,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
return_timestamps=True,
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### P ###""")
pipe = pipeline(model=model_alias,
punctuator=True,
stable_ts=False,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
return_timestamps=True,
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### S ###""")
pipe = pipeline(model=model_alias,
punctuator=False,
stable_ts=True,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
return_timestamps=True,
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### RAW ###""")
pipe = pipeline(model=model_alias,
punctuator=False,
stable_ts=False,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
return_timestamps=True,
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### P + S ###""")
pipe = pipeline(model=model_alias,
punctuator=True,
stable_ts=True,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### P ###""")
pipe = pipeline(model=model_alias,
punctuator=True,
stable_ts=False,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### S ###""")
pipe = pipeline(model=model_alias,
punctuator=False,
stable_ts=True,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break
print("""### RAW ###""")
pipe = pipeline(model=model_alias,
punctuator=False,
stable_ts=False,
chunk_length_s=15,
batch_size=16,
trust_remote_code=True)
dataset = load_dataset("kotoba-tech/kotoba-whisper-eval", split="train")
for i in dataset:
if i["audio"]["path"] == "long_interview_1.mp3":
i["audio"]["array"] = i["audio"]["array"][:7938000]
prediction = pipe(
i["audio"],
generate_kwargs={"language": "japanese", "task": "transcribe"}
)
pprint(prediction)
break

14
preprocessor_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"chunk_length": 30,
"feature_extractor_type": "WhisperFeatureExtractor",
"feature_size": 128,
"hop_length": 160,
"n_fft": 400,
"n_samples": 480000,
"nb_max_frames": 3000,
"padding_side": "right",
"padding_value": 0.0,
"processor_class": "WhisperProcessor",
"return_attention_mask": false,
"sampling_rate": 16000
}

139
special_tokens_map.json Normal file
View File

@@ -0,0 +1,139 @@
{
"additional_special_tokens": [
"<|startoftranscript|>",
"<|en|>",
"<|zh|>",
"<|de|>",
"<|es|>",
"<|ru|>",
"<|ko|>",
"<|fr|>",
"<|ja|>",
"<|pt|>",
"<|tr|>",
"<|pl|>",
"<|ca|>",
"<|nl|>",
"<|ar|>",
"<|sv|>",
"<|it|>",
"<|id|>",
"<|hi|>",
"<|fi|>",
"<|vi|>",
"<|he|>",
"<|uk|>",
"<|el|>",
"<|ms|>",
"<|cs|>",
"<|ro|>",
"<|da|>",
"<|hu|>",
"<|ta|>",
"<|no|>",
"<|th|>",
"<|ur|>",
"<|hr|>",
"<|bg|>",
"<|lt|>",
"<|la|>",
"<|mi|>",
"<|ml|>",
"<|cy|>",
"<|sk|>",
"<|te|>",
"<|fa|>",
"<|lv|>",
"<|bn|>",
"<|sr|>",
"<|az|>",
"<|sl|>",
"<|kn|>",
"<|et|>",
"<|mk|>",
"<|br|>",
"<|eu|>",
"<|is|>",
"<|hy|>",
"<|ne|>",
"<|mn|>",
"<|bs|>",
"<|kk|>",
"<|sq|>",
"<|sw|>",
"<|gl|>",
"<|mr|>",
"<|pa|>",
"<|si|>",
"<|km|>",
"<|sn|>",
"<|yo|>",
"<|so|>",
"<|af|>",
"<|oc|>",
"<|ka|>",
"<|be|>",
"<|tg|>",
"<|sd|>",
"<|gu|>",
"<|am|>",
"<|yi|>",
"<|lo|>",
"<|uz|>",
"<|fo|>",
"<|ht|>",
"<|ps|>",
"<|tk|>",
"<|nn|>",
"<|mt|>",
"<|sa|>",
"<|lb|>",
"<|my|>",
"<|bo|>",
"<|tl|>",
"<|mg|>",
"<|as|>",
"<|tt|>",
"<|haw|>",
"<|ln|>",
"<|ha|>",
"<|ba|>",
"<|jw|>",
"<|su|>",
"<|yue|>",
"<|translate|>",
"<|transcribe|>",
"<|startoflm|>",
"<|startofprev|>",
"<|nospeech|>",
"<|notimestamps|>"
],
"bos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

114904
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

12997
tokenizer_config.json Normal file
View File

File diff suppressed because it is too large Load Diff

1
vocab.json Normal file
View File

File diff suppressed because one or more lines are too long