初始化项目，由ModelHub XC社区提供模型

Model: lgris/wav2vec2-large-xlsr-open-brazilian-portuguese-v2
Source: Original Platform

.gitattributes

@@ -0,0 +1,18 @@
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+pytorch_model.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,168 @@
+---
+language: pt
+datasets:
+- common_voice 
+- mls
+- cetuc
+- lapsbm
+- voxforge
+metrics:
+- wer
+tags:
+- audio
+- speech
+- wav2vec2
+- pt
+- portuguese-speech-corpus
+- automatic-speech-recognition
+- speech
+- PyTorch
+- hf-asr-leaderboard
+model-index:
+- name: wav2vec2-large-xlsr-open-brazilian-portuguese-v2
+  results:
+  - task:
+      name: Automatic Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Common Voice 
+      type: common_voice
+      args: pt
+    metrics:
+    - name: Test WER
+      type: wer
+      value: 10.69
+license: apache-2.0
+---
+
+# Wav2vec 2.0 With Open Brazilian Portuguese Datasets v2
+
+This a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the following datasets:
+
+- [CETUC](http://www02.smt.ufrj.br/~igor.quintanilha/alcaim.tar.gz): contains approximately 145 hours of Brazilian Portuguese speech distributed among 50 male and 50 female speakers, each pronouncing approximately 1,000 phonetically balanced sentences selected from the [CETEN-Folha](https://www.linguateca.pt/cetenfolha/) corpus.
+- [Multilingual Librispeech (MLS)](https://arxiv.org/abs/2012.03411): a massive dataset available in many languages. The MLS is based on audiobook recordings in public domain like [LibriVox](https://librivox.org/). The dataset contains a total of 6k hours of transcribed data in many languages. The set in Portuguese [used in this work](http://www.openslr.org/94/) (mostly Brazilian variant) has approximately 284 hours of speech, obtained from 55 audiobooks read by 62 speakers.
+- [VoxForge](http://www.voxforge.org/): is a project with the goal to build open datasets for acoustic models. The corpus contains approximately 100 speakers and 4,130 utterances of Brazilian Portuguese, with sample rates varying from 16kHz to 44.1kHz.
+- [Common Voice 6.1](https://commonvoice.mozilla.org/pt): is a project proposed by Mozilla Foundation with the goal to create a wide open dataset in different languages to train ASR models. In this project, volunteers donate and validate speech using the [oficial site](https://commonvoice.mozilla.org/pt). The set in Portuguese (mostly Brazilian variant) used in this work is the 6.1 version (pt_63h_2020-12-11) that contains about 50 validated hours and 1,120 unique speakers.
+- [Lapsbm](https://github.com/falabrasil/gitlab-resources): "Falabrasil - UFPA" is a dataset used by the Fala Brasil group to benchmark ASR systems in Brazilian Portuguese. Contains 35 speakers (10 females), each one pronouncing 20 unique sentences, totalling  700 utterances in Brazilian Portuguese. The audios were recorded in 22.05 kHz without environment control.
+
+These datasets were combined to build a larger Brazilian Portuguese dataset. All data was used for training except Common Voice dev/test sets, that were used for validation/test respectively.
+
+The original model was fine-tuned using [fairseq](https://github.com/pytorch/fairseq). This notebook uses a converted version of the original one.
+
+__NOTE: The common voice test reports 10% of WER, however, this model was trained using all the validated instances of Common Voice, except the instances of the test set. This means that some speakers of the train set can be present on the test set.__
+
+## Imports and dependencies
+
+
+```python
+%%capture
+!pip install datasets
+!pip install jiwer
+!pip install torchaudio
+!pip install transformers
+!pip install soundfile
+```
+
+
+```python
+import torchaudio
+from datasets import load_dataset, load_metric
+from transformers import (
+    Wav2Vec2ForCTC,
+    Wav2Vec2Processor,
+)
+import torch
+import re
+import sys
+```
+
+## Preparation
+
+
+```python
+chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605
+wer = load_metric("wer")
+device = "cuda"
+```
+
+
+```python
+model_name = 'lgris/wav2vec2-large-xlsr-open-brazilian-portuguese-v2'
+model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
+processor = Wav2Vec2Processor.from_pretrained(model_name)
+```
+
+
+```python
+def map_to_pred(batch):
+    features = processor(batch["speech"], sampling_rate=batch["sampling_rate"][0], padding=True, return_tensors="pt")
+    input_values = features.input_values.to(device)
+    attention_mask = features.attention_mask.to(device)
+    with torch.no_grad():
+        logits = model(input_values, attention_mask=attention_mask).logits
+    pred_ids = torch.argmax(logits, dim=-1)
+    batch["predicted"] = processor.batch_decode(pred_ids)
+    batch["predicted"] = [pred.lower() for pred in batch["predicted"]]
+    batch["target"] = batch["sentence"]
+    return batch
+```
+
+## Tests
+
+### Test against Common Voice (In-domain)
+
+
+```python
+dataset = load_dataset("common_voice", "pt", split="test", data_dir="./cv-corpus-6.1-2020-12-11")
+
+resampler = torchaudio.transforms.Resample(orig_freq=48_000, new_freq=16_000)
+
+def map_to_array(batch):
+    speech, _ = torchaudio.load(batch["path"])
+    batch["speech"] = resampler.forward(speech.squeeze(0)).numpy()
+    batch["sampling_rate"] = resampler.new_freq
+    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
+    return batch
+```
+
+
+```python
+ds = dataset.map(map_to_array)
+result = ds.map(map_to_pred, batched=True, batch_size=1, remove_columns=list(ds.features.keys()))
+print(wer.compute(predictions=result["predicted"], references=result["target"]))
+for pred, target in zip(result["predicted"][:10], result["target"][:10]):
+    print(pred, "|", target) 
+```
+
+**Result**: 10.69%
+
+### Test against [TEDx](http://www.openslr.org/100/) (Out-of-domain)
+
+
+```python
+!gdown --id 1HJEnvthaGYwcV_whHEywgH2daIN4bQna
+!tar -xf tedx.tar.gz
+```
+
+
+```python
+dataset = load_dataset('csv', data_files={'test': 'test.csv'})['test']
+
+def map_to_array(batch):
+    speech, _ = torchaudio.load(batch["path"])
+    batch["speech"] = speech.squeeze(0).numpy()
+    batch["sampling_rate"] = resampler.new_freq
+    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
+    return batch
+```
+
+
+```python
+ds = dataset.map(map_to_array)
+result = ds.map(map_to_pred, batched=True, batch_size=1, remove_columns=list(ds.features.keys()))
+print(wer.compute(predictions=result["predicted"], references=result["target"]))
+for pred, target in zip(result["predicted"][:10], result["target"][:10]):
+    print(pred, "|", target) 
+```
+
+**Result**: 34.53%

config.json

@@ -0,0 +1,69 @@
+{
+  "_name_or_path": "facebook/wav2vec2-large-xlsr-53-portuguese",
+  "activation_dropout": 0.1,
+  "apply_spec_augment": true,
+  "architectures": [
+    "Wav2Vec2ForCTC"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 1,
+  "conv_bias": true,
+  "conv_dim": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "conv_kernel": [
+    10,
+    3,
+    3,
+    3,
+    3,
+    2,
+    2
+  ],
+  "conv_stride": [
+    5,
+    2,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "ctc_loss_reduction": "sum",
+  "ctc_zero_infinity": false,
+  "do_stable_layer_norm": true,
+  "eos_token_id": 2,
+  "feat_extract_activation": "gelu",
+  "feat_extract_dropout": 0.0,
+  "feat_extract_norm": "layer",
+  "feat_proj_dropout": 0.1,
+  "final_dropout": 0.1,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout": 0.1,
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "layerdrop": 0.1,
+  "mask_feature_length": 10,
+  "mask_feature_prob": 0.0,
+  "mask_time_length": 10,
+  "mask_time_prob": 0.05,
+  "model_type": "wav2vec2",
+  "num_attention_heads": 16,
+  "num_conv_pos_embedding_groups": 16,
+  "num_conv_pos_embeddings": 128,
+  "num_feat_extract_layers": 7,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "transformers_version": "4.6.1",
+  "vocab_size": 44
+}

preprocessor_config.json

@@ -0,0 +1,9 @@
+{
+  "do_normalize": true,
+  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
+  "feature_size": 1,
+  "padding_side": "right",
+  "padding_value": 0,
+  "return_attention_mask": true,
+  "sampling_rate": 16000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2e0a5a233bf51a5d7e71c61dde967d3a0dd78a4348bb99acb328a08bc1e55b6
+size 1262108311

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"unk_token": "<unk>", "bos_token": "<s>", "eos_token": "</s>", "pad_token": "<pad>", "do_lower_case": false, "word_delimiter_token": "|"}

vocab.json

@@ -0,0 +1 @@
+{"": 0, "<pad>": 1, "</s>": 2, "<unk>": 3, "|": 4, "A": 5, "E": 6, "O": 7, "S": 8, "R": 9, "I": 10, "N": 11, "D": 12, "M": 13, "T": 14, "U": 15, "C": 16, "L": 17, "P": 18, "V": 19, "G": 20, "F": 21, "H": 22, "Q": 23, "B": 24, "Ã": 25, "Ç": 26, "É": 27, "Á": 28, "Z": 29, "J": 30, "X": 31, "Í": 32, "Ó": 33, "Ê": 34, "-": 35, "Õ": 36, "À": 37, "Ú": 38, "Ô": 39, "Â": 40, "Y": 41, "K": 42, "W": 43}