初始化项目，由ModelHub XC社区提供模型

Model: intfloat/simlm-msmarco-reranker
Source: Original Platform

.gitattributes

@@ -0,0 +1,31 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,76 @@
+---
+license: mit
+language:
+- en
+library_name: sentence-transformers
+pipeline_tag: text-ranking
+---
+# SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval 
+
+paper available at [https://arxiv.org/pdf/2207.02578](https://arxiv.org/pdf/2207.02578)
+
+code available at [https://github.com/microsoft/unilm/tree/master/simlm](https://github.com/microsoft/unilm/tree/master/simlm)
+
+## Paper abstract
+
+In this paper, we propose SimLM (Similarity matching with Language Model pre-training), a simple yet effective pre-training method for dense passage retrieval. 
+It employs a simple bottleneck architecture that learns to compress the passage information into a dense vector through self-supervised pre-training. 
+We use a replaced language modeling objective, which is inspired by ELECTRA, 
+to improve the sample efficiency and reduce the mismatch of the input distribution between pre-training and fine-tuning. 
+SimLM only requires access to unlabeled corpus, and is more broadly applicable when there are no labeled data or queries. 
+We conduct experiments on several large-scale passage retrieval datasets, and show substantial improvements over strong baselines under various settings. 
+Remarkably, SimLM even outperforms multi-vector approaches such as ColBERTv2 which incurs significantly more storage cost.
+
+## Results on MS-MARCO passage ranking task
+
+| Model | dev MRR@10 | dev R@50  | dev R@1k  |  TREC DL 2019 nDCG@10 | TREC DL 2020 nDCG@10  |
+|--|---|---|---|---|---|
+| **SimLM (this model)** | 43.8 |  89.2 | 98.6  | 74.6 | 72.7 |
+
+## Usage
+
+Since we use a listwise loss to train the re-ranker,
+the relevance score is not bounded to a specific numerical range.
+Higher scores mean more relevant between the given query and passage.
+
+Get relevance score from our re-ranker:
+
+```python
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer, BatchEncoding, PreTrainedTokenizerFast
+from transformers.modeling_outputs import SequenceClassifierOutput
+
+def encode(tokenizer: PreTrainedTokenizerFast,
+           query: str, passage: str, title: str = '-') -> BatchEncoding:
+    return tokenizer(query,
+                     text_pair='{}: {}'.format(title, passage),
+                     max_length=192,
+                     padding=True,
+                     truncation=True,
+                     return_tensors='pt')
+
+tokenizer = AutoTokenizer.from_pretrained('intfloat/simlm-msmarco-reranker')
+model = AutoModelForSequenceClassification.from_pretrained('intfloat/simlm-msmarco-reranker')
+model.eval()
+
+with torch.no_grad():
+    batch_dict = encode(tokenizer, 'how long is super bowl game', 'The Super Bowl is typically four hours long. The game itself takes about three and a half hours, with a 30 minute halftime show built in.')
+    outputs: SequenceClassifierOutput = model(**batch_dict, return_dict=True)
+    print(outputs.logits[0])
+
+    batch_dict = encode(tokenizer, 'how long is super bowl game', 'The cost of a Super Bowl commercial runs about $5 million for 30 seconds of airtime. But the benefits that the spot can bring to a brand can help to justify the cost.')
+    outputs: SequenceClassifierOutput = model(**batch_dict, return_dict=True)
+    print(outputs.logits[0])
+```
+
+## Citation
+
+```bibtex
+@article{Wang2022SimLMPW,
+  title={SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval},
+  author={Liang Wang and Nan Yang and Xiaolong Huang and Binxing Jiao and Linjun Yang and Daxin Jiang and Rangan Majumder and Furu Wei},
+  journal={ArXiv},
+  year={2022},
+  volume={abs/2207.02578}
+}
+```

config.json

@@ -0,0 +1,36 @@
+{
+  "_name_or_path": "google/electra-base-discriminator",
+  "architectures": [
+    "ElectraForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "embedding_size": 768,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "electra",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "summary_activation": "gelu",
+  "summary_last_dropout": 0.1,
+  "summary_type": "first",
+  "summary_use_proj": true,
+  "torch_dtype": "float16",
+  "transformers_version": "4.15.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e238673496cebca899c7ff63e6f3e9dcf856dc7abf8001e5892e0b0b03bff38
+size 437960167

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "google/electra-base-discriminator", "tokenizer_class": "ElectraTokenizer"}