初始化项目,由ModelHub XC社区提供模型
Model: navteca/multi-qa-mpnet-base-cos-v1 Source: Original Platform
This commit is contained in:
27
.gitattributes
vendored
Normal file
27
.gitattributes
vendored
Normal file
@@ -0,0 +1,27 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
82
README.md
Executable file
82
README.md
Executable file
@@ -0,0 +1,82 @@
|
||||
---
|
||||
language: en
|
||||
license: mit
|
||||
pipeline_tag: sentence-similarity
|
||||
tags:
|
||||
- feature-extraction
|
||||
- sentence-similarity
|
||||
- sentence-transformers
|
||||
---
|
||||
|
||||
# Multi QA MPNet base model for Semantic Search
|
||||
|
||||
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and was designed for **semantic search**. It has been trained on 215M (question, answer) pairs from diverse sources.
|
||||
|
||||
This model uses [`mpnet-base`](https://huggingface.co/microsoft/mpnet-base).
|
||||
|
||||
## Training Data
|
||||
We use the concatenation from multiple datasets to fine-tune this model. In total we have about 215M (question, answer) pairs. The model was trained with [MultipleNegativesRankingLoss](https://www.sbert.net/docs/package_reference/losses.html#multiplenegativesrankingloss) using Mean-pooling, cosine-similarity as similarity function, and a scale of 20.
|
||||
|
||||
|
||||
| Dataset | Number of training tuples |
|
||||
|--------------------------------------------------------|:--------------------------:|
|
||||
| [WikiAnswers](https://github.com/afader/oqa#wikianswers-corpus) Duplicate question pairs from WikiAnswers | 77,427,422 |
|
||||
| [PAQ](https://github.com/facebookresearch/PAQ) Automatically generated (Question, Paragraph) pairs for each paragraph in Wikipedia | 64,371,441 |
|
||||
| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) (Title, Body) pairs from all StackExchanges | 25,316,456 |
|
||||
| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) (Title, Answer) pairs from all StackExchanges | 21,396,559 |
|
||||
| [MS MARCO](https://microsoft.github.io/msmarco/) Triplets (query, answer, hard_negative) for 500k queries from Bing search engine | 17,579,773 |
|
||||
| [GOOAQ: Open Question Answering with Diverse Answer Types](https://github.com/allenai/gooaq) (query, answer) pairs for 3M Google queries and Google featured snippet | 3,012,496 |
|
||||
| [Amazon-QA](http://jmcauley.ucsd.edu/data/amazon/qa/) (Question, Answer) pairs from Amazon product pages | 2,448,839
|
||||
| [Yahoo Answers](https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset) (Title, Answer) pairs from Yahoo Answers | 1,198,260 |
|
||||
| [Yahoo Answers](https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset) (Question, Answer) pairs from Yahoo Answers | 681,164 |
|
||||
| [Yahoo Answers](https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset) (Title, Question) pairs from Yahoo Answers | 659,896 |
|
||||
| [SearchQA](https://huggingface.co/datasets/search_qa) (Question, Answer) pairs for 140k questions, each with Top5 Google snippets on that question | 582,261 |
|
||||
| [ELI5](https://huggingface.co/datasets/eli5) (Question, Answer) pairs from Reddit ELI5 (explainlikeimfive) | 325,475 |
|
||||
| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) Duplicate questions pairs (titles) | 304,525 |
|
||||
| [Quora Question Triplets](https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs) (Question, Duplicate_Question, Hard_Negative) triplets for Quora Questions Pairs dataset | 103,663 |
|
||||
| [Natural Questions (NQ)](https://ai.google.com/research/NaturalQuestions) (Question, Paragraph) pairs for 100k real Google queries with relevant Wikipedia paragraph | 100,231 |
|
||||
| [SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) (Question, Paragraph) pairs from SQuAD2.0 dataset | 87,599 |
|
||||
| [TriviaQA](https://huggingface.co/datasets/trivia_qa) (Question, Evidence) pairs | 73,346 |
|
||||
| **Total** | **214,988,242** |
|
||||
|
||||
## Technical Details
|
||||
In the following some technical details how this model must be used:
|
||||
|
||||
| Setting | Value |
|
||||
| --- | :---: |
|
||||
| Dimensions | 768 |
|
||||
| Produces normalized embeddings | Yes |
|
||||
| Pooling-Method | Mean pooling |
|
||||
| Suitable score functions | dot-product, cosine-similarity, or euclidean distance |
|
||||
|
||||
Note: This model produces normalized embeddings with length 1. In that case, dot-product and cosine-similarity are equivalent. dot-product is preferred as it is faster. Euclidean distance is proportional to dot-product and can also be used.
|
||||
|
||||
## Usage and Performance
|
||||
The trained model can be used like this:
|
||||
```python
|
||||
from sentence_transformers import SentenceTransformer, util
|
||||
|
||||
question = "That is a happy person"
|
||||
contexts = [
|
||||
"That is a happy dog",
|
||||
"That is a very happy person",
|
||||
"Today is a sunny day"
|
||||
]
|
||||
|
||||
# Load the model
|
||||
model = SentenceTransformer('navteca//multi-qa-mpnet-base-cos-v1')
|
||||
|
||||
# Encode question and contexts
|
||||
question_emb = model.encode(question)
|
||||
contexts_emb = model.encode(contexts)
|
||||
|
||||
# Compute dot score between question and all contexts embeddings
|
||||
result = util.dot_score(question_emb, contexts_emb)[0].cpu().tolist()
|
||||
|
||||
print(result)
|
||||
|
||||
#[
|
||||
# 0.60806852579116820,
|
||||
# 0.94949364662170410,
|
||||
# 0.29836517572402954
|
||||
#]
|
||||
23
config.json
Normal file
23
config.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"_name_or_path": "microsoft/mpnet-base",
|
||||
"architectures": [
|
||||
"MPNetForMaskedLM"
|
||||
],
|
||||
"attention_probs_dropout_prob": 0.1,
|
||||
"bos_token_id": 0,
|
||||
"eos_token_id": 2,
|
||||
"hidden_act": "gelu",
|
||||
"hidden_dropout_prob": 0.1,
|
||||
"hidden_size": 768,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 3072,
|
||||
"layer_norm_eps": 0.00001,
|
||||
"max_position_embeddings": 514,
|
||||
"model_type": "mpnet",
|
||||
"num_attention_heads": 12,
|
||||
"num_hidden_layers": 12,
|
||||
"pad_token_id": 1,
|
||||
"relative_attention_num_buckets": 32,
|
||||
"transformers_version": "4.8.2",
|
||||
"vocab_size": 30527
|
||||
}
|
||||
3
pytorch_model.bin
Normal file
3
pytorch_model.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5aba022a15fe19a16a4a78271c9289705066308dbf65765618cc7f4856bcd582
|
||||
size 438011953
|
||||
15
special_tokens_map.json
Normal file
15
special_tokens_map.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"bos_token": "<s>",
|
||||
"cls_token": "<s>",
|
||||
"eos_token": "</s>",
|
||||
"mask_token": {
|
||||
"content": "<mask>",
|
||||
"lstrip": true,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "<pad>",
|
||||
"sep_token": "</s>",
|
||||
"unk_token": "[UNK]"
|
||||
}
|
||||
16
tokenizer_config.json
Normal file
16
tokenizer_config.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"bos_token": "<s>",
|
||||
"cls_token": "<s>",
|
||||
"do_lower_case": true,
|
||||
"eos_token": "</s>",
|
||||
"mask_token": "<mask>",
|
||||
"model_max_length": 512,
|
||||
"name_or_path": "microsoft/mpnet-base",
|
||||
"pad_token": "<pad>",
|
||||
"sep_token": "</s>",
|
||||
"special_tokens_map_file": null,
|
||||
"strip_accents": null,
|
||||
"tokenize_chinese_chars": true,
|
||||
"tokenizer_class": "MPNetTokenizer",
|
||||
"unk_token": "[UNK]"
|
||||
}
|
||||
Reference in New Issue
Block a user