e5-large-v2/README.md

tags, model-index, language, license

tags	model-index	language	license
mteb Sentence Transformers sentence-similarity sentence-transformers	name results e5-large-v2 task dataset metrics type Classification type name config split revision mteb/amazon_counterfactual MTEB AmazonCounterfactualClassification (en) en test e8379541af4e31359cca9fbcf4b00f2671dba205 type value accuracy 79.22388059701493 type value ap 43.20816505595132 type value f1 73.27811303522058 task dataset metrics type Classification type name config split revision mteb/amazon_polarity MTEB AmazonPolarityClassification default test e2d317d38cd51312af73b3d32a06d1a08b442046 type value accuracy 93.748325 type value ap 90.72534979701297 type value f1 93.73895874282185 task dataset metrics type Classification type name config split revision mteb/amazon_reviews_multi MTEB AmazonReviewsClassification (en) en test 1399c76144fd37290681b995c656ef9b2e06e26d type value accuracy 48.612 type value f1 47.61157345898393 task dataset metrics type Retrieval type name config split revision arguana MTEB ArguAna default test None type value map_at_1 23.541999999999998 type value map_at_10 38.208 type value map_at_100 39.417 type value map_at_1000 39.428999999999995 type value map_at_3 33.95 type value map_at_5 36.329 type value mrr_at_1 23.755000000000003 type value mrr_at_10 38.288 type value mrr_at_100 39.511 type value mrr_at_1000 39.523 type value mrr_at_3 34.009 type value mrr_at_5 36.434 type value ndcg_at_1 23.541999999999998 type value ndcg_at_10 46.417 type value ndcg_at_100 51.812000000000005 type value ndcg_at_1000 52.137 type value ndcg_at_3 37.528 type value ndcg_at_5 41.81 type value precision_at_1 23.541999999999998 type value precision_at_10 7.269 type value precision_at_100 0.9690000000000001 type value precision_at_1000 0.099 type value precision_at_3 15.979 type value precision_at_5 11.664 type value recall_at_1 23.541999999999998 type value recall_at_10 72.688 type value recall_at_100 96.871 type value recall_at_1000 99.431 type value recall_at_3 47.937000000000005 type value recall_at_5 58.321 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-p2p MTEB ArxivClusteringP2P default test a122ad7f3f0291bf49cc6f4d32aa80929df69d5d type value v_measure 45.546499570522094 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-s2s MTEB ArxivClusteringS2S default test f910caf1a6075f7329cdf8c1a6135696f37dbd53 type value v_measure 41.01607489943561 task dataset metrics type Reranking type name config split revision mteb/askubuntudupquestions-reranking MTEB AskUbuntuDupQuestions default test 2000358ca161889fa9c082cb41daa8dcfb161a54 type value map 59.616107510107774 type value mrr 72.75106626214661 task dataset metrics type STS type name config split revision mteb/biosses-sts MTEB BIOSSES default test d3fb88f8f02e40887cd149695127462bbcf29b4a type value cos_sim_pearson 84.33018094733868 type value cos_sim_spearman 83.60190492611737 type value euclidean_pearson 82.1492450218961 type value euclidean_spearman 82.70308926526991 type value manhattan_pearson 81.93959600076842 type value manhattan_spearman 82.73260801016369 task dataset metrics type Classification type name config split revision mteb/banking77 MTEB Banking77Classification default test 0fd18e25b25c072e09e0d92ab615fda904d66300 type value accuracy 84.54545454545455 type value f1 84.49582530928923 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-p2p MTEB BiorxivClusteringP2P default test 65b79d1d13f80053f67aca9498d9402c2d9f1f40 type value v_measure 37.362725540120096 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-s2s MTEB BiorxivClusteringS2S default test 258694dd0231531bc1fd9de6ceb52a0853c6d908 type value v_measure 34.849509608178145 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackAndroidRetrieval default test None type value map_at_1 31.502999999999997 type value map_at_10 43.323 type value map_at_100 44.708999999999996 type value map_at_1000 44.838 type value map_at_3 38.987 type value map_at_5 41.516999999999996 type value mrr_at_1 38.769999999999996 type value mrr_at_10 49.13 type value mrr_at_100 49.697 type value mrr_at_1000 49.741 type value mrr_at_3 45.804 type value mrr_at_5 47.842 type value ndcg_at_1 38.769999999999996 type value ndcg_at_10 50.266999999999996 type value ndcg_at_100 54.967 type value ndcg_at_1000 56.976000000000006 type value ndcg_at_3 43.823 type value ndcg_at_5 47.12 type value precision_at_1 38.769999999999996 type value precision_at_10 10.057 type value precision_at_100 1.554 type value precision_at_1000 0.202 type value precision_at_3 21.125 type value precision_at_5 15.851 type value recall_at_1 31.502999999999997 type value recall_at_10 63.715999999999994 type value recall_at_100 83.61800000000001 type value recall_at_1000 96.63199999999999 type value recall_at_3 45.403 type value recall_at_5 54.481 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackEnglishRetrieval default test None type value map_at_1 27.833000000000002 type value map_at_10 37.330999999999996 type value map_at_100 38.580999999999996 type value map_at_1000 38.708 type value map_at_3 34.713 type value map_at_5 36.104 type value mrr_at_1 35.223 type value mrr_at_10 43.419000000000004 type value mrr_at_100 44.198 type value mrr_at_1000 44.249 type value mrr_at_3 41.614000000000004 type value mrr_at_5 42.553000000000004 type value ndcg_at_1 35.223 type value ndcg_at_10 42.687999999999995 type value ndcg_at_100 47.447 type value ndcg_at_1000 49.701 type value ndcg_at_3 39.162 type value ndcg_at_5 40.557 type value precision_at_1 35.223 type value precision_at_10 7.962 type value precision_at_100 1.304 type value precision_at_1000 0.18 type value precision_at_3 19.023 type value precision_at_5 13.184999999999999 type value recall_at_1 27.833000000000002 type value recall_at_10 51.881 type value recall_at_100 72.04 type value recall_at_1000 86.644 type value recall_at_3 40.778 type value recall_at_5 45.176 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGamingRetrieval default test None type value map_at_1 38.175 type value map_at_10 51.174 type value map_at_100 52.26499999999999 type value map_at_1000 52.315999999999995 type value map_at_3 47.897 type value map_at_5 49.703 type value mrr_at_1 43.448 type value mrr_at_10 54.505 type value mrr_at_100 55.216 type value mrr_at_1000 55.242000000000004 type value mrr_at_3 51.98500000000001 type value mrr_at_5 53.434000000000005 type value ndcg_at_1 43.448 type value ndcg_at_10 57.282 type value ndcg_at_100 61.537 type value ndcg_at_1000 62.546 type value ndcg_at_3 51.73799999999999 type value ndcg_at_5 54.324 type value precision_at_1 43.448 type value precision_at_10 9.292 type value precision_at_100 1.233 type value precision_at_1000 0.136 type value precision_at_3 23.218 type value precision_at_5 15.887 type value recall_at_1 38.175 type value recall_at_10 72.00999999999999 type value recall_at_100 90.155 type value recall_at_1000 97.257 type value recall_at_3 57.133 type value recall_at_5 63.424 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGisRetrieval default test None type value map_at_1 22.405 type value map_at_10 30.043 type value map_at_100 31.191000000000003 type value map_at_1000 31.275 type value map_at_3 27.034000000000002 type value map_at_5 28.688000000000002 type value mrr_at_1 24.068 type value mrr_at_10 31.993 type value mrr_at_100 32.992 type value mrr_at_1000 33.050000000000004 type value mrr_at_3 28.964000000000002 type value mrr_at_5 30.653000000000002 type value ndcg_at_1 24.068 type value ndcg_at_10 35.198 type value ndcg_at_100 40.709 type value ndcg_at_1000 42.855 type value ndcg_at_3 29.139 type value ndcg_at_5 32.045 type value precision_at_1 24.068 type value precision_at_10 5.65 type value precision_at_100 0.885 type value precision_at_1000 0.11199999999999999 type value precision_at_3 12.279 type value precision_at_5 8.994 type value recall_at_1 22.405 type value recall_at_10 49.391 type value recall_at_100 74.53699999999999 type value recall_at_1000 90.605 type value recall_at_3 33.126 type value recall_at_5 40.073 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackMathematicaRetrieval default test None type value map_at_1 13.309999999999999 type value map_at_10 20.688000000000002 type value map_at_100 22.022 type value map_at_1000 22.152 type value map_at_3 17.954 type value map_at_5 19.439 type value mrr_at_1 16.294 type value mrr_at_10 24.479 type value mrr_at_100 25.515 type value mrr_at_1000 25.593 type value mrr_at_3 21.642 type value mrr_at_5 23.189999999999998 type value ndcg_at_1 16.294 type value ndcg_at_10 25.833000000000002 type value ndcg_at_100 32.074999999999996 type value ndcg_at_1000 35.083 type value ndcg_at_3 20.493 type value ndcg_at_5 22.949 type value precision_at_1 16.294 type value precision_at_10 5.112 type value precision_at_100 0.96 type value precision_at_1000 0.134 type value precision_at_3 9.908999999999999 type value precision_at_5 7.587000000000001 type value recall_at_1 13.309999999999999 type value recall_at_10 37.851 type value recall_at_100 64.835 type value recall_at_1000 86.334 type value recall_at_3 23.493 type value recall_at_5 29.528 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackPhysicsRetrieval default test None type value map_at_1 25.857999999999997 type value map_at_10 35.503 type value map_at_100 36.957 type value map_at_1000 37.065 type value map_at_3 32.275999999999996 type value map_at_5 34.119 type value mrr_at_1 31.954 type value mrr_at_10 40.851 type value mrr_at_100 41.863 type value mrr_at_1000 41.900999999999996 type value mrr_at_3 38.129999999999995 type value mrr_at_5 39.737 type value ndcg_at_1 31.954 type value ndcg_at_10 41.343999999999994 type value ndcg_at_100 47.397 type value ndcg_at_1000 49.501 type value ndcg_at_3 36.047000000000004 type value ndcg_at_5 38.639 type value precision_at_1 31.954 type value precision_at_10 7.68 type value precision_at_100 1.247 type value precision_at_1000 0.16199999999999998 type value precision_at_3 17.132 type value precision_at_5 12.589 type value recall_at_1 25.857999999999997 type value recall_at_10 53.43599999999999 type value recall_at_100 78.82400000000001 type value recall_at_1000 92.78999999999999 type value recall_at_3 38.655 type value recall_at_5 45.216 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackProgrammersRetrieval default test None type value map_at_1 24.709 type value map_at_10 34.318 type value map_at_100 35.657 type value map_at_1000 35.783 type value map_at_3 31.326999999999998 type value map_at_5 33.021 type value mrr_at_1 30.137000000000004 type value mrr_at_10 39.093 type value mrr_at_100 39.992 type value mrr_at_1000 40.056999999999995 type value mrr_at_3 36.606 type value mrr_at_5 37.861 type value ndcg_at_1 30.137000000000004 type value ndcg_at_10 39.974 type value ndcg_at_100 45.647999999999996 type value ndcg_at_1000 48.259 type value ndcg_at_3 35.028 type value ndcg_at_5 37.175999999999995 type value precision_at_1 30.137000000000004 type value precision_at_10 7.363 type value precision_at_100 1.184 type value precision_at_1000 0.161 type value precision_at_3 16.857 type value precision_at_5 11.963 type value recall_at_1 24.709 type value recall_at_10 52.087 type value recall_at_100 76.125 type value recall_at_1000 93.82300000000001 type value recall_at_3 38.149 type value recall_at_5 43.984 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackRetrieval default test None type value map_at_1 23.40791666666667 type value map_at_10 32.458083333333335 type value map_at_100 33.691916666666664 type value map_at_1000 33.81191666666666 type value map_at_3 29.51625 type value map_at_5 31.168083333333335 type value mrr_at_1 27.96591666666666 type value mrr_at_10 36.528583333333344 type value mrr_at_100 37.404 type value mrr_at_1000 37.464333333333336 type value mrr_at_3 33.92883333333333 type value mrr_at_5 35.41933333333333 type value ndcg_at_1 27.96591666666666 type value ndcg_at_10 37.89141666666666 type value ndcg_at_100 43.23066666666666 type value ndcg_at_1000 45.63258333333333 type value ndcg_at_3 32.811249999999994 type value ndcg_at_5 35.22566666666667 type value precision_at_1 27.96591666666666 type value precision_at_10 6.834083333333332 type value precision_at_100 1.12225 type value precision_at_1000 0.15241666666666667 type value precision_at_3 15.264333333333335 type value precision_at_5 11.039416666666666 type value recall_at_1 23.40791666666667 type value recall_at_10 49.927083333333336 type value recall_at_100 73.44641666666668 type value recall_at_1000 90.19950000000001 type value recall_at_3 35.88341666666667 type value recall_at_5 42.061249999999994 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackStatsRetrieval default test None type value map_at_1 19.592000000000002 type value map_at_10 26.895999999999997 type value map_at_100 27.921000000000003 type value map_at_1000 28.02 type value map_at_3 24.883 type value map_at_5 25.812 type value mrr_at_1 22.698999999999998 type value mrr_at_10 29.520999999999997 type value mrr_at_100 30.458000000000002 type value mrr_at_1000 30.526999999999997 type value mrr_at_3 27.633000000000003 type value mrr_at_5 28.483999999999998 type value ndcg_at_1 22.698999999999998 type value ndcg_at_10 31.061 type value ndcg_at_100 36.398 type value ndcg_at_1000 38.89 type value ndcg_at_3 27.149 type value ndcg_at_5 28.627000000000002 type value precision_at_1 22.698999999999998 type value precision_at_10 5.106999999999999 type value precision_at_100 0.857 type value precision_at_1000 0.11499999999999999 type value precision_at_3 11.963 type value precision_at_5 8.221 type value recall_at_1 19.592000000000002 type value recall_at_10 41.329 type value recall_at_100 66.094 type value recall_at_1000 84.511 type value recall_at_3 30.61 type value recall_at_5 34.213 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackTexRetrieval default test None type value map_at_1 14.71 type value map_at_10 20.965 type value map_at_100 21.994 type value map_at_1000 22.133 type value map_at_3 18.741 type value map_at_5 19.951 type value mrr_at_1 18.307000000000002 type value mrr_at_10 24.66 type value mrr_at_100 25.540000000000003 type value mrr_at_1000 25.629 type value mrr_at_3 22.511 type value mrr_at_5 23.72 type value ndcg_at_1 18.307000000000002 type value ndcg_at_10 25.153 type value ndcg_at_100 30.229 type value ndcg_at_1000 33.623 type value ndcg_at_3 21.203 type value ndcg_at_5 23.006999999999998 type value precision_at_1 18.307000000000002 type value precision_at_10 4.725 type value precision_at_100 0.8659999999999999 type value precision_at_1000 0.133 type value precision_at_3 10.14 type value precision_at_5 7.481 type value recall_at_1 14.71 type value recall_at_10 34.087 type value recall_at_100 57.147999999999996 type value recall_at_1000 81.777 type value recall_at_3 22.996 type value recall_at_5 27.73 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackUnixRetrieval default test None type value map_at_1 23.472 type value map_at_10 32.699 type value map_at_100 33.867000000000004 type value map_at_1000 33.967000000000006 type value map_at_3 29.718 type value map_at_5 31.345 type value mrr_at_1 28.265 type value mrr_at_10 36.945 type value mrr_at_100 37.794 type value mrr_at_1000 37.857 type value mrr_at_3 34.266000000000005 type value mrr_at_5 35.768 type value ndcg_at_1 28.265 type value ndcg_at_10 38.35 type value ndcg_at_100 43.739 type value ndcg_at_1000 46.087 type value ndcg_at_3 33.004 type value ndcg_at_5 35.411 type value precision_at_1 28.265 type value precision_at_10 6.715999999999999 type value precision_at_100 1.059 type value precision_at_1000 0.13799999999999998 type value precision_at_3 15.299 type value precision_at_5 10.951 type value recall_at_1 23.472 type value recall_at_10 51.413 type value recall_at_100 75.17 type value recall_at_1000 91.577 type value recall_at_3 36.651 type value recall_at_5 42.814 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWebmastersRetrieval default test None type value map_at_1 23.666 type value map_at_10 32.963 type value map_at_100 34.544999999999995 type value map_at_1000 34.792 type value map_at_3 29.74 type value map_at_5 31.5 type value mrr_at_1 29.051 type value mrr_at_10 38.013000000000005 type value mrr_at_100 38.997 type value mrr_at_1000 39.055 type value mrr_at_3 34.947 type value mrr_at_5 36.815 type value ndcg_at_1 29.051 type value ndcg_at_10 39.361000000000004 type value ndcg_at_100 45.186 type value ndcg_at_1000 47.867 type value ndcg_at_3 33.797 type value ndcg_at_5 36.456 type value precision_at_1 29.051 type value precision_at_10 7.668 type value precision_at_100 1.532 type value precision_at_1000 0.247 type value precision_at_3 15.876000000000001 type value precision_at_5 11.779 type value recall_at_1 23.666 type value recall_at_10 51.858000000000004 type value recall_at_100 77.805 type value recall_at_1000 94.504 type value recall_at_3 36.207 type value recall_at_5 43.094 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWordpressRetrieval default test None type value map_at_1 15.662 type value map_at_10 23.594 type value map_at_100 24.593999999999998 type value map_at_1000 24.694 type value map_at_3 20.925 type value map_at_5 22.817999999999998 type value mrr_at_1 17.375 type value mrr_at_10 25.734 type value mrr_at_100 26.586 type value mrr_at_1000 26.671 type value mrr_at_3 23.044 type value mrr_at_5 24.975 type value ndcg_at_1 17.375 type value ndcg_at_10 28.186 type value ndcg_at_100 33.436 type value ndcg_at_1000 36.203 type value ndcg_at_3 23.152 type value ndcg_at_5 26.397 type value precision_at_1 17.375 type value precision_at_10 4.677 type value precision_at_100 0.786 type value precision_at_1000 0.109 type value precision_at_3 10.351 type value precision_at_5 7.985 type value recall_at_1 15.662 type value recall_at_10 40.066 type value recall_at_100 65.006 type value recall_at_1000 85.94000000000001 type value recall_at_3 27.400000000000002 type value recall_at_5 35.002 task dataset metrics type Retrieval type name config split revision climate-fever MTEB ClimateFEVER default test None type value map_at_1 8.853 type value map_at_10 15.568000000000001 type value map_at_100 17.383000000000003 type value map_at_1000 17.584 type value map_at_3 12.561 type value map_at_5 14.056 type value mrr_at_1 18.958 type value mrr_at_10 28.288000000000004 type value mrr_at_100 29.432000000000002 type value mrr_at_1000 29.498 type value mrr_at_3 25.049 type value mrr_at_5 26.857 type value ndcg_at_1 18.958 type value ndcg_at_10 22.21 type value ndcg_at_100 29.596 type value ndcg_at_1000 33.583 type value ndcg_at_3 16.994999999999997 type value ndcg_at_5 18.95 type value precision_at_1 18.958 type value precision_at_10 7.192 type value precision_at_100 1.5 type value precision_at_1000 0.22399999999999998 type value precision_at_3 12.573 type value precision_at_5 10.202 type value recall_at_1 8.853 type value recall_at_10 28.087 type value recall_at_100 53.701 type value recall_at_1000 76.29899999999999 type value recall_at_3 15.913 type value recall_at_5 20.658 task dataset metrics type Retrieval type name config split revision dbpedia-entity MTEB DBPedia default test None type value map_at_1 9.077 type value map_at_10 20.788999999999998 type value map_at_100 30.429000000000002 type value map_at_1000 32.143 type value map_at_3 14.692 type value map_at_5 17.139 type value mrr_at_1 70.75 type value mrr_at_10 78.036 type value mrr_at_100 78.401 type value mrr_at_1000 78.404 type value mrr_at_3 76.75 type value mrr_at_5 77.47500000000001 type value ndcg_at_1 58.12500000000001 type value ndcg_at_10 44.015 type value ndcg_at_100 49.247 type value ndcg_at_1000 56.211999999999996 type value ndcg_at_3 49.151 type value ndcg_at_5 46.195 type value precision_at_1 70.75 type value precision_at_10 35.5 type value precision_at_100 11.355 type value precision_at_1000 2.1950000000000003 type value precision_at_3 53.083000000000006 type value precision_at_5 44.800000000000004 type value recall_at_1 9.077 type value recall_at_10 26.259 type value recall_at_100 56.547000000000004 type value recall_at_1000 78.551 type value recall_at_3 16.162000000000003 type value recall_at_5 19.753999999999998 task dataset metrics type Classification type name config split revision mteb/emotion MTEB EmotionClassification default test 4f58c6b202a23cf9a4da393831edf4f9183cad37 type value accuracy 49.44500000000001 type value f1 44.67067691783401 task dataset metrics type Retrieval type name config split revision fever MTEB FEVER default test None type value map_at_1 68.182 type value map_at_10 78.223 type value map_at_100 78.498 type value map_at_1000 78.512 type value map_at_3 76.71 type value map_at_5 77.725 type value mrr_at_1 73.177 type value mrr_at_10 82.513 type value mrr_at_100 82.633 type value mrr_at_1000 82.635 type value mrr_at_3 81.376 type value mrr_at_5 82.182 type value ndcg_at_1 73.177 type value ndcg_at_10 82.829 type value ndcg_at_100 83.84 type value ndcg_at_1000 84.07900000000001 type value ndcg_at_3 80.303 type value ndcg_at_5 81.846 type value precision_at_1 73.177 type value precision_at_10 10.241999999999999 type value precision_at_100 1.099 type value precision_at_1000 0.11399999999999999 type value precision_at_3 31.247999999999998 type value precision_at_5 19.697 type value recall_at_1 68.182 type value recall_at_10 92.657 type value recall_at_100 96.709 type value recall_at_1000 98.184 type value recall_at_3 85.9 type value recall_at_5 89.755 task dataset metrics type Retrieval type name config split revision fiqa MTEB FiQA2018 default test None type value map_at_1 21.108 type value map_at_10 33.342 type value map_at_100 35.281 type value map_at_1000 35.478 type value map_at_3 29.067 type value map_at_5 31.563000000000002 type value mrr_at_1 41.667 type value mrr_at_10 49.913000000000004 type value mrr_at_100 50.724000000000004 type value mrr_at_1000 50.766 type value mrr_at_3 47.504999999999995 type value mrr_at_5 49.033 type value ndcg_at_1 41.667 type value ndcg_at_10 41.144 type value ndcg_at_100 48.326 type value ndcg_at_1000 51.486 type value ndcg_at_3 37.486999999999995 type value ndcg_at_5 38.78 type value precision_at_1 41.667 type value precision_at_10 11.358 type value precision_at_100 1.873 type value precision_at_1000 0.244 type value precision_at_3 25 type value precision_at_5 18.519 type value recall_at_1 21.108 type value recall_at_10 47.249 type value recall_at_100 74.52 type value recall_at_1000 93.31 type value recall_at_3 33.271 type value recall_at_5 39.723000000000006 task dataset metrics type Retrieval type name config split revision hotpotqa MTEB HotpotQA default test None type value map_at_1 40.317 type value map_at_10 64.861 type value map_at_100 65.697 type value map_at_1000 65.755 type value map_at_3 61.258 type value map_at_5 63.590999999999994 type value mrr_at_1 80.635 type value mrr_at_10 86.528 type value mrr_at_100 86.66199999999999 type value mrr_at_1000 86.666 type value mrr_at_3 85.744 type value mrr_at_5 86.24300000000001 type value ndcg_at_1 80.635 type value ndcg_at_10 73.13199999999999 type value ndcg_at_100 75.927 type value ndcg_at_1000 76.976 type value ndcg_at_3 68.241 type value ndcg_at_5 71.071 type value precision_at_1 80.635 type value precision_at_10 15.326 type value precision_at_100 1.7500000000000002 type value precision_at_1000 0.189 type value precision_at_3 43.961 type value precision_at_5 28.599999999999998 type value recall_at_1 40.317 type value recall_at_10 76.631 type value recall_at_100 87.495 type value recall_at_1000 94.362 type value recall_at_3 65.94200000000001 type value recall_at_5 71.499 task dataset metrics type Classification type name config split revision mteb/imdb MTEB ImdbClassification default test 3d86128a09e091d6018b6d26cad27f2739fc2db7 type value accuracy 91.686 type value ap 87.5577120393173 type value f1 91.6629447355139 task dataset metrics type Retrieval type name config split revision msmarco MTEB MSMARCO default dev None type value map_at_1 23.702 type value map_at_10 36.414 type value map_at_100 37.561 type value map_at_1000 37.605 type value map_at_3 32.456 type value map_at_5 34.827000000000005 type value mrr_at_1 24.355 type value mrr_at_10 37.01 type value mrr_at_100 38.085 type value mrr_at_1000 38.123000000000005 type value mrr_at_3 33.117999999999995 type value mrr_at_5 35.452 type value ndcg_at_1 24.384 type value ndcg_at_10 43.456 type value ndcg_at_100 48.892 type value ndcg_at_1000 49.964 type value ndcg_at_3 35.475 type value ndcg_at_5 39.711 type value precision_at_1 24.384 type value precision_at_10 6.7940000000000005 type value precision_at_100 0.951 type value precision_at_1000 0.104 type value precision_at_3 15.052999999999999 type value precision_at_5 11.189 type value recall_at_1 23.702 type value recall_at_10 65.057 type value recall_at_100 90.021 type value recall_at_1000 98.142 type value recall_at_3 43.551 type value recall_at_5 53.738 task dataset metrics type Classification type name config split revision mteb/mtop_domain MTEB MTOPDomainClassification (en) en test d80d48c1eb48d3562165c59d59d0034df9fff0bf type value accuracy 94.62380300957591 type value f1 94.49871222100734 task dataset metrics type Classification type name config split revision mteb/mtop_intent MTEB MTOPIntentClassification (en) en test ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba type value accuracy 77.14090287277702 type value f1 60.32101258220515 task dataset metrics type Classification type name config split revision mteb/amazon_massive_intent MTEB MassiveIntentClassification (en) en test 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 type value accuracy 73.84330867518494 type value f1 71.92248688515255 task dataset metrics type Classification type name config split revision mteb/amazon_massive_scenario MTEB MassiveScenarioClassification (en) en test 7d571f92784cd94a019292a1f45445077d0ef634 type value accuracy 78.10692669804976 type value f1 77.9904839122866 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-p2p MTEB MedrxivClusteringP2P default test e7a26af6f3ae46b30dde8737f02c07b1505bcc73 type value v_measure 31.822988923078444 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-s2s MTEB MedrxivClusteringS2S default test 35191c8c0dca72d8ff3efcd72aa802307d469663 type value v_measure 30.38394880253403 task dataset metrics type Reranking type name config split revision mteb/mind_small MTEB MindSmallReranking default test 3bdac13927fdc888b903db93b2ffdbd90b295a69 type value map 31.82504612539082 type value mrr 32.84462298174977 task dataset metrics type Retrieval type name config split revision nfcorpus MTEB NFCorpus default test None type value map_at_1 6.029 type value map_at_10 14.088999999999999 type value map_at_100 17.601 type value map_at_1000 19.144 type value map_at_3 10.156 type value map_at_5 11.892 type value mrr_at_1 46.44 type value mrr_at_10 56.596999999999994 type value mrr_at_100 57.11000000000001 type value mrr_at_1000 57.14 type value mrr_at_3 54.334 type value mrr_at_5 55.774 type value ndcg_at_1 44.891999999999996 type value ndcg_at_10 37.134 type value ndcg_at_100 33.652 type value ndcg_at_1000 42.548 type value ndcg_at_3 41.851 type value ndcg_at_5 39.842 type value precision_at_1 46.44 type value precision_at_10 27.647 type value precision_at_100 8.309999999999999 type value precision_at_1000 2.146 type value precision_at_3 39.422000000000004 type value precision_at_5 34.675 type value recall_at_1 6.029 type value recall_at_10 18.907 type value recall_at_100 33.76 type value recall_at_1000 65.14999999999999 type value recall_at_3 11.584999999999999 type value recall_at_5 14.626 task dataset metrics type Retrieval type name config split revision nq MTEB NQ default test None type value map_at_1 39.373000000000005 type value map_at_10 55.836 type value map_at_100 56.611999999999995 type value map_at_1000 56.63 type value map_at_3 51.747 type value map_at_5 54.337999999999994 type value mrr_at_1 44.147999999999996 type value mrr_at_10 58.42699999999999 type value mrr_at_100 58.902 type value mrr_at_1000 58.914 type value mrr_at_3 55.156000000000006 type value mrr_at_5 57.291000000000004 type value ndcg_at_1 44.119 type value ndcg_at_10 63.444 type value ndcg_at_100 66.40599999999999 type value ndcg_at_1000 66.822 type value ndcg_at_3 55.962 type value ndcg_at_5 60.228 type value precision_at_1 44.119 type value precision_at_10 10.006 type value precision_at_100 1.17 type value precision_at_1000 0.121 type value precision_at_3 25.135 type value precision_at_5 17.59 type value recall_at_1 39.373000000000005 type value recall_at_10 83.78999999999999 type value recall_at_100 96.246 type value recall_at_1000 99.324 type value recall_at_3 64.71900000000001 type value recall_at_5 74.508 task dataset metrics type Retrieval type name config split revision quora MTEB QuoraRetrieval default test None type value map_at_1 69.199 type value map_at_10 82.892 type value map_at_100 83.578 type value map_at_1000 83.598 type value map_at_3 79.948 type value map_at_5 81.779 type value mrr_at_1 79.67 type value mrr_at_10 86.115 type value mrr_at_100 86.249 type value mrr_at_1000 86.251 type value mrr_at_3 85.08200000000001 type value mrr_at_5 85.783 type value ndcg_at_1 79.67 type value ndcg_at_10 86.839 type value ndcg_at_100 88.252 type value ndcg_at_1000 88.401 type value ndcg_at_3 83.86200000000001 type value ndcg_at_5 85.473 type value precision_at_1 79.67 type value precision_at_10 13.19 type value precision_at_100 1.521 type value precision_at_1000 0.157 type value precision_at_3 36.677 type value precision_at_5 24.118000000000002 type value recall_at_1 69.199 type value recall_at_10 94.321 type value recall_at_100 99.20400000000001 type value recall_at_1000 99.947 type value recall_at_3 85.787 type value recall_at_5 90.365 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering MTEB RedditClustering default test 24640382cdbf8abc73003fb0fa6d111a705499eb type value v_measure 55.82810046856353 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering-p2p MTEB RedditClusteringP2P default test 282350215ef01743dc01b456c7f5241fa8937f16 type value v_measure 63.38132611783628 task dataset metrics type Retrieval type name config split revision scidocs MTEB SCIDOCS default test None type value map_at_1 5.127000000000001 type value map_at_10 12.235 type value map_at_100 14.417 type value map_at_1000 14.75 type value map_at_3 8.906 type value map_at_5 10.591000000000001 type value mrr_at_1 25.2 type value mrr_at_10 35.879 type value mrr_at_100 36.935 type value mrr_at_1000 36.997 type value mrr_at_3 32.783 type value mrr_at_5 34.367999999999995 type value ndcg_at_1 25.2 type value ndcg_at_10 20.509 type value ndcg_at_100 28.67 type value ndcg_at_1000 34.42 type value ndcg_at_3 19.948 type value ndcg_at_5 17.166 type value precision_at_1 25.2 type value precision_at_10 10.440000000000001 type value precision_at_100 2.214 type value precision_at_1000 0.359 type value precision_at_3 18.533 type value precision_at_5 14.860000000000001 type value recall_at_1 5.127000000000001 type value recall_at_10 21.147 type value recall_at_100 44.946999999999996 type value recall_at_1000 72.89 type value recall_at_3 11.277 type value recall_at_5 15.042 task dataset metrics type STS type name config split revision mteb/sickr-sts MTEB SICK-R default test a6ea5a8cab320b040a23452cc28066d9beae2cee type value cos_sim_pearson 83.0373011786213 type value cos_sim_spearman 79.27889560856613 type value euclidean_pearson 80.31186315495655 type value euclidean_spearman 79.41630415280811 type value manhattan_pearson 80.31755140442013 type value manhattan_spearman 79.43069870027611 task dataset metrics type STS type name config split revision mteb/sts12-sts MTEB STS12 default test a0d554a64d88156834ff5ae9920b964011b16384 type value cos_sim_pearson 84.8659751342045 type value cos_sim_spearman 76.95377612997667 type value euclidean_pearson 81.24552945497848 type value euclidean_spearman 77.18236963555253 type value manhattan_pearson 81.26477607759037 type value manhattan_spearman 77.13821753062756 task dataset metrics type STS type name config split revision mteb/sts13-sts MTEB STS13 default test 7e90230a92c190f1bf69ae9002b8cea547a64cca type value cos_sim_pearson 83.34597139044875 type value cos_sim_spearman 84.124169425592 type value euclidean_pearson 83.68590721511401 type value euclidean_spearman 84.18846190846398 type value manhattan_pearson 83.57630235061498 type value manhattan_spearman 84.10244043726902 task dataset metrics type STS type name config split revision mteb/sts14-sts MTEB STS14 default test 6031580fec1f6af667f0bd2da0a551cf4f0b2375 type value cos_sim_pearson 82.67641885599572 type value cos_sim_spearman 80.46450725650428 type value euclidean_pearson 81.61645042715865 type value euclidean_spearman 80.61418394236874 type value manhattan_pearson 81.55712034928871 type value manhattan_spearman 80.57905670523951 task dataset metrics type STS type name config split revision mteb/sts15-sts MTEB STS15 default test ae752c7c21bf194d8b67fd573edf7ae58183cbe3 type value cos_sim_pearson 88.86650310886782 type value cos_sim_spearman 89.76081629222328 type value euclidean_pearson 89.1530747029954 type value euclidean_spearman 89.80990657280248 type value manhattan_pearson 89.10640563278132 type value manhattan_spearman 89.76282108434047 task dataset metrics type STS type name config split revision mteb/sts16-sts MTEB STS16 default test 4d8694f8f0e0100860b497b999b3dbed754a0513 type value cos_sim_pearson 83.93864027911118 type value cos_sim_spearman 85.47096193999023 type value euclidean_pearson 85.03141840870533 type value euclidean_spearman 85.43124029598181 type value manhattan_pearson 84.99002664393512 type value manhattan_spearman 85.39169195120834 task dataset metrics type STS type name config split revision mteb/sts17-crosslingual-sts MTEB STS17 (en-en) en-en test af5e6fb845001ecf41f4c1e033ce921939a2a68d type value cos_sim_pearson 88.7045343749832 type value cos_sim_spearman 89.03262221146677 type value euclidean_pearson 89.56078218264365 type value euclidean_spearman 89.17827006466868 type value manhattan_pearson 89.52717595468582 type value manhattan_spearman 89.15878115952923 task dataset metrics type STS type name config split revision mteb/sts22-crosslingual-sts MTEB STS22 (en) en test 6d1ba47164174a496b7fa5d3569dae26a6813b80 type value cos_sim_pearson 64.20191302875551 type value cos_sim_spearman 64.11446552557646 type value euclidean_pearson 64.6918197393619 type value euclidean_spearman 63.440182631197764 type value manhattan_pearson 64.55692904121835 type value manhattan_spearman 63.424877742756266 task dataset metrics type STS type name config split revision mteb/stsbenchmark-sts MTEB STSBenchmark default test b0fddb56ed78048fa8b90373c8a3cfc37b684831 type value cos_sim_pearson 86.37793104662344 type value cos_sim_spearman 87.7357802629067 type value euclidean_pearson 87.4286301545109 type value euclidean_spearman 87.78452920777421 type value manhattan_pearson 87.42445169331255 type value manhattan_spearman 87.78537677249598 task dataset metrics type Reranking type name config split revision mteb/scidocs-reranking MTEB SciDocsRR default test d3c5e1fc0b855ab6097bf1cda04dd73947d7caab type value map 84.31465405081792 type value mrr 95.7173781193389 task dataset metrics type Retrieval type name config split revision scifact MTEB SciFact default test None type value map_at_1 57.760999999999996 type value map_at_10 67.904 type value map_at_100 68.539 type value map_at_1000 68.562 type value map_at_3 65.415 type value map_at_5 66.788 type value mrr_at_1 60.333000000000006 type value mrr_at_10 68.797 type value mrr_at_100 69.236 type value mrr_at_1000 69.257 type value mrr_at_3 66.667 type value mrr_at_5 67.967 type value ndcg_at_1 60.333000000000006 type value ndcg_at_10 72.24199999999999 type value ndcg_at_100 74.86 type value ndcg_at_1000 75.354 type value ndcg_at_3 67.93400000000001 type value ndcg_at_5 70.02199999999999 type value precision_at_1 60.333000000000006 type value precision_at_10 9.533 type value precision_at_100 1.09 type value precision_at_1000 0.11299999999999999 type value precision_at_3 26.778000000000002 type value precision_at_5 17.467 type value recall_at_1 57.760999999999996 type value recall_at_10 84.383 type value recall_at_100 96.267 type value recall_at_1000 100 type value recall_at_3 72.628 type value recall_at_5 78.094 task dataset metrics type PairClassification type name config split revision mteb/sprintduplicatequestions-pairclassification MTEB SprintDuplicateQuestions default test d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 type value cos_sim_accuracy 99.8029702970297 type value cos_sim_ap 94.9210324173411 type value cos_sim_f1 89.8521162672106 type value cos_sim_precision 91.67533818938605 type value cos_sim_recall 88.1 type value dot_accuracy 99.69504950495049 type value dot_ap 90.4919719146181 type value dot_f1 84.72289156626506 type value dot_precision 81.76744186046511 type value dot_recall 87.9 type value euclidean_accuracy 99.79702970297029 type value euclidean_ap 94.87827463795753 type value euclidean_f1 89.55680081507896 type value euclidean_precision 91.27725856697819 type value euclidean_recall 87.9 type value manhattan_accuracy 99.7990099009901 type value manhattan_ap 94.87587025149682 type value manhattan_f1 89.76298537569339 type value manhattan_precision 90.53916581892166 type value manhattan_recall 89 type value max_accuracy 99.8029702970297 type value max_ap 94.9210324173411 type value max_f1 89.8521162672106 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering MTEB StackExchangeClustering default test 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 type value v_measure 65.92385753948724 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering-p2p MTEB StackExchangeClusteringP2P default test 815ca46b2622cec33ccafc3735d572c266efdb44 type value v_measure 33.671756975431144 task dataset metrics type Reranking type name config split revision mteb/stackoverflowdupquestions-reranking MTEB StackOverflowDupQuestions default test e185fbe320c72810689fc5848eb6114e1ef5ec69 type value map 50.677928036739004 type value mrr 51.56413133435193 task dataset metrics type Summarization type name config split revision mteb/summeval MTEB SummEval default test cda12ad7615edc362dbf25a00fdd61d3b1eaf93c type value cos_sim_pearson 30.523589340819683 type value cos_sim_spearman 30.187407518823235 type value dot_pearson 29.039713969699015 type value dot_spearman 29.114740651155508 task dataset metrics type Retrieval type name config split revision trec-covid MTEB TRECCOVID default test None type value map_at_1 0.211 type value map_at_10 1.6199999999999999 type value map_at_100 8.658000000000001 type value map_at_1000 21.538 type value map_at_3 0.575 type value map_at_5 0.919 type value mrr_at_1 78 type value mrr_at_10 86.18599999999999 type value mrr_at_100 86.18599999999999 type value mrr_at_1000 86.18599999999999 type value mrr_at_3 85 type value mrr_at_5 85.9 type value ndcg_at_1 74 type value ndcg_at_10 66.542 type value ndcg_at_100 50.163999999999994 type value ndcg_at_1000 45.696999999999996 type value ndcg_at_3 71.531 type value ndcg_at_5 70.45 type value precision_at_1 78 type value precision_at_10 69.39999999999999 type value precision_at_100 51.06 type value precision_at_1000 20.022000000000002 type value precision_at_3 76 type value precision_at_5 74.8 type value recall_at_1 0.211 type value recall_at_10 1.813 type value recall_at_100 12.098 type value recall_at_1000 42.618 type value recall_at_3 0.603 type value recall_at_5 0.987 task dataset metrics type Retrieval type name config split revision webis-touche2020 MTEB Touche2020 default test None type value map_at_1 2.2079999999999997 type value map_at_10 7.777000000000001 type value map_at_100 12.825000000000001 type value map_at_1000 14.196 type value map_at_3 4.285 type value map_at_5 6.177 type value mrr_at_1 30.612000000000002 type value mrr_at_10 42.635 type value mrr_at_100 43.955 type value mrr_at_1000 43.955 type value mrr_at_3 38.435 type value mrr_at_5 41.088 type value ndcg_at_1 28.571 type value ndcg_at_10 20.666999999999998 type value ndcg_at_100 31.840000000000003 type value ndcg_at_1000 43.191 type value ndcg_at_3 23.45 type value ndcg_at_5 22.994 type value precision_at_1 30.612000000000002 type value precision_at_10 17.959 type value precision_at_100 6.755 type value precision_at_1000 1.4200000000000002 type value precision_at_3 23.810000000000002 type value precision_at_5 23.673 type value recall_at_1 2.2079999999999997 type value recall_at_10 13.144 type value recall_at_100 42.491 type value recall_at_1000 77.04299999999999 type value recall_at_3 5.3469999999999995 type value recall_at_5 9.139 task dataset metrics type Classification type name config split revision mteb/toxic_conversations_50k MTEB ToxicConversationsClassification default test d7c0de2777da35d6aae2200a62c6e0e5af397c4c type value accuracy 70.9044 type value ap 14.625783489340755 type value f1 54.814936562590546 task dataset metrics type Classification type name config split revision mteb/tweet_sentiment_extraction MTEB TweetSentimentExtractionClassification default test d604517c81ca91fe16a244d1248fc021f9ecee7a type value accuracy 60.94227504244483 type value f1 61.22516038508854 task dataset metrics type Clustering type name config split revision mteb/twentynewsgroups-clustering MTEB TwentyNewsgroupsClustering default test 6125ec4e24fa026cec8a478383ee943acfbd5449 type value v_measure 49.602409155145864 task dataset metrics type PairClassification type name config split revision mteb/twittersemeval2015-pairclassification MTEB TwitterSemEval2015 default test 70970daeab8776df92f5ea462b6173c0b46fd2d1 type value cos_sim_accuracy 86.94641473445789 type value cos_sim_ap 76.91572747061197 type value cos_sim_f1 70.14348097317529 type value cos_sim_precision 66.53254437869822 type value cos_sim_recall 74.1688654353562 type value dot_accuracy 84.80061989628658 type value dot_ap 70.7952548895177 type value dot_f1 65.44780728844965 type value dot_precision 61.53310104529617 type value dot_recall 69.89445910290237 type value euclidean_accuracy 86.94641473445789 type value euclidean_ap 76.80774009393652 type value euclidean_f1 70.30522503879979 type value euclidean_precision 68.94977168949772 type value euclidean_recall 71.71503957783642 type value manhattan_accuracy 86.8629671574179 type value manhattan_ap 76.76518632600317 type value manhattan_f1 70.16056518946692 type value manhattan_precision 68.360450563204 type value manhattan_recall 72.0580474934037 type value max_accuracy 86.94641473445789 type value max_ap 76.91572747061197 type value max_f1 70.30522503879979 task dataset metrics type PairClassification type name config split revision mteb/twitterurlcorpus-pairclassification MTEB TwitterURLCorpus default test 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf type value cos_sim_accuracy 89.10428066907285 type value cos_sim_ap 86.25114759921435 type value cos_sim_f1 78.37857884586856 type value cos_sim_precision 75.60818546078993 type value cos_sim_recall 81.35971666153372 type value dot_accuracy 87.41995575736406 type value dot_ap 81.51838010086782 type value dot_f1 74.77398015435503 type value dot_precision 71.53002390662354 type value dot_recall 78.32614721281182 type value euclidean_accuracy 89.12368533395428 type value euclidean_ap 86.33456799874504 type value euclidean_f1 78.45496750232127 type value euclidean_precision 75.78388462366364 type value euclidean_recall 81.32121958731136 type value manhattan_accuracy 89.10622113556099 type value manhattan_ap 86.31215061745333 type value manhattan_f1 78.40684906011539 type value manhattan_precision 75.89536643366722 type value manhattan_recall 81.09023714197721 type value max_accuracy 89.12368533395428 type value max_ap 86.33456799874504 type value max_f1 78.45496750232127	en	mit

E5-large-v2

Text Embeddings by Weakly-Supervised Contrastive Pre-training. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022

This model has 24 layers and the embedding size is 1024.

Usage

Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset.

import torch.nn.functional as F

from torch import Tensor
from transformers import AutoTokenizer, AutoModel


def average_pool(last_hidden_states: Tensor,
                 attention_mask: Tensor) -> Tensor:
    last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
    return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]


# Each input text should start with "query: " or "passage: ".
# For tasks other than retrieval, you can simply use the "query: " prefix.
input_texts = ['query: how much protein should a female eat',
               'query: summit define',
               "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
               "passage: Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."]

tokenizer = AutoTokenizer.from_pretrained('intfloat/e5-large-v2')
model = AutoModel.from_pretrained('intfloat/e5-large-v2')

# Tokenize the input texts
batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')

outputs = model(**batch_dict)
embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask'])

# normalize embeddings
embeddings = F.normalize(embeddings, p=2, dim=1)
scores = (embeddings[:2] @ embeddings[2:].T) * 100
print(scores.tolist())

Training Details

Please refer to our paper at https://arxiv.org/pdf/2212.03533.pdf.

Benchmark Evaluation

Check out unilm/e5 to reproduce evaluation results on the BEIR and MTEB benchmark.

Support for Sentence Transformers

Below is an example for usage with sentence_transformers.

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('intfloat/e5-large-v2')
input_texts = [
    'query: how much protein should a female eat',
    'query: summit define',
    "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
    "passage: Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."
]
embeddings = model.encode(input_texts, normalize_embeddings=True)

Package requirements

pip install sentence_transformers~=2.2.2

Contributors: michaelfeil

FAQ

1. Do I need to add the prefix "query: " and "passage: " to input texts?

Yes, this is how the model is trained, otherwise you will see a performance degradation.

Here are some rules of thumb:

• Use "query: " and "passage: " correspondingly for asymmetric tasks such as passage retrieval in open QA, ad-hoc information retrieval.

• Use "query: " prefix for symmetric tasks such as semantic similarity, paraphrase retrieval.

• Use "query: " prefix if you want to use embeddings as features, such as linear probing classification, clustering.

2. Why are my reproduced results slightly different from reported in the model card?

Different versions of transformers and pytorch could cause negligible but non-zero performance differences.

3. Why does the cosine similarity scores distribute around 0.7 to 1.0?

This is a known and expected behavior as we use a low temperature 0.01 for InfoNCE contrastive loss.

For text embedding tasks like text retrieval or semantic similarity, what matters is the relative order of the scores instead of the absolute values, so this should not be an issue.

Citation

If you find our paper or models helpful, please consider cite as follows:

@article{wang2022text,
  title={Text Embeddings by Weakly-Supervised Contrastive Pre-training},
  author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Jiao, Binxing and Yang, Linjun and Jiang, Daxin and Majumder, Rangan and Wei, Furu},
  journal={arXiv preprint arXiv:2212.03533},
  year={2022}
}

Limitations

This model only works for English texts. Long texts will be truncated to at most 512 tokens.