e5-small/README.md

tags, model-index, language, license

tags	model-index	language	license
mteb Sentence Transformers sentence-similarity sentence-transformers	name results e5-small task dataset metrics type Classification type name config split revision mteb/amazon_counterfactual MTEB AmazonCounterfactualClassification (en) en test e8379541af4e31359cca9fbcf4b00f2671dba205 type value accuracy 76.22388059701493 type value ap 40.27466219523129 type value f1 70.60533006025108 task dataset metrics type Classification type name config split revision mteb/amazon_polarity MTEB AmazonPolarityClassification default test e2d317d38cd51312af73b3d32a06d1a08b442046 type value accuracy 87.525775 type value ap 83.51063993897611 type value f1 87.49342736805572 task dataset metrics type Classification type name config split revision mteb/amazon_reviews_multi MTEB AmazonReviewsClassification (en) en test 1399c76144fd37290681b995c656ef9b2e06e26d type value accuracy 42.611999999999995 type value f1 42.05088045932892 task dataset metrics type Retrieval type name config split revision arguana MTEB ArguAna default test None type value map_at_1 23.826 type value map_at_10 38.269 type value map_at_100 39.322 type value map_at_1000 39.344 type value map_at_3 33.428000000000004 type value map_at_5 36.063 type value mrr_at_1 24.253 type value mrr_at_10 38.425 type value mrr_at_100 39.478 type value mrr_at_1000 39.5 type value mrr_at_3 33.606 type value mrr_at_5 36.195 type value ndcg_at_1 23.826 type value ndcg_at_10 46.693 type value ndcg_at_100 51.469 type value ndcg_at_1000 52.002 type value ndcg_at_3 36.603 type value ndcg_at_5 41.365 type value precision_at_1 23.826 type value precision_at_10 7.383000000000001 type value precision_at_100 0.9530000000000001 type value precision_at_1000 0.099 type value precision_at_3 15.268 type value precision_at_5 11.479000000000001 type value recall_at_1 23.826 type value recall_at_10 73.82600000000001 type value recall_at_100 95.306 type value recall_at_1000 99.431 type value recall_at_3 45.804 type value recall_at_5 57.397 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-p2p MTEB ArxivClusteringP2P default test a122ad7f3f0291bf49cc6f4d32aa80929df69d5d type value v_measure 44.13995374767436 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-s2s MTEB ArxivClusteringS2S default test f910caf1a6075f7329cdf8c1a6135696f37dbd53 type value v_measure 37.13950072624313 task dataset metrics type Reranking type name config split revision mteb/askubuntudupquestions-reranking MTEB AskUbuntuDupQuestions default test 2000358ca161889fa9c082cb41daa8dcfb161a54 type value map 59.35843292105327 type value mrr 73.72312359846987 task dataset metrics type STS type name config split revision mteb/biosses-sts MTEB BIOSSES default test d3fb88f8f02e40887cd149695127462bbcf29b4a type value cos_sim_pearson 84.55140418324174 type value cos_sim_spearman 84.21637675860022 type value euclidean_pearson 81.26069614610006 type value euclidean_spearman 83.25069210421785 type value manhattan_pearson 80.17441422581014 type value manhattan_spearman 81.87596198487877 task dataset metrics type Classification type name config split revision mteb/banking77 MTEB Banking77Classification default test 0fd18e25b25c072e09e0d92ab615fda904d66300 type value accuracy 81.87337662337661 type value f1 81.76647866926402 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-p2p MTEB BiorxivClusteringP2P default test 65b79d1d13f80053f67aca9498d9402c2d9f1f40 type value v_measure 35.80600542614507 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-s2s MTEB BiorxivClusteringS2S default test 258694dd0231531bc1fd9de6ceb52a0853c6d908 type value v_measure 31.86321613256603 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackAndroidRetrieval default test None type value map_at_1 32.054 type value map_at_10 40.699999999999996 type value map_at_100 41.818 type value map_at_1000 41.959999999999994 type value map_at_3 37.742 type value map_at_5 39.427 type value mrr_at_1 38.769999999999996 type value mrr_at_10 46.150000000000006 type value mrr_at_100 46.865 type value mrr_at_1000 46.925 type value mrr_at_3 43.705 type value mrr_at_5 45.214999999999996 type value ndcg_at_1 38.769999999999996 type value ndcg_at_10 45.778 type value ndcg_at_100 50.38 type value ndcg_at_1000 52.922999999999995 type value ndcg_at_3 41.597 type value ndcg_at_5 43.631 type value precision_at_1 38.769999999999996 type value precision_at_10 8.269 type value precision_at_100 1.278 type value precision_at_1000 0.178 type value precision_at_3 19.266 type value precision_at_5 13.705 type value recall_at_1 32.054 type value recall_at_10 54.947 type value recall_at_100 74.79599999999999 type value recall_at_1000 91.40899999999999 type value recall_at_3 42.431000000000004 type value recall_at_5 48.519 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackEnglishRetrieval default test None type value map_at_1 29.035 type value map_at_10 38.007000000000005 type value map_at_100 39.125 type value map_at_1000 39.251999999999995 type value map_at_3 35.77 type value map_at_5 37.057 type value mrr_at_1 36.497 type value mrr_at_10 44.077 type value mrr_at_100 44.743 type value mrr_at_1000 44.79 type value mrr_at_3 42.123 type value mrr_at_5 43.308 type value ndcg_at_1 36.497 type value ndcg_at_10 42.986000000000004 type value ndcg_at_100 47.323 type value ndcg_at_1000 49.624 type value ndcg_at_3 39.805 type value ndcg_at_5 41.286 type value precision_at_1 36.497 type value precision_at_10 7.8340000000000005 type value precision_at_100 1.269 type value precision_at_1000 0.178 type value precision_at_3 19.023 type value precision_at_5 13.248 type value recall_at_1 29.035 type value recall_at_10 51.06 type value recall_at_100 69.64099999999999 type value recall_at_1000 84.49 type value recall_at_3 41.333999999999996 type value recall_at_5 45.663 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGamingRetrieval default test None type value map_at_1 37.239 type value map_at_10 47.873 type value map_at_100 48.842999999999996 type value map_at_1000 48.913000000000004 type value map_at_3 45.050000000000004 type value map_at_5 46.498 type value mrr_at_1 42.508 type value mrr_at_10 51.44 type value mrr_at_100 52.087 type value mrr_at_1000 52.129999999999995 type value mrr_at_3 49.164 type value mrr_at_5 50.343 type value ndcg_at_1 42.508 type value ndcg_at_10 53.31399999999999 type value ndcg_at_100 57.245000000000005 type value ndcg_at_1000 58.794000000000004 type value ndcg_at_3 48.295 type value ndcg_at_5 50.415 type value precision_at_1 42.508 type value precision_at_10 8.458 type value precision_at_100 1.133 type value precision_at_1000 0.132 type value precision_at_3 21.191 type value precision_at_5 14.307 type value recall_at_1 37.239 type value recall_at_10 65.99000000000001 type value recall_at_100 82.99499999999999 type value recall_at_1000 94.128 type value recall_at_3 52.382 type value recall_at_5 57.648999999999994 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGisRetrieval default test None type value map_at_1 23.039 type value map_at_10 29.694 type value map_at_100 30.587999999999997 type value map_at_1000 30.692999999999998 type value map_at_3 27.708 type value map_at_5 28.774 type value mrr_at_1 24.633 type value mrr_at_10 31.478 type value mrr_at_100 32.299 type value mrr_at_1000 32.381 type value mrr_at_3 29.435 type value mrr_at_5 30.446 type value ndcg_at_1 24.633 type value ndcg_at_10 33.697 type value ndcg_at_100 38.080000000000005 type value ndcg_at_1000 40.812 type value ndcg_at_3 29.654000000000003 type value ndcg_at_5 31.474000000000004 type value precision_at_1 24.633 type value precision_at_10 5.0729999999999995 type value precision_at_100 0.753 type value precision_at_1000 0.10300000000000001 type value precision_at_3 12.279 type value precision_at_5 8.452 type value recall_at_1 23.039 type value recall_at_10 44.275999999999996 type value recall_at_100 64.4 type value recall_at_1000 85.135 type value recall_at_3 33.394 type value recall_at_5 37.687 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackMathematicaRetrieval default test None type value map_at_1 13.594999999999999 type value map_at_10 19.933999999999997 type value map_at_100 20.966 type value map_at_1000 21.087 type value map_at_3 17.749000000000002 type value map_at_5 19.156000000000002 type value mrr_at_1 17.662 type value mrr_at_10 24.407 type value mrr_at_100 25.385 type value mrr_at_1000 25.465 type value mrr_at_3 22.056 type value mrr_at_5 23.630000000000003 type value ndcg_at_1 17.662 type value ndcg_at_10 24.391 type value ndcg_at_100 29.681 type value ndcg_at_1000 32.923 type value ndcg_at_3 20.271 type value ndcg_at_5 22.621 type value precision_at_1 17.662 type value precision_at_10 4.44 type value precision_at_100 0.8200000000000001 type value precision_at_1000 0.125 type value precision_at_3 9.577 type value precision_at_5 7.313 type value recall_at_1 13.594999999999999 type value recall_at_10 33.976 type value recall_at_100 57.43000000000001 type value recall_at_1000 80.958 type value recall_at_3 22.897000000000002 type value recall_at_5 28.714000000000002 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackPhysicsRetrieval default test None type value map_at_1 26.683 type value map_at_10 35.068 type value map_at_100 36.311 type value map_at_1000 36.436 type value map_at_3 32.371 type value map_at_5 33.761 type value mrr_at_1 32.435 type value mrr_at_10 40.721000000000004 type value mrr_at_100 41.535 type value mrr_at_1000 41.593 type value mrr_at_3 38.401999999999994 type value mrr_at_5 39.567 type value ndcg_at_1 32.435 type value ndcg_at_10 40.538000000000004 type value ndcg_at_100 45.963 type value ndcg_at_1000 48.400999999999996 type value ndcg_at_3 36.048 type value ndcg_at_5 37.899 type value precision_at_1 32.435 type value precision_at_10 7.1129999999999995 type value precision_at_100 1.162 type value precision_at_1000 0.156 type value precision_at_3 16.683 type value precision_at_5 11.684 type value recall_at_1 26.683 type value recall_at_10 51.517 type value recall_at_100 74.553 type value recall_at_1000 90.649 type value recall_at_3 38.495000000000005 type value recall_at_5 43.495 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackProgrammersRetrieval default test None type value map_at_1 24.186 type value map_at_10 31.972 type value map_at_100 33.117000000000004 type value map_at_1000 33.243 type value map_at_3 29.423 type value map_at_5 30.847 type value mrr_at_1 29.794999999999998 type value mrr_at_10 36.767 type value mrr_at_100 37.645 type value mrr_at_1000 37.716 type value mrr_at_3 34.513 type value mrr_at_5 35.791000000000004 type value ndcg_at_1 29.794999999999998 type value ndcg_at_10 36.786 type value ndcg_at_100 41.94 type value ndcg_at_1000 44.830999999999996 type value ndcg_at_3 32.504 type value ndcg_at_5 34.404 type value precision_at_1 29.794999999999998 type value precision_at_10 6.518 type value precision_at_100 1.0659999999999998 type value precision_at_1000 0.149 type value precision_at_3 15.296999999999999 type value precision_at_5 10.731 type value recall_at_1 24.186 type value recall_at_10 46.617 type value recall_at_100 68.75 type value recall_at_1000 88.864 type value recall_at_3 34.199 type value recall_at_5 39.462 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackRetrieval default test None type value map_at_1 24.22083333333333 type value map_at_10 31.606666666666662 type value map_at_100 32.6195 type value map_at_1000 32.739999999999995 type value map_at_3 29.37825 type value map_at_5 30.596083333333336 type value mrr_at_1 28.607916666666668 type value mrr_at_10 35.54591666666666 type value mrr_at_100 36.33683333333333 type value mrr_at_1000 36.40624999999999 type value mrr_at_3 33.526250000000005 type value mrr_at_5 34.6605 type value ndcg_at_1 28.607916666666668 type value ndcg_at_10 36.07966666666667 type value ndcg_at_100 40.73308333333333 type value ndcg_at_1000 43.40666666666666 type value ndcg_at_3 32.23525 type value ndcg_at_5 33.97083333333333 type value precision_at_1 28.607916666666668 type value precision_at_10 6.120333333333335 type value precision_at_100 0.9921666666666668 type value precision_at_1000 0.14091666666666666 type value precision_at_3 14.54975 type value precision_at_5 10.153166666666667 type value recall_at_1 24.22083333333333 type value recall_at_10 45.49183333333334 type value recall_at_100 66.28133333333332 type value recall_at_1000 85.16541666666667 type value recall_at_3 34.6485 type value recall_at_5 39.229749999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackStatsRetrieval default test None type value map_at_1 21.842 type value map_at_10 27.573999999999998 type value map_at_100 28.410999999999998 type value map_at_1000 28.502 type value map_at_3 25.921 type value map_at_5 26.888 type value mrr_at_1 24.08 type value mrr_at_10 29.915999999999997 type value mrr_at_100 30.669 type value mrr_at_1000 30.746000000000002 type value mrr_at_3 28.349000000000004 type value mrr_at_5 29.246 type value ndcg_at_1 24.08 type value ndcg_at_10 30.898999999999997 type value ndcg_at_100 35.272999999999996 type value ndcg_at_1000 37.679 type value ndcg_at_3 27.881 type value ndcg_at_5 29.432000000000002 type value precision_at_1 24.08 type value precision_at_10 4.678 type value precision_at_100 0.744 type value precision_at_1000 0.10300000000000001 type value precision_at_3 11.860999999999999 type value precision_at_5 8.16 type value recall_at_1 21.842 type value recall_at_10 38.66 type value recall_at_100 59.169000000000004 type value recall_at_1000 76.887 type value recall_at_3 30.532999999999998 type value recall_at_5 34.354 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackTexRetrieval default test None type value map_at_1 17.145 type value map_at_10 22.729 type value map_at_100 23.574 type value map_at_1000 23.695 type value map_at_3 21.044 type value map_at_5 21.981 type value mrr_at_1 20.888 type value mrr_at_10 26.529000000000003 type value mrr_at_100 27.308 type value mrr_at_1000 27.389000000000003 type value mrr_at_3 24.868000000000002 type value mrr_at_5 25.825 type value ndcg_at_1 20.888 type value ndcg_at_10 26.457000000000004 type value ndcg_at_100 30.764000000000003 type value ndcg_at_1000 33.825 type value ndcg_at_3 23.483999999999998 type value ndcg_at_5 24.836 type value precision_at_1 20.888 type value precision_at_10 4.58 type value precision_at_100 0.784 type value precision_at_1000 0.121 type value precision_at_3 10.874 type value precision_at_5 7.639 type value recall_at_1 17.145 type value recall_at_10 33.938 type value recall_at_100 53.672 type value recall_at_1000 76.023 type value recall_at_3 25.363000000000003 type value recall_at_5 29.023 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackUnixRetrieval default test None type value map_at_1 24.275 type value map_at_10 30.438 type value map_at_100 31.489 type value map_at_1000 31.601000000000003 type value map_at_3 28.647 type value map_at_5 29.660999999999998 type value mrr_at_1 28.077999999999996 type value mrr_at_10 34.098 type value mrr_at_100 35.025 type value mrr_at_1000 35.109 type value mrr_at_3 32.4 type value mrr_at_5 33.379999999999995 type value ndcg_at_1 28.077999999999996 type value ndcg_at_10 34.271 type value ndcg_at_100 39.352 type value ndcg_at_1000 42.199 type value ndcg_at_3 30.978 type value ndcg_at_5 32.498 type value precision_at_1 28.077999999999996 type value precision_at_10 5.345 type value precision_at_100 0.897 type value precision_at_1000 0.125 type value precision_at_3 13.526 type value precision_at_5 9.16 type value recall_at_1 24.275 type value recall_at_10 42.362 type value recall_at_100 64.461 type value recall_at_1000 84.981 type value recall_at_3 33.249 type value recall_at_5 37.214999999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWebmastersRetrieval default test None type value map_at_1 22.358 type value map_at_10 30.062 type value map_at_100 31.189 type value map_at_1000 31.386999999999997 type value map_at_3 27.672 type value map_at_5 28.76 type value mrr_at_1 26.877000000000002 type value mrr_at_10 33.948 type value mrr_at_100 34.746 type value mrr_at_1000 34.816 type value mrr_at_3 31.884 type value mrr_at_5 33.001000000000005 type value ndcg_at_1 26.877000000000002 type value ndcg_at_10 34.977000000000004 type value ndcg_at_100 39.753 type value ndcg_at_1000 42.866 type value ndcg_at_3 30.956 type value ndcg_at_5 32.381 type value precision_at_1 26.877000000000002 type value precision_at_10 6.7 type value precision_at_100 1.287 type value precision_at_1000 0.215 type value precision_at_3 14.360999999999999 type value precision_at_5 10.119 type value recall_at_1 22.358 type value recall_at_10 44.183 type value recall_at_100 67.14 type value recall_at_1000 87.53999999999999 type value recall_at_3 32.79 type value recall_at_5 36.829 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWordpressRetrieval default test None type value map_at_1 19.198999999999998 type value map_at_10 25.229000000000003 type value map_at_100 26.003 type value map_at_1000 26.111 type value map_at_3 23.442 type value map_at_5 24.343 type value mrr_at_1 21.072 type value mrr_at_10 27.02 type value mrr_at_100 27.735 type value mrr_at_1000 27.815 type value mrr_at_3 25.416 type value mrr_at_5 26.173999999999996 type value ndcg_at_1 21.072 type value ndcg_at_10 28.862 type value ndcg_at_100 33.043 type value ndcg_at_1000 36.003 type value ndcg_at_3 25.35 type value ndcg_at_5 26.773000000000003 type value precision_at_1 21.072 type value precision_at_10 4.436 type value precision_at_100 0.713 type value precision_at_1000 0.106 type value precision_at_3 10.659 type value precision_at_5 7.32 type value recall_at_1 19.198999999999998 type value recall_at_10 38.376 type value recall_at_100 58.36900000000001 type value recall_at_1000 80.92099999999999 type value recall_at_3 28.715000000000003 type value recall_at_5 32.147 task dataset metrics type Retrieval type name config split revision climate-fever MTEB ClimateFEVER default test None type value map_at_1 5.9319999999999995 type value map_at_10 10.483 type value map_at_100 11.97 type value map_at_1000 12.171999999999999 type value map_at_3 8.477 type value map_at_5 9.495000000000001 type value mrr_at_1 13.094 type value mrr_at_10 21.282 type value mrr_at_100 22.556 type value mrr_at_1000 22.628999999999998 type value mrr_at_3 18.218999999999998 type value mrr_at_5 19.900000000000002 type value ndcg_at_1 13.094 type value ndcg_at_10 15.811 type value ndcg_at_100 23.035 type value ndcg_at_1000 27.089999999999996 type value ndcg_at_3 11.905000000000001 type value ndcg_at_5 13.377 type value precision_at_1 13.094 type value precision_at_10 5.225 type value precision_at_100 1.2970000000000002 type value precision_at_1000 0.203 type value precision_at_3 8.86 type value precision_at_5 7.309 type value recall_at_1 5.9319999999999995 type value recall_at_10 20.305 type value recall_at_100 46.314 type value recall_at_1000 69.612 type value recall_at_3 11.21 type value recall_at_5 14.773 task dataset metrics type Retrieval type name config split revision dbpedia-entity MTEB DBPedia default test None type value map_at_1 8.674 type value map_at_10 17.822 type value map_at_100 24.794 type value map_at_1000 26.214 type value map_at_3 12.690999999999999 type value map_at_5 15.033 type value mrr_at_1 61.75000000000001 type value mrr_at_10 71.58 type value mrr_at_100 71.923 type value mrr_at_1000 71.932 type value mrr_at_3 70.125 type value mrr_at_5 71.038 type value ndcg_at_1 51 type value ndcg_at_10 38.637 type value ndcg_at_100 42.398 type value ndcg_at_1000 48.962 type value ndcg_at_3 43.29 type value ndcg_at_5 40.763 type value precision_at_1 61.75000000000001 type value precision_at_10 30.125 type value precision_at_100 9.53 type value precision_at_1000 1.9619999999999997 type value precision_at_3 45.583 type value precision_at_5 38.95 type value recall_at_1 8.674 type value recall_at_10 23.122 type value recall_at_100 47.46 type value recall_at_1000 67.662 type value recall_at_3 13.946 type value recall_at_5 17.768 task dataset metrics type Classification type name config split revision mteb/emotion MTEB EmotionClassification default test 4f58c6b202a23cf9a4da393831edf4f9183cad37 type value accuracy 46.86000000000001 type value f1 41.343580452760776 task dataset metrics type Retrieval type name config split revision fever MTEB FEVER default test None type value map_at_1 36.609 type value map_at_10 47.552 type value map_at_100 48.283 type value map_at_1000 48.321 type value map_at_3 44.869 type value map_at_5 46.509 type value mrr_at_1 39.214 type value mrr_at_10 50.434999999999995 type value mrr_at_100 51.122 type value mrr_at_1000 51.151 type value mrr_at_3 47.735 type value mrr_at_5 49.394 type value ndcg_at_1 39.214 type value ndcg_at_10 53.52400000000001 type value ndcg_at_100 56.997 type value ndcg_at_1000 57.975 type value ndcg_at_3 48.173 type value ndcg_at_5 51.05800000000001 type value precision_at_1 39.214 type value precision_at_10 7.573 type value precision_at_100 0.9440000000000001 type value precision_at_1000 0.104 type value precision_at_3 19.782 type value precision_at_5 13.453000000000001 type value recall_at_1 36.609 type value recall_at_10 69.247 type value recall_at_100 84.99600000000001 type value recall_at_1000 92.40899999999999 type value recall_at_3 54.856 type value recall_at_5 61.797000000000004 task dataset metrics type Retrieval type name config split revision fiqa MTEB FiQA2018 default test None type value map_at_1 16.466 type value map_at_10 27.060000000000002 type value map_at_100 28.511999999999997 type value map_at_1000 28.693 type value map_at_3 22.777 type value map_at_5 25.086000000000002 type value mrr_at_1 32.716 type value mrr_at_10 41.593999999999994 type value mrr_at_100 42.370000000000005 type value mrr_at_1000 42.419000000000004 type value mrr_at_3 38.143 type value mrr_at_5 40.288000000000004 type value ndcg_at_1 32.716 type value ndcg_at_10 34.795 type value ndcg_at_100 40.58 type value ndcg_at_1000 43.993 type value ndcg_at_3 29.573 type value ndcg_at_5 31.583 type value precision_at_1 32.716 type value precision_at_10 9.937999999999999 type value precision_at_100 1.585 type value precision_at_1000 0.22 type value precision_at_3 19.496 type value precision_at_5 15.247 type value recall_at_1 16.466 type value recall_at_10 42.886 type value recall_at_100 64.724 type value recall_at_1000 85.347 type value recall_at_3 26.765 type value recall_at_5 33.603 task dataset metrics type Retrieval type name config split revision hotpotqa MTEB HotpotQA default test None type value map_at_1 33.025 type value map_at_10 47.343 type value map_at_100 48.207 type value map_at_1000 48.281 type value map_at_3 44.519 type value map_at_5 46.217000000000006 type value mrr_at_1 66.05 type value mrr_at_10 72.94699999999999 type value mrr_at_100 73.289 type value mrr_at_1000 73.30499999999999 type value mrr_at_3 71.686 type value mrr_at_5 72.491 type value ndcg_at_1 66.05 type value ndcg_at_10 56.338 type value ndcg_at_100 59.599999999999994 type value ndcg_at_1000 61.138000000000005 type value ndcg_at_3 52.034000000000006 type value ndcg_at_5 54.352000000000004 type value precision_at_1 66.05 type value precision_at_10 11.693000000000001 type value precision_at_100 1.425 type value precision_at_1000 0.163 type value precision_at_3 32.613 type value precision_at_5 21.401999999999997 type value recall_at_1 33.025 type value recall_at_10 58.467 type value recall_at_100 71.242 type value recall_at_1000 81.452 type value recall_at_3 48.92 type value recall_at_5 53.504 task dataset metrics type Classification type name config split revision mteb/imdb MTEB ImdbClassification default test 3d86128a09e091d6018b6d26cad27f2739fc2db7 type value accuracy 75.5492 type value ap 69.42911637216271 type value f1 75.39113704261024 task dataset metrics type Retrieval type name config split revision msmarco MTEB MSMARCO default dev None type value map_at_1 23.173 type value map_at_10 35.453 type value map_at_100 36.573 type value map_at_1000 36.620999999999995 type value map_at_3 31.655 type value map_at_5 33.823 type value mrr_at_1 23.868000000000002 type value mrr_at_10 36.085 type value mrr_at_100 37.15 type value mrr_at_1000 37.193 type value mrr_at_3 32.376 type value mrr_at_5 34.501 type value ndcg_at_1 23.854 type value ndcg_at_10 42.33 type value ndcg_at_100 47.705999999999996 type value ndcg_at_1000 48.91 type value ndcg_at_3 34.604 type value ndcg_at_5 38.473 type value precision_at_1 23.854 type value precision_at_10 6.639 type value precision_at_100 0.932 type value precision_at_1000 0.104 type value precision_at_3 14.685 type value precision_at_5 10.782 type value recall_at_1 23.173 type value recall_at_10 63.441 type value recall_at_100 88.25 type value recall_at_1000 97.438 type value recall_at_3 42.434 type value recall_at_5 51.745 task dataset metrics type Classification type name config split revision mteb/mtop_domain MTEB MTOPDomainClassification (en) en test d80d48c1eb48d3562165c59d59d0034df9fff0bf type value accuracy 92.05426356589147 type value f1 91.88068588063942 task dataset metrics type Classification type name config split revision mteb/mtop_intent MTEB MTOPIntentClassification (en) en test ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba type value accuracy 73.23985408116735 type value f1 55.858906745287506 task dataset metrics type Classification type name config split revision mteb/amazon_massive_intent MTEB MassiveIntentClassification (en) en test 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 type value accuracy 72.21923335574984 type value f1 70.0174116204253 task dataset metrics type Classification type name config split revision mteb/amazon_massive_scenario MTEB MassiveScenarioClassification (en) en test 7d571f92784cd94a019292a1f45445077d0ef634 type value accuracy 75.77673167451245 type value f1 75.44811354778666 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-p2p MTEB MedrxivClusteringP2P default test e7a26af6f3ae46b30dde8737f02c07b1505bcc73 type value v_measure 31.340414710728737 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-s2s MTEB MedrxivClusteringS2S default test 35191c8c0dca72d8ff3efcd72aa802307d469663 type value v_measure 28.196676760061578 task dataset metrics type Reranking type name config split revision mteb/mind_small MTEB MindSmallReranking default test 3bdac13927fdc888b903db93b2ffdbd90b295a69 type value map 29.564149683482206 type value mrr 30.28995474250486 task dataset metrics type Retrieval type name config split revision nfcorpus MTEB NFCorpus default test None type value map_at_1 5.93 type value map_at_10 12.828000000000001 type value map_at_100 15.501000000000001 type value map_at_1000 16.791 type value map_at_3 9.727 type value map_at_5 11.318999999999999 type value mrr_at_1 47.678 type value mrr_at_10 55.893 type value mrr_at_100 56.491 type value mrr_at_1000 56.53 type value mrr_at_3 54.386 type value mrr_at_5 55.516 type value ndcg_at_1 45.975 type value ndcg_at_10 33.928999999999995 type value ndcg_at_100 30.164 type value ndcg_at_1000 38.756 type value ndcg_at_3 41.077000000000005 type value ndcg_at_5 38.415 type value precision_at_1 47.678 type value precision_at_10 24.365000000000002 type value precision_at_100 7.344 type value precision_at_1000 1.994 type value precision_at_3 38.184000000000005 type value precision_at_5 33.003 type value recall_at_1 5.93 type value recall_at_10 16.239 type value recall_at_100 28.782999999999998 type value recall_at_1000 60.11 type value recall_at_3 10.700999999999999 type value recall_at_5 13.584 task dataset metrics type Retrieval type name config split revision nq MTEB NQ default test None type value map_at_1 36.163000000000004 type value map_at_10 51.520999999999994 type value map_at_100 52.449 type value map_at_1000 52.473000000000006 type value map_at_3 47.666 type value map_at_5 50.043000000000006 type value mrr_at_1 40.266999999999996 type value mrr_at_10 54.074 type value mrr_at_100 54.722 type value mrr_at_1000 54.739000000000004 type value mrr_at_3 51.043000000000006 type value mrr_at_5 52.956 type value ndcg_at_1 40.238 type value ndcg_at_10 58.73199999999999 type value ndcg_at_100 62.470000000000006 type value ndcg_at_1000 63.083999999999996 type value ndcg_at_3 51.672 type value ndcg_at_5 55.564 type value precision_at_1 40.238 type value precision_at_10 9.279 type value precision_at_100 1.139 type value precision_at_1000 0.12 type value precision_at_3 23.078000000000003 type value precision_at_5 16.176 type value recall_at_1 36.163000000000004 type value recall_at_10 77.88199999999999 type value recall_at_100 93.83399999999999 type value recall_at_1000 98.465 type value recall_at_3 59.857000000000006 type value recall_at_5 68.73599999999999 task dataset metrics type Retrieval type name config split revision quora MTEB QuoraRetrieval default test None type value map_at_1 70.344 type value map_at_10 83.907 type value map_at_100 84.536 type value map_at_1000 84.557 type value map_at_3 80.984 type value map_at_5 82.844 type value mrr_at_1 81.02000000000001 type value mrr_at_10 87.158 type value mrr_at_100 87.268 type value mrr_at_1000 87.26899999999999 type value mrr_at_3 86.17 type value mrr_at_5 86.87 type value ndcg_at_1 81.02000000000001 type value ndcg_at_10 87.70700000000001 type value ndcg_at_100 89.004 type value ndcg_at_1000 89.139 type value ndcg_at_3 84.841 type value ndcg_at_5 86.455 type value precision_at_1 81.02000000000001 type value precision_at_10 13.248999999999999 type value precision_at_100 1.516 type value precision_at_1000 0.156 type value precision_at_3 36.963 type value precision_at_5 24.33 type value recall_at_1 70.344 type value recall_at_10 94.75099999999999 type value recall_at_100 99.30499999999999 type value recall_at_1000 99.928 type value recall_at_3 86.506 type value recall_at_5 91.083 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering MTEB RedditClustering default test 24640382cdbf8abc73003fb0fa6d111a705499eb type value v_measure 42.873718018378305 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering-p2p MTEB RedditClusteringP2P default test 282350215ef01743dc01b456c7f5241fa8937f16 type value v_measure 56.39477366450528 task dataset metrics type Retrieval type name config split revision scidocs MTEB SCIDOCS default test None type value map_at_1 3.868 type value map_at_10 9.611 type value map_at_100 11.087 type value map_at_1000 11.332 type value map_at_3 6.813 type value map_at_5 8.233 type value mrr_at_1 19 type value mrr_at_10 28.457 type value mrr_at_100 29.613 type value mrr_at_1000 29.695 type value mrr_at_3 25.55 type value mrr_at_5 27.29 type value ndcg_at_1 19 type value ndcg_at_10 16.419 type value ndcg_at_100 22.817999999999998 type value ndcg_at_1000 27.72 type value ndcg_at_3 15.379000000000001 type value ndcg_at_5 13.645 type value precision_at_1 19 type value precision_at_10 8.540000000000001 type value precision_at_100 1.7819999999999998 type value precision_at_1000 0.297 type value precision_at_3 14.267 type value precision_at_5 12.04 type value recall_at_1 3.868 type value recall_at_10 17.288 type value recall_at_100 36.144999999999996 type value recall_at_1000 60.199999999999996 type value recall_at_3 8.688 type value recall_at_5 12.198 task dataset metrics type STS type name config split revision mteb/sickr-sts MTEB SICK-R default test a6ea5a8cab320b040a23452cc28066d9beae2cee type value cos_sim_pearson 83.96614722598582 type value cos_sim_spearman 78.9003023008781 type value euclidean_pearson 81.01829384436505 type value euclidean_spearman 78.93248416788914 type value manhattan_pearson 81.1665428926402 type value manhattan_spearman 78.93264116287453 task dataset metrics type STS type name config split revision mteb/sts12-sts MTEB STS12 default test a0d554a64d88156834ff5ae9920b964011b16384 type value cos_sim_pearson 83.54613363895993 type value cos_sim_spearman 75.1883451602451 type value euclidean_pearson 79.70320886899894 type value euclidean_spearman 74.5917140136796 type value manhattan_pearson 79.82157067185999 type value manhattan_spearman 74.74185720594735 task dataset metrics type STS type name config split revision mteb/sts13-sts MTEB STS13 default test 7e90230a92c190f1bf69ae9002b8cea547a64cca type value cos_sim_pearson 81.30430156721782 type value cos_sim_spearman 81.79962989974364 type value euclidean_pearson 80.89058823224924 type value euclidean_spearman 81.35929372984597 type value manhattan_pearson 81.12204370487478 type value manhattan_spearman 81.6248963282232 task dataset metrics type STS type name config split revision mteb/sts14-sts MTEB STS14 default test 6031580fec1f6af667f0bd2da0a551cf4f0b2375 type value cos_sim_pearson 81.13064504403134 type value cos_sim_spearman 78.48371403924872 type value euclidean_pearson 80.16794919665591 type value euclidean_spearman 78.29216082221699 type value manhattan_pearson 80.22308565207301 type value manhattan_spearman 78.37829229948022 task dataset metrics type STS type name config split revision mteb/sts15-sts MTEB STS15 default test ae752c7c21bf194d8b67fd573edf7ae58183cbe3 type value cos_sim_pearson 86.52918899541099 type value cos_sim_spearman 87.49276894673142 type value euclidean_pearson 86.77440570164254 type value euclidean_spearman 87.5753295736756 type value manhattan_pearson 86.86098573892133 type value manhattan_spearman 87.65848591821947 task dataset metrics type STS type name config split revision mteb/sts16-sts MTEB STS16 default test 4d8694f8f0e0100860b497b999b3dbed754a0513 type value cos_sim_pearson 82.86805307244882 type value cos_sim_spearman 84.58066253757511 type value euclidean_pearson 84.38377000876991 type value euclidean_spearman 85.1837278784528 type value manhattan_pearson 84.41903291363842 type value manhattan_spearman 85.19023736251052 task dataset metrics type STS type name config split revision mteb/sts17-crosslingual-sts MTEB STS17 (en-en) en-en test af5e6fb845001ecf41f4c1e033ce921939a2a68d type value cos_sim_pearson 86.77218560282436 type value cos_sim_spearman 87.94243515296604 type value euclidean_pearson 88.22800939214864 type value euclidean_spearman 87.91106839439841 type value manhattan_pearson 88.17063269848741 type value manhattan_spearman 87.72751904126062 task dataset metrics type STS type name config split revision mteb/sts22-crosslingual-sts MTEB STS22 (en) en test 6d1ba47164174a496b7fa5d3569dae26a6813b80 type value cos_sim_pearson 60.40731554300387 type value cos_sim_spearman 63.76300532966479 type value euclidean_pearson 62.94727878229085 type value euclidean_spearman 63.678039531461216 type value manhattan_pearson 63.00661039863549 type value manhattan_spearman 63.6282591984376 task dataset metrics type STS type name config split revision mteb/stsbenchmark-sts MTEB STSBenchmark default test b0fddb56ed78048fa8b90373c8a3cfc37b684831 type value cos_sim_pearson 84.92731569745344 type value cos_sim_spearman 86.36336704300167 type value euclidean_pearson 86.09122224841195 type value euclidean_spearman 86.2116149319238 type value manhattan_pearson 86.07879456717032 type value manhattan_spearman 86.2022069635119 task dataset metrics type Reranking type name config split revision mteb/scidocs-reranking MTEB SciDocsRR default test d3c5e1fc0b855ab6097bf1cda04dd73947d7caab type value map 79.75976311752326 type value mrr 94.15782837351466 task dataset metrics type Retrieval type name config split revision scifact MTEB SciFact default test None type value map_at_1 51.193999999999996 type value map_at_10 61.224999999999994 type value map_at_100 62.031000000000006 type value map_at_1000 62.066 type value map_at_3 59.269000000000005 type value map_at_5 60.159 type value mrr_at_1 53.667 type value mrr_at_10 62.74999999999999 type value mrr_at_100 63.39399999999999 type value mrr_at_1000 63.425 type value mrr_at_3 61.389 type value mrr_at_5 61.989000000000004 type value ndcg_at_1 53.667 type value ndcg_at_10 65.596 type value ndcg_at_100 68.906 type value ndcg_at_1000 69.78999999999999 type value ndcg_at_3 62.261 type value ndcg_at_5 63.453 type value precision_at_1 53.667 type value precision_at_10 8.667 type value precision_at_100 1.04 type value precision_at_1000 0.11100000000000002 type value precision_at_3 24.556 type value precision_at_5 15.6 type value recall_at_1 51.193999999999996 type value recall_at_10 77.156 type value recall_at_100 91.43299999999999 type value recall_at_1000 98.333 type value recall_at_3 67.994 type value recall_at_5 71.14399999999999 task dataset metrics type PairClassification type name config split revision mteb/sprintduplicatequestions-pairclassification MTEB SprintDuplicateQuestions default test d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 type value cos_sim_accuracy 99.81485148514851 type value cos_sim_ap 95.28896513388551 type value cos_sim_f1 90.43478260869566 type value cos_sim_precision 92.56544502617801 type value cos_sim_recall 88.4 type value dot_accuracy 99.30594059405941 type value dot_ap 61.6432597455472 type value dot_f1 59.46481665014866 type value dot_precision 58.93909626719057 type value dot_recall 60 type value euclidean_accuracy 99.81980198019802 type value euclidean_ap 95.21411049527 type value euclidean_f1 91.06090373280944 type value euclidean_precision 89.47876447876449 type value euclidean_recall 92.7 type value manhattan_accuracy 99.81782178217821 type value manhattan_ap 95.32449994414968 type value manhattan_f1 90.86395233366436 type value manhattan_precision 90.23668639053254 type value manhattan_recall 91.5 type value max_accuracy 99.81980198019802 type value max_ap 95.32449994414968 type value max_f1 91.06090373280944 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering MTEB StackExchangeClustering default test 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 type value v_measure 59.08045614613064 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering-p2p MTEB StackExchangeClusteringP2P default test 815ca46b2622cec33ccafc3735d572c266efdb44 type value v_measure 30.297802606804748 task dataset metrics type Reranking type name config split revision mteb/stackoverflowdupquestions-reranking MTEB StackOverflowDupQuestions default test e185fbe320c72810689fc5848eb6114e1ef5ec69 type value map 49.12801740706292 type value mrr 50.05592956879722 task dataset metrics type Summarization type name config split revision mteb/summeval MTEB SummEval default test cda12ad7615edc362dbf25a00fdd61d3b1eaf93c type value cos_sim_pearson 31.523347880124497 type value cos_sim_spearman 31.388214436391014 type value dot_pearson 24.55403435439901 type value dot_spearman 23.50153210841191 task dataset metrics type Retrieval type name config split revision trec-covid MTEB TRECCOVID default test None type value map_at_1 0.243 type value map_at_10 1.886 type value map_at_100 10.040000000000001 type value map_at_1000 23.768 type value map_at_3 0.674 type value map_at_5 1.079 type value mrr_at_1 88 type value mrr_at_10 93.667 type value mrr_at_100 93.667 type value mrr_at_1000 93.667 type value mrr_at_3 93.667 type value mrr_at_5 93.667 type value ndcg_at_1 83 type value ndcg_at_10 76.777 type value ndcg_at_100 55.153 type value ndcg_at_1000 47.912 type value ndcg_at_3 81.358 type value ndcg_at_5 80.74799999999999 type value precision_at_1 88 type value precision_at_10 80.80000000000001 type value precision_at_100 56.02 type value precision_at_1000 21.51 type value precision_at_3 86 type value precision_at_5 86 type value recall_at_1 0.243 type value recall_at_10 2.0869999999999997 type value recall_at_100 13.014000000000001 type value recall_at_1000 44.433 type value recall_at_3 0.6910000000000001 type value recall_at_5 1.1440000000000001 task dataset metrics type Retrieval type name config split revision webis-touche2020 MTEB Touche2020 default test None type value map_at_1 3.066 type value map_at_10 10.615 type value map_at_100 16.463 type value map_at_1000 17.815 type value map_at_3 5.7860000000000005 type value map_at_5 7.353999999999999 type value mrr_at_1 38.775999999999996 type value mrr_at_10 53.846000000000004 type value mrr_at_100 54.37 type value mrr_at_1000 54.37 type value mrr_at_3 48.980000000000004 type value mrr_at_5 51.735 type value ndcg_at_1 34.694 type value ndcg_at_10 26.811 type value ndcg_at_100 37.342999999999996 type value ndcg_at_1000 47.964 type value ndcg_at_3 30.906 type value ndcg_at_5 27.77 type value precision_at_1 38.775999999999996 type value precision_at_10 23.878 type value precision_at_100 7.632999999999999 type value precision_at_1000 1.469 type value precision_at_3 31.973000000000003 type value precision_at_5 26.939 type value recall_at_1 3.066 type value recall_at_10 17.112 type value recall_at_100 47.723 type value recall_at_1000 79.50500000000001 type value recall_at_3 6.825 type value recall_at_5 9.584 task dataset metrics type Classification type name config split revision mteb/toxic_conversations_50k MTEB ToxicConversationsClassification default test d7c0de2777da35d6aae2200a62c6e0e5af397c4c type value accuracy 72.76460000000002 type value ap 14.944240012137053 type value f1 55.89805777266571 task dataset metrics type Classification type name config split revision mteb/tweet_sentiment_extraction MTEB TweetSentimentExtractionClassification default test d604517c81ca91fe16a244d1248fc021f9ecee7a type value accuracy 63.30503678551217 type value f1 63.57492701921179 task dataset metrics type Clustering type name config split revision mteb/twentynewsgroups-clustering MTEB TwentyNewsgroupsClustering default test 6125ec4e24fa026cec8a478383ee943acfbd5449 type value v_measure 37.51066495006874 task dataset metrics type PairClassification type name config split revision mteb/twittersemeval2015-pairclassification MTEB TwitterSemEval2015 default test 70970daeab8776df92f5ea462b6173c0b46fd2d1 type value cos_sim_accuracy 86.07021517553794 type value cos_sim_ap 74.15520712370555 type value cos_sim_f1 68.64321608040201 type value cos_sim_precision 65.51558752997602 type value cos_sim_recall 72.0844327176781 type value dot_accuracy 80.23484532395541 type value dot_ap 54.298763810214176 type value dot_f1 53.22254659779924 type value dot_precision 46.32525410476936 type value dot_recall 62.532981530343015 type value euclidean_accuracy 86.04637301066937 type value euclidean_ap 73.85333854233123 type value euclidean_f1 68.77723660599845 type value euclidean_precision 66.87437686939182 type value euclidean_recall 70.79155672823218 type value manhattan_accuracy 85.98676759849795 type value manhattan_ap 73.56016090035973 type value manhattan_f1 68.48878539036647 type value manhattan_precision 63.9505607690547 type value manhattan_recall 73.7203166226913 type value max_accuracy 86.07021517553794 type value max_ap 74.15520712370555 type value max_f1 68.77723660599845 task dataset metrics type PairClassification type name config split revision mteb/twitterurlcorpus-pairclassification MTEB TwitterURLCorpus default test 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf type value cos_sim_accuracy 88.92769821865176 type value cos_sim_ap 85.78879502899773 type value cos_sim_f1 78.14414083990464 type value cos_sim_precision 74.61651607480563 type value cos_sim_recall 82.0218663381583 type value dot_accuracy 84.95750378390964 type value dot_ap 75.80219641857563 type value dot_f1 70.13966179585681 type value dot_precision 65.71140262361251 type value dot_recall 75.20788420080073 type value euclidean_accuracy 88.93546008460433 type value euclidean_ap 85.72056428301667 type value euclidean_f1 78.14387902598124 type value euclidean_precision 75.3376688344172 type value euclidean_recall 81.16723129042192 type value manhattan_accuracy 88.96262661543835 type value manhattan_ap 85.76605136314335 type value manhattan_f1 78.26696165191743 type value manhattan_precision 75.0990659496179 type value manhattan_recall 81.71388974437943 type value max_accuracy 88.96262661543835 type value max_ap 85.78879502899773 type value max_f1 78.26696165191743	en	mit

E5-small

News (May 2023): please switch to e5-small-v2, which has better performance and same method of usage.

Text Embeddings by Weakly-Supervised Contrastive Pre-training. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022

This model has 12 layers and the embedding size is 384.

Usage

Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset.

import torch.nn.functional as F

from torch import Tensor
from transformers import AutoTokenizer, AutoModel


def average_pool(last_hidden_states: Tensor,
                 attention_mask: Tensor) -> Tensor:
    last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
    return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]


# Each input text should start with "query: " or "passage: ".
# For tasks other than retrieval, you can simply use the "query: " prefix.
input_texts = ['query: how much protein should a female eat',
               'query: summit define',
               "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
               "passage: Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."]

tokenizer = AutoTokenizer.from_pretrained('intfloat/e5-small')
model = AutoModel.from_pretrained('intfloat/e5-small')

# Tokenize the input texts
batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')

outputs = model(**batch_dict)
embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask'])

# normalize embeddings
embeddings = F.normalize(embeddings, p=2, dim=1)
scores = (embeddings[:2] @ embeddings[2:].T) * 100
print(scores.tolist())

Training Details

Please refer to our paper at https://arxiv.org/pdf/2212.03533.pdf.

Benchmark Evaluation

Check out unilm/e5 to reproduce evaluation results on the BEIR and MTEB benchmark.

Support for Sentence Transformers

Below is an example for usage with sentence_transformers.

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('intfloat/e5-small')
input_texts = [
    'query: how much protein should a female eat',
    'query: summit define',
    "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
    "passage: Definition of summit for English Language Learners. : 1  the highest point of a mountain : the top of a mountain. : 2  the highest level. : 3  a meeting or series of meetings between the leaders of two or more governments."
]
embeddings = model.encode(input_texts, normalize_embeddings=True)

Package requirements

pip install sentence_transformers~=2.2.2

Contributors: michaelfeil

FAQ

1. Do I need to add the prefix "query: " and "passage: " to input texts?

Yes, this is how the model is trained, otherwise you will see a performance degradation.

Here are some rules of thumb:

• Use "query: " and "passage: " correspondingly for asymmetric tasks such as passage retrieval in open QA, ad-hoc information retrieval.

• Use "query: " prefix for symmetric tasks such as semantic similarity, paraphrase retrieval.

• Use "query: " prefix if you want to use embeddings as features, such as linear probing classification, clustering.

2. Why are my reproduced results slightly different from reported in the model card?

Different versions of transformers and pytorch could cause negligible but non-zero performance differences.

3. Why does the cosine similarity scores distribute around 0.7 to 1.0?

This is a known and expected behavior as we use a low temperature 0.01 for InfoNCE contrastive loss.

For text embedding tasks like text retrieval or semantic similarity, what matters is the relative order of the scores instead of the absolute values, so this should not be an issue.

Citation

If you find our paper or models helpful, please consider cite as follows:

@article{wang2022text,
  title={Text Embeddings by Weakly-Supervised Contrastive Pre-training},
  author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Jiao, Binxing and Yang, Linjun and Jiang, Daxin and Majumder, Rangan and Wei, Furu},
  journal={arXiv preprint arXiv:2212.03533},
  year={2022}
}

Limitations

This model only works for English texts. Long texts will be truncated to at most 512 tokens.