SGPT-2.7B-weightedmean-msmarco-specb-bitfit/README.md

pipeline_tag, tags, model-index

pipeline_tag	tags	model-index
sentence-similarity	sentence-transformers feature-extraction sentence-similarity mteb	name results SGPT-2.7B-weightedmean-msmarco-specb-bitfit task dataset metrics type Classification type name config split revision mteb/amazon_counterfactual MTEB AmazonCounterfactualClassification (en) en test 2d8a100785abf0ae21420d2a55b0c56e3e1ea996 type value accuracy 67.56716417910448 type value ap 30.75574629595259 type value f1 61.805121301858655 task dataset metrics type Classification type name config split revision mteb/amazon_polarity MTEB AmazonPolarityClassification default test 80714f8dcf8cefc218ef4f8c5a966dd83f75a0e1 type value accuracy 71.439575 type value ap 65.91341330532453 type value f1 70.90561852619555 task dataset metrics type Classification type name config split revision mteb/amazon_reviews_multi MTEB AmazonReviewsClassification (en) en test c379a6705fec24a2493fa68e011692605f44e119 type value accuracy 35.748000000000005 type value f1 35.48576287186347 task dataset metrics type Retrieval type name config split revision arguana MTEB ArguAna default test 5b3e3697907184a9b77a3c99ee9ea1a9cbb1e4e3 type value map_at_1 25.96 type value map_at_10 41.619 type value map_at_100 42.673 type value map_at_1000 42.684 type value map_at_3 36.569 type value map_at_5 39.397 type value mrr_at_1 26.316 type value mrr_at_10 41.772 type value mrr_at_100 42.82 type value mrr_at_1000 42.83 type value mrr_at_3 36.724000000000004 type value mrr_at_5 39.528999999999996 type value ndcg_at_1 25.96 type value ndcg_at_10 50.491 type value ndcg_at_100 54.864999999999995 type value ndcg_at_1000 55.10699999999999 type value ndcg_at_3 40.053 type value ndcg_at_5 45.134 type value precision_at_1 25.96 type value precision_at_10 7.8950000000000005 type value precision_at_100 0.9780000000000001 type value precision_at_1000 0.1 type value precision_at_3 16.714000000000002 type value precision_at_5 12.489 type value recall_at_1 25.96 type value recall_at_10 78.947 type value recall_at_100 97.795 type value recall_at_1000 99.644 type value recall_at_3 50.141999999999996 type value recall_at_5 62.446999999999996 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-p2p MTEB ArxivClusteringP2P default test 0bbdb47bcbe3a90093699aefeed338a0f28a7ee8 type value v_measure 44.72125714642202 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-s2s MTEB ArxivClusteringS2S default test b73bd54100e5abfa6e3a23dcafb46fe4d2438dc3 type value v_measure 35.081451519142064 task dataset metrics type Reranking type name config split revision mteb/askubuntudupquestions-reranking MTEB AskUbuntuDupQuestions default test 4d853f94cd57d85ec13805aeeac3ae3e5eb4c49c type value map 59.634661990392054 type value mrr 73.6813525040672 task dataset metrics type STS type name config split revision mteb/biosses-sts MTEB BIOSSES default test 9ee918f184421b6bd48b78f6c714d86546106103 type value cos_sim_pearson 87.42754550496836 type value cos_sim_spearman 84.84289705838664 type value euclidean_pearson 85.59331970450859 type value euclidean_spearman 85.8525586184271 type value manhattan_pearson 85.41233134466698 type value manhattan_spearman 85.52303303767404 task dataset metrics type Classification type name config split revision mteb/banking77 MTEB Banking77Classification default test 44fa15921b4c889113cc5df03dd4901b49161ab7 type value accuracy 83.21753246753246 type value f1 83.15394543120915 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-p2p MTEB BiorxivClusteringP2P default test 11d0121201d1f1f280e8cc8f3d98fb9c4d9f9c55 type value v_measure 34.41414219680629 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-s2s MTEB BiorxivClusteringS2S default test c0fab014e1bcb8d3a5e31b2088972a1e01547dc1 type value v_measure 30.533275862270028 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackAndroidRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 30.808999999999997 type value map_at_10 40.617 type value map_at_100 41.894999999999996 type value map_at_1000 42.025 type value map_at_3 37.0 type value map_at_5 38.993 type value mrr_at_1 37.482 type value mrr_at_10 46.497 type value mrr_at_100 47.144000000000005 type value mrr_at_1000 47.189 type value mrr_at_3 43.705 type value mrr_at_5 45.193 type value ndcg_at_1 37.482 type value ndcg_at_10 46.688 type value ndcg_at_100 51.726000000000006 type value ndcg_at_1000 53.825 type value ndcg_at_3 41.242000000000004 type value ndcg_at_5 43.657000000000004 type value precision_at_1 37.482 type value precision_at_10 8.827 type value precision_at_100 1.393 type value precision_at_1000 0.186 type value precision_at_3 19.361 type value precision_at_5 14.106 type value recall_at_1 30.808999999999997 type value recall_at_10 58.47 type value recall_at_100 80.51899999999999 type value recall_at_1000 93.809 type value recall_at_3 42.462 type value recall_at_5 49.385 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackEnglishRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 26.962000000000003 type value map_at_10 36.93 type value map_at_100 38.102000000000004 type value map_at_1000 38.22 type value map_at_3 34.065 type value map_at_5 35.72 type value mrr_at_1 33.567 type value mrr_at_10 42.269 type value mrr_at_100 42.99 type value mrr_at_1000 43.033 type value mrr_at_3 40.064 type value mrr_at_5 41.258 type value ndcg_at_1 33.567 type value ndcg_at_10 42.405 type value ndcg_at_100 46.847 type value ndcg_at_1000 48.951 type value ndcg_at_3 38.312000000000005 type value ndcg_at_5 40.242 type value precision_at_1 33.567 type value precision_at_10 8.032 type value precision_at_100 1.295 type value precision_at_1000 0.17600000000000002 type value precision_at_3 18.662 type value precision_at_5 13.299 type value recall_at_1 26.962000000000003 type value recall_at_10 52.489 type value recall_at_100 71.635 type value recall_at_1000 85.141 type value recall_at_3 40.28 type value recall_at_5 45.757 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGamingRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 36.318 type value map_at_10 47.97 type value map_at_100 49.003 type value map_at_1000 49.065999999999995 type value map_at_3 45.031 type value map_at_5 46.633 type value mrr_at_1 41.504999999999995 type value mrr_at_10 51.431000000000004 type value mrr_at_100 52.129000000000005 type value mrr_at_1000 52.161 type value mrr_at_3 48.934 type value mrr_at_5 50.42 type value ndcg_at_1 41.504999999999995 type value ndcg_at_10 53.676 type value ndcg_at_100 57.867000000000004 type value ndcg_at_1000 59.166 type value ndcg_at_3 48.516 type value ndcg_at_5 50.983999999999995 type value precision_at_1 41.504999999999995 type value precision_at_10 8.608 type value precision_at_100 1.1560000000000001 type value precision_at_1000 0.133 type value precision_at_3 21.462999999999997 type value precision_at_5 14.721 type value recall_at_1 36.318 type value recall_at_10 67.066 type value recall_at_100 85.34 type value recall_at_1000 94.491 type value recall_at_3 53.215999999999994 type value recall_at_5 59.214 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGisRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 22.167 type value map_at_10 29.543999999999997 type value map_at_100 30.579 type value map_at_1000 30.669999999999998 type value map_at_3 26.982 type value map_at_5 28.474 type value mrr_at_1 24.068 type value mrr_at_10 31.237 type value mrr_at_100 32.222 type value mrr_at_1000 32.292 type value mrr_at_3 28.776000000000003 type value mrr_at_5 30.233999999999998 type value ndcg_at_1 24.068 type value ndcg_at_10 33.973 type value ndcg_at_100 39.135 type value ndcg_at_1000 41.443999999999996 type value ndcg_at_3 29.018 type value ndcg_at_5 31.558999999999997 type value precision_at_1 24.068 type value precision_at_10 5.299 type value precision_at_100 0.823 type value precision_at_1000 0.106 type value precision_at_3 12.166 type value precision_at_5 8.767999999999999 type value recall_at_1 22.167 type value recall_at_10 46.115 type value recall_at_100 69.867 type value recall_at_1000 87.234 type value recall_at_3 32.798 type value recall_at_5 38.951 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackMathematicaRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 12.033000000000001 type value map_at_10 19.314 type value map_at_100 20.562 type value map_at_1000 20.695 type value map_at_3 16.946 type value map_at_5 18.076999999999998 type value mrr_at_1 14.801 type value mrr_at_10 22.74 type value mrr_at_100 23.876 type value mrr_at_1000 23.949 type value mrr_at_3 20.211000000000002 type value mrr_at_5 21.573 type value ndcg_at_1 14.801 type value ndcg_at_10 24.038 type value ndcg_at_100 30.186 type value ndcg_at_1000 33.321 type value ndcg_at_3 19.431 type value ndcg_at_5 21.34 type value precision_at_1 14.801 type value precision_at_10 4.776 type value precision_at_100 0.897 type value precision_at_1000 0.133 type value precision_at_3 9.66 type value precision_at_5 7.239 type value recall_at_1 12.033000000000001 type value recall_at_10 35.098 type value recall_at_100 62.175000000000004 type value recall_at_1000 84.17099999999999 type value recall_at_3 22.61 type value recall_at_5 27.278999999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackPhysicsRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 26.651000000000003 type value map_at_10 36.901 type value map_at_100 38.249 type value map_at_1000 38.361000000000004 type value map_at_3 33.891 type value map_at_5 35.439 type value mrr_at_1 32.724 type value mrr_at_10 42.504 type value mrr_at_100 43.391999999999996 type value mrr_at_1000 43.436 type value mrr_at_3 39.989999999999995 type value mrr_at_5 41.347 type value ndcg_at_1 32.724 type value ndcg_at_10 43.007 type value ndcg_at_100 48.601 type value ndcg_at_1000 50.697 type value ndcg_at_3 37.99 type value ndcg_at_5 40.083999999999996 type value precision_at_1 32.724 type value precision_at_10 7.872999999999999 type value precision_at_100 1.247 type value precision_at_1000 0.16199999999999998 type value precision_at_3 18.062 type value precision_at_5 12.666 type value recall_at_1 26.651000000000003 type value recall_at_10 55.674 type value recall_at_100 78.904 type value recall_at_1000 92.55799999999999 type value recall_at_3 41.36 type value recall_at_5 46.983999999999995 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackProgrammersRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 22.589000000000002 type value map_at_10 32.244 type value map_at_100 33.46 type value map_at_1000 33.593 type value map_at_3 29.21 type value map_at_5 31.019999999999996 type value mrr_at_1 28.425 type value mrr_at_10 37.282 type value mrr_at_100 38.187 type value mrr_at_1000 38.248 type value mrr_at_3 34.684 type value mrr_at_5 36.123 type value ndcg_at_1 28.425 type value ndcg_at_10 37.942 type value ndcg_at_100 43.443 type value ndcg_at_1000 45.995999999999995 type value ndcg_at_3 32.873999999999995 type value ndcg_at_5 35.325 type value precision_at_1 28.425 type value precision_at_10 7.1 type value precision_at_100 1.166 type value precision_at_1000 0.158 type value precision_at_3 16.02 type value precision_at_5 11.644 type value recall_at_1 22.589000000000002 type value recall_at_10 50.03999999999999 type value recall_at_100 73.973 type value recall_at_1000 91.128 type value recall_at_3 35.882999999999996 type value recall_at_5 42.187999999999995 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 23.190833333333334 type value map_at_10 31.504916666666666 type value map_at_100 32.64908333333334 type value map_at_1000 32.77075 type value map_at_3 28.82575 type value map_at_5 30.2755 type value mrr_at_1 27.427499999999995 type value mrr_at_10 35.36483333333334 type value mrr_at_100 36.23441666666666 type value mrr_at_1000 36.297583333333336 type value mrr_at_3 32.97966666666667 type value mrr_at_5 34.294583333333335 type value ndcg_at_1 27.427499999999995 type value ndcg_at_10 36.53358333333333 type value ndcg_at_100 41.64508333333333 type value ndcg_at_1000 44.14499999999999 type value ndcg_at_3 31.88908333333333 type value ndcg_at_5 33.98433333333333 type value precision_at_1 27.427499999999995 type value precision_at_10 6.481083333333333 type value precision_at_100 1.0610833333333334 type value precision_at_1000 0.14691666666666667 type value precision_at_3 14.656749999999999 type value precision_at_5 10.493583333333332 type value recall_at_1 23.190833333333334 type value recall_at_10 47.65175 type value recall_at_100 70.41016666666667 type value recall_at_1000 87.82708333333332 type value recall_at_3 34.637583333333325 type value recall_at_5 40.05008333333333 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackStatsRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 20.409 type value map_at_10 26.794 type value map_at_100 27.682000000000002 type value map_at_1000 27.783 type value map_at_3 24.461 type value map_at_5 25.668000000000003 type value mrr_at_1 22.853 type value mrr_at_10 29.296 type value mrr_at_100 30.103 type value mrr_at_1000 30.179000000000002 type value mrr_at_3 27.173000000000002 type value mrr_at_5 28.223 type value ndcg_at_1 22.853 type value ndcg_at_10 31.007 type value ndcg_at_100 35.581 type value ndcg_at_1000 38.147 type value ndcg_at_3 26.590999999999998 type value ndcg_at_5 28.43 type value precision_at_1 22.853 type value precision_at_10 5.031 type value precision_at_100 0.7939999999999999 type value precision_at_1000 0.11 type value precision_at_3 11.401 type value precision_at_5 8.16 type value recall_at_1 20.409 type value recall_at_10 41.766 type value recall_at_100 62.964 type value recall_at_1000 81.682 type value recall_at_3 29.281000000000002 type value recall_at_5 33.83 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackTexRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 14.549000000000001 type value map_at_10 20.315 type value map_at_100 21.301000000000002 type value map_at_1000 21.425 type value map_at_3 18.132 type value map_at_5 19.429 type value mrr_at_1 17.86 type value mrr_at_10 23.860999999999997 type value mrr_at_100 24.737000000000002 type value mrr_at_1000 24.82 type value mrr_at_3 21.685 type value mrr_at_5 23.008 type value ndcg_at_1 17.86 type value ndcg_at_10 24.396 type value ndcg_at_100 29.328 type value ndcg_at_1000 32.486 type value ndcg_at_3 20.375 type value ndcg_at_5 22.411 type value precision_at_1 17.86 type value precision_at_10 4.47 type value precision_at_100 0.8099999999999999 type value precision_at_1000 0.125 type value precision_at_3 9.475 type value precision_at_5 7.170999999999999 type value recall_at_1 14.549000000000001 type value recall_at_10 33.365 type value recall_at_100 55.797 type value recall_at_1000 78.632 type value recall_at_3 22.229 type value recall_at_5 27.339000000000002 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackUnixRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 23.286 type value map_at_10 30.728 type value map_at_100 31.840000000000003 type value map_at_1000 31.953 type value map_at_3 28.302 type value map_at_5 29.615000000000002 type value mrr_at_1 27.239 type value mrr_at_10 34.408 type value mrr_at_100 35.335 type value mrr_at_1000 35.405 type value mrr_at_3 32.151999999999994 type value mrr_at_5 33.355000000000004 type value ndcg_at_1 27.239 type value ndcg_at_10 35.324 type value ndcg_at_100 40.866 type value ndcg_at_1000 43.584 type value ndcg_at_3 30.898999999999997 type value ndcg_at_5 32.812999999999995 type value precision_at_1 27.239 type value precision_at_10 5.896 type value precision_at_100 0.979 type value precision_at_1000 0.133 type value precision_at_3 13.713000000000001 type value precision_at_5 9.683 type value recall_at_1 23.286 type value recall_at_10 45.711 type value recall_at_100 70.611 type value recall_at_1000 90.029 type value recall_at_3 33.615 type value recall_at_5 38.41 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWebmastersRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 23.962 type value map_at_10 31.942999999999998 type value map_at_100 33.384 type value map_at_1000 33.611000000000004 type value map_at_3 29.243000000000002 type value map_at_5 30.446 type value mrr_at_1 28.458 type value mrr_at_10 36.157000000000004 type value mrr_at_100 37.092999999999996 type value mrr_at_1000 37.163000000000004 type value mrr_at_3 33.86 type value mrr_at_5 35.086 type value ndcg_at_1 28.458 type value ndcg_at_10 37.201 type value ndcg_at_100 42.591 type value ndcg_at_1000 45.539 type value ndcg_at_3 32.889 type value ndcg_at_5 34.483000000000004 type value precision_at_1 28.458 type value precision_at_10 7.332 type value precision_at_100 1.437 type value precision_at_1000 0.233 type value precision_at_3 15.547 type value precision_at_5 11.146 type value recall_at_1 23.962 type value recall_at_10 46.751 type value recall_at_100 71.626 type value recall_at_1000 90.93900000000001 type value recall_at_3 34.138000000000005 type value recall_at_5 38.673 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWordpressRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 18.555 type value map_at_10 24.759 type value map_at_100 25.732 type value map_at_1000 25.846999999999998 type value map_at_3 22.646 type value map_at_5 23.791999999999998 type value mrr_at_1 20.148 type value mrr_at_10 26.695999999999998 type value mrr_at_100 27.605 type value mrr_at_1000 27.695999999999998 type value mrr_at_3 24.522 type value mrr_at_5 25.715 type value ndcg_at_1 20.148 type value ndcg_at_10 28.746 type value ndcg_at_100 33.57 type value ndcg_at_1000 36.584 type value ndcg_at_3 24.532 type value ndcg_at_5 26.484 type value precision_at_1 20.148 type value precision_at_10 4.529 type value precision_at_100 0.736 type value precision_at_1000 0.108 type value precision_at_3 10.351 type value precision_at_5 7.32 type value recall_at_1 18.555 type value recall_at_10 39.275999999999996 type value recall_at_100 61.511 type value recall_at_1000 84.111 type value recall_at_3 27.778999999999996 type value recall_at_5 32.591 task dataset metrics type Retrieval type name config split revision climate-fever MTEB ClimateFEVER default test 392b78eb68c07badcd7c2cd8f39af108375dfcce type value map_at_1 10.366999999999999 type value map_at_10 18.953999999999997 type value map_at_100 20.674999999999997 type value map_at_1000 20.868000000000002 type value map_at_3 15.486 type value map_at_5 17.347 type value mrr_at_1 23.257 type value mrr_at_10 35.419 type value mrr_at_100 36.361 type value mrr_at_1000 36.403 type value mrr_at_3 31.747999999999998 type value mrr_at_5 34.077 type value ndcg_at_1 23.257 type value ndcg_at_10 27.11 type value ndcg_at_100 33.981 type value ndcg_at_1000 37.444 type value ndcg_at_3 21.471999999999998 type value ndcg_at_5 23.769000000000002 type value precision_at_1 23.257 type value precision_at_10 8.704 type value precision_at_100 1.606 type value precision_at_1000 0.22499999999999998 type value precision_at_3 16.287 type value precision_at_5 13.068 type value recall_at_1 10.366999999999999 type value recall_at_10 33.706 type value recall_at_100 57.375 type value recall_at_1000 76.79 type value recall_at_3 20.18 type value recall_at_5 26.215 task dataset metrics type Retrieval type name config split revision dbpedia-entity MTEB DBPedia default test f097057d03ed98220bc7309ddb10b71a54d667d6 type value map_at_1 8.246 type value map_at_10 15.979 type value map_at_100 21.025 type value map_at_1000 22.189999999999998 type value map_at_3 11.997 type value map_at_5 13.697000000000001 type value mrr_at_1 60.75000000000001 type value mrr_at_10 68.70100000000001 type value mrr_at_100 69.1 type value mrr_at_1000 69.111 type value mrr_at_3 66.583 type value mrr_at_5 67.87100000000001 type value ndcg_at_1 49.75 type value ndcg_at_10 34.702 type value ndcg_at_100 37.607 type value ndcg_at_1000 44.322 type value ndcg_at_3 39.555 type value ndcg_at_5 36.684 type value precision_at_1 60.75000000000001 type value precision_at_10 26.625 type value precision_at_100 7.969999999999999 type value precision_at_1000 1.678 type value precision_at_3 41.833 type value precision_at_5 34.5 type value recall_at_1 8.246 type value recall_at_10 20.968 type value recall_at_100 42.065000000000005 type value recall_at_1000 63.671 type value recall_at_3 13.039000000000001 type value recall_at_5 16.042 task dataset metrics type Classification type name config split revision mteb/emotion MTEB EmotionClassification default test 829147f8f75a25f005913200eb5ed41fae320aa1 type value accuracy 49.214999999999996 type value f1 44.85952451163755 task dataset metrics type Retrieval type name config split revision fever MTEB FEVER default test 1429cf27e393599b8b359b9b72c666f96b2525f9 type value map_at_1 56.769000000000005 type value map_at_10 67.30199999999999 type value map_at_100 67.692 type value map_at_1000 67.712 type value map_at_3 65.346 type value map_at_5 66.574 type value mrr_at_1 61.370999999999995 type value mrr_at_10 71.875 type value mrr_at_100 72.195 type value mrr_at_1000 72.206 type value mrr_at_3 70.04 type value mrr_at_5 71.224 type value ndcg_at_1 61.370999999999995 type value ndcg_at_10 72.731 type value ndcg_at_100 74.468 type value ndcg_at_1000 74.91600000000001 type value ndcg_at_3 69.077 type value ndcg_at_5 71.111 type value precision_at_1 61.370999999999995 type value precision_at_10 9.325999999999999 type value precision_at_100 1.03 type value precision_at_1000 0.108 type value precision_at_3 27.303 type value precision_at_5 17.525 type value recall_at_1 56.769000000000005 type value recall_at_10 85.06 type value recall_at_100 92.767 type value recall_at_1000 95.933 type value recall_at_3 75.131 type value recall_at_5 80.17 task dataset metrics type Retrieval type name config split revision fiqa MTEB FiQA2018 default test 41b686a7f28c59bcaaa5791efd47c67c8ebe28be type value map_at_1 15.753 type value map_at_10 25.875999999999998 type value map_at_100 27.415 type value map_at_1000 27.590999999999998 type value map_at_3 22.17 type value map_at_5 24.236 type value mrr_at_1 31.019000000000002 type value mrr_at_10 39.977000000000004 type value mrr_at_100 40.788999999999994 type value mrr_at_1000 40.832 type value mrr_at_3 37.088 type value mrr_at_5 38.655 type value ndcg_at_1 31.019000000000002 type value ndcg_at_10 33.286 type value ndcg_at_100 39.528999999999996 type value ndcg_at_1000 42.934 type value ndcg_at_3 29.29 type value ndcg_at_5 30.615 type value precision_at_1 31.019000000000002 type value precision_at_10 9.383 type value precision_at_100 1.6019999999999999 type value precision_at_1000 0.22200000000000003 type value precision_at_3 19.753 type value precision_at_5 14.815000000000001 type value recall_at_1 15.753 type value recall_at_10 40.896 type value recall_at_100 64.443 type value recall_at_1000 85.218 type value recall_at_3 26.526 type value recall_at_5 32.452999999999996 task dataset metrics type Retrieval type name config split revision hotpotqa MTEB HotpotQA default test 766870b35a1b9ca65e67a0d1913899973551fc6c type value map_at_1 32.153999999999996 type value map_at_10 43.651 type value map_at_100 44.41 type value map_at_1000 44.487 type value map_at_3 41.239 type value map_at_5 42.659000000000006 type value mrr_at_1 64.30799999999999 type value mrr_at_10 71.22500000000001 type value mrr_at_100 71.57 type value mrr_at_1000 71.59100000000001 type value mrr_at_3 69.95 type value mrr_at_5 70.738 type value ndcg_at_1 64.30799999999999 type value ndcg_at_10 52.835 type value ndcg_at_100 55.840999999999994 type value ndcg_at_1000 57.484 type value ndcg_at_3 49.014 type value ndcg_at_5 51.01599999999999 type value precision_at_1 64.30799999999999 type value precision_at_10 10.77 type value precision_at_100 1.315 type value precision_at_1000 0.153 type value precision_at_3 30.223 type value precision_at_5 19.716 type value recall_at_1 32.153999999999996 type value recall_at_10 53.849000000000004 type value recall_at_100 65.75999999999999 type value recall_at_1000 76.705 type value recall_at_3 45.334 type value recall_at_5 49.291000000000004 task dataset metrics type Classification type name config split revision mteb/imdb MTEB ImdbClassification default test 8d743909f834c38949e8323a8a6ce8721ea6c7f4 type value accuracy 63.5316 type value ap 58.90084300359825 type value f1 63.35727889030892 task dataset metrics type Retrieval type name config split revision msmarco MTEB MSMARCO default validation e6838a846e2408f22cf5cc337ebc83e0bcf77849 type value map_at_1 20.566000000000003 type value map_at_10 32.229 type value map_at_100 33.445 type value map_at_1000 33.501 type value map_at_3 28.504 type value map_at_5 30.681000000000004 type value mrr_at_1 21.218 type value mrr_at_10 32.816 type value mrr_at_100 33.986 type value mrr_at_1000 34.035 type value mrr_at_3 29.15 type value mrr_at_5 31.290000000000003 type value ndcg_at_1 21.218 type value ndcg_at_10 38.832 type value ndcg_at_100 44.743 type value ndcg_at_1000 46.138 type value ndcg_at_3 31.232 type value ndcg_at_5 35.099999999999994 type value precision_at_1 21.218 type value precision_at_10 6.186 type value precision_at_100 0.914 type value precision_at_1000 0.10300000000000001 type value precision_at_3 13.314 type value precision_at_5 9.943 type value recall_at_1 20.566000000000003 type value recall_at_10 59.192 type value recall_at_100 86.626 type value recall_at_1000 97.283 type value recall_at_3 38.492 type value recall_at_5 47.760000000000005 task dataset metrics type Classification type name config split revision mteb/mtop_domain MTEB MTOPDomainClassification (en) en test a7e2a951126a26fc8c6a69f835f33a346ba259e3 type value accuracy 92.56269949840402 type value f1 92.1020975473988 task dataset metrics type Classification type name config split revision mteb/mtop_intent MTEB MTOPIntentClassification (en) en test 6299947a7777084cc2d4b64235bf7190381ce755 type value accuracy 71.8467852257182 type value f1 53.652719348592015 task dataset metrics type Classification type name config split revision mteb/amazon_massive_intent MTEB MassiveIntentClassification (en) en test 072a486a144adf7f4479a4a0dddb2152e161e1ea type value accuracy 69.00806993947546 type value f1 67.41429618885515 task dataset metrics type Classification type name config split revision mteb/amazon_massive_scenario MTEB MassiveScenarioClassification (en) en test 7d571f92784cd94a019292a1f45445077d0ef634 type value accuracy 75.90114324142569 type value f1 76.25183590651454 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-p2p MTEB MedrxivClusteringP2P default test dcefc037ef84348e49b0d29109e891c01067226b type value v_measure 31.350109978273395 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-s2s MTEB MedrxivClusteringS2S default test 3cd0e71dfbe09d4de0f9e5ecba43e7ce280959dc type value v_measure 28.768923695767327 task dataset metrics type Reranking type name config split revision mteb/mind_small MTEB MindSmallReranking default test 3bdac13927fdc888b903db93b2ffdbd90b295a69 type value map 31.716396735210754 type value mrr 32.88970538547634 task dataset metrics type Retrieval type name config split revision nfcorpus MTEB NFCorpus default test 7eb63cc0c1eb59324d709ebed25fcab851fa7610 type value map_at_1 5.604 type value map_at_10 12.379999999999999 type value map_at_100 15.791 type value map_at_1000 17.327 type value map_at_3 9.15 type value map_at_5 10.599 type value mrr_at_1 45.201 type value mrr_at_10 53.374 type value mrr_at_100 54.089 type value mrr_at_1000 54.123 type value mrr_at_3 51.44499999999999 type value mrr_at_5 52.59 type value ndcg_at_1 42.879 type value ndcg_at_10 33.891 type value ndcg_at_100 31.391999999999996 type value ndcg_at_1000 40.36 type value ndcg_at_3 39.076 type value ndcg_at_5 37.047000000000004 type value precision_at_1 44.582 type value precision_at_10 25.294 type value precision_at_100 8.285 type value precision_at_1000 2.1479999999999997 type value precision_at_3 36.120000000000005 type value precision_at_5 31.95 type value recall_at_1 5.604 type value recall_at_10 16.239 type value recall_at_100 32.16 type value recall_at_1000 64.513 type value recall_at_3 10.406 type value recall_at_5 12.684999999999999 task dataset metrics type Retrieval type name config split revision nq MTEB NQ default test 6062aefc120bfe8ece5897809fb2e53bfe0d128c type value map_at_1 25.881 type value map_at_10 39.501 type value map_at_100 40.615 type value map_at_1000 40.661 type value map_at_3 35.559000000000005 type value map_at_5 37.773 type value mrr_at_1 29.229 type value mrr_at_10 41.955999999999996 type value mrr_at_100 42.86 type value mrr_at_1000 42.893 type value mrr_at_3 38.562000000000005 type value mrr_at_5 40.542 type value ndcg_at_1 29.2 type value ndcg_at_10 46.703 type value ndcg_at_100 51.644 type value ndcg_at_1000 52.771 type value ndcg_at_3 39.141999999999996 type value ndcg_at_5 42.892 type value precision_at_1 29.2 type value precision_at_10 7.920000000000001 type value precision_at_100 1.0659999999999998 type value precision_at_1000 0.117 type value precision_at_3 18.105 type value precision_at_5 13.036 type value recall_at_1 25.881 type value recall_at_10 66.266 type value recall_at_100 88.116 type value recall_at_1000 96.58200000000001 type value recall_at_3 46.526 type value recall_at_5 55.154 task dataset metrics type Retrieval type name config split revision quora MTEB QuoraRetrieval default test 6205996560df11e3a3da9ab4f926788fc30a7db4 type value map_at_1 67.553 type value map_at_10 81.34 type value map_at_100 82.002 type value map_at_1000 82.027 type value map_at_3 78.281 type value map_at_5 80.149 type value mrr_at_1 77.72 type value mrr_at_10 84.733 type value mrr_at_100 84.878 type value mrr_at_1000 84.879 type value mrr_at_3 83.587 type value mrr_at_5 84.32600000000001 type value ndcg_at_1 77.75 type value ndcg_at_10 85.603 type value ndcg_at_100 87.069 type value ndcg_at_1000 87.25 type value ndcg_at_3 82.303 type value ndcg_at_5 84.03699999999999 type value precision_at_1 77.75 type value precision_at_10 13.04 type value precision_at_100 1.5070000000000001 type value precision_at_1000 0.156 type value precision_at_3 35.903 type value precision_at_5 23.738 type value recall_at_1 67.553 type value recall_at_10 93.903 type value recall_at_100 99.062 type value recall_at_1000 99.935 type value recall_at_3 84.58099999999999 type value recall_at_5 89.316 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering MTEB RedditClustering default test b2805658ae38990172679479369a78b86de8c390 type value v_measure 46.46887711230235 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering-p2p MTEB RedditClusteringP2P default test 385e3cb46b4cfa89021f56c4380204149d0efe33 type value v_measure 54.166876298246926 task dataset metrics type Retrieval type name config split revision scidocs MTEB SCIDOCS default test 5c59ef3e437a0a9651c8fe6fde943e7dce59fba5 type value map_at_1 4.053 type value map_at_10 9.693999999999999 type value map_at_100 11.387 type value map_at_1000 11.654 type value map_at_3 7.053 type value map_at_5 8.439 type value mrr_at_1 19.900000000000002 type value mrr_at_10 29.359 type value mrr_at_100 30.484 type value mrr_at_1000 30.553 type value mrr_at_3 26.200000000000003 type value mrr_at_5 28.115000000000002 type value ndcg_at_1 19.900000000000002 type value ndcg_at_10 16.575 type value ndcg_at_100 23.655 type value ndcg_at_1000 28.853 type value ndcg_at_3 15.848 type value ndcg_at_5 14.026 type value precision_at_1 19.900000000000002 type value precision_at_10 8.450000000000001 type value precision_at_100 1.872 type value precision_at_1000 0.313 type value precision_at_3 14.667 type value precision_at_5 12.32 type value recall_at_1 4.053 type value recall_at_10 17.169999999999998 type value recall_at_100 38.025 type value recall_at_1000 63.571999999999996 type value recall_at_3 8.903 type value recall_at_5 12.477 task dataset metrics type STS type name config split revision mteb/sickr-sts MTEB SICK-R default test 20a6d6f312dd54037fe07a32d58e5e168867909d type value cos_sim_pearson 77.7548748519677 type value cos_sim_spearman 68.19926431966059 type value euclidean_pearson 71.69016204991725 type value euclidean_spearman 66.98099673026834 type value manhattan_pearson 71.62994072488664 type value manhattan_spearman 67.03435950744577 task dataset metrics type STS type name config split revision mteb/sts12-sts MTEB STS12 default test fdf84275bb8ce4b49c971d02e84dd1abc677a50f type value cos_sim_pearson 75.91051402657887 type value cos_sim_spearman 66.99390786191645 type value euclidean_pearson 71.54128036454578 type value euclidean_spearman 69.25605675649068 type value manhattan_pearson 71.60981030780171 type value manhattan_spearman 69.27513670128046 task dataset metrics type STS type name config split revision mteb/sts13-sts MTEB STS13 default test 1591bfcbe8c69d4bf7fe2a16e2451017832cafb9 type value cos_sim_pearson 77.23835466417793 type value cos_sim_spearman 77.57623085766706 type value euclidean_pearson 77.5090992200725 type value euclidean_spearman 77.88601688144924 type value manhattan_pearson 77.39045060647423 type value manhattan_spearman 77.77552718279098 task dataset metrics type STS type name config split revision mteb/sts14-sts MTEB STS14 default test e2125984e7df8b7871f6ae9949cf6b6795e7c54b type value cos_sim_pearson 77.91692485139602 type value cos_sim_spearman 72.78258293483495 type value euclidean_pearson 74.64773017077789 type value euclidean_spearman 71.81662299104619 type value manhattan_pearson 74.71043337995533 type value manhattan_spearman 71.83960860845646 task dataset metrics type STS type name config split revision mteb/sts15-sts MTEB STS15 default test 1cd7298cac12a96a373b6a2f18738bb3e739a9b6 type value cos_sim_pearson 82.13422113617578 type value cos_sim_spearman 82.61707296911949 type value euclidean_pearson 81.42487480400861 type value euclidean_spearman 82.17970991273835 type value manhattan_pearson 81.41985055477845 type value manhattan_spearman 82.15823204362937 task dataset metrics type STS type name config split revision mteb/sts16-sts MTEB STS16 default test 360a0b2dff98700d09e634a01e1cc1624d3e42cd type value cos_sim_pearson 79.07989542843826 type value cos_sim_spearman 80.09839524406284 type value euclidean_pearson 76.43186028364195 type value euclidean_spearman 76.76720323266471 type value manhattan_pearson 76.4674747409161 type value manhattan_spearman 76.81797407068667 task dataset metrics type STS type name config split revision mteb/sts17-crosslingual-sts MTEB STS17 (en-en) en-en test 9fc37e8c632af1c87a3d23e685d49552a02582a0 type value cos_sim_pearson 87.0420983224933 type value cos_sim_spearman 87.25017540413702 type value euclidean_pearson 84.56384596473421 type value euclidean_spearman 84.72557417564886 type value manhattan_pearson 84.7329954474549 type value manhattan_spearman 84.75071371008909 task dataset metrics type STS type name config split revision mteb/sts22-crosslingual-sts MTEB STS22 (en) en test 2de6ce8c1921b71a755b262c6b57fef195dd7906 type value cos_sim_pearson 68.47031320016424 type value cos_sim_spearman 68.7486910762485 type value euclidean_pearson 71.30330985913915 type value euclidean_spearman 71.59666258520735 type value manhattan_pearson 71.4423884279027 type value manhattan_spearman 71.67460706861044 task dataset metrics type STS type name config split revision mteb/stsbenchmark-sts MTEB STSBenchmark default test 8913289635987208e6e7c72789e4be2fe94b6abd type value cos_sim_pearson 80.79514366062675 type value cos_sim_spearman 79.20585637461048 type value euclidean_pearson 78.6591557395699 type value euclidean_spearman 77.86455794285718 type value manhattan_pearson 78.67754806486865 type value manhattan_spearman 77.88178687200732 task dataset metrics type Reranking type name config split revision mteb/scidocs-reranking MTEB SciDocsRR default test 56a6d0140cf6356659e2a7c1413286a774468d44 type value map 77.71580844366375 type value mrr 93.04215845882513 task dataset metrics type Retrieval type name config split revision scifact MTEB SciFact default test a75ae049398addde9b70f6b268875f5cbce99089 type value map_at_1 56.39999999999999 type value map_at_10 65.701 type value map_at_100 66.32000000000001 type value map_at_1000 66.34100000000001 type value map_at_3 62.641999999999996 type value map_at_5 64.342 type value mrr_at_1 58.667 type value mrr_at_10 66.45299999999999 type value mrr_at_100 66.967 type value mrr_at_1000 66.988 type value mrr_at_3 64.11099999999999 type value mrr_at_5 65.411 type value ndcg_at_1 58.667 type value ndcg_at_10 70.165 type value ndcg_at_100 72.938 type value ndcg_at_1000 73.456 type value ndcg_at_3 64.79 type value ndcg_at_5 67.28 type value precision_at_1 58.667 type value precision_at_10 9.4 type value precision_at_100 1.087 type value precision_at_1000 0.11299999999999999 type value precision_at_3 24.889 type value precision_at_5 16.667 type value recall_at_1 56.39999999999999 type value recall_at_10 83.122 type value recall_at_100 95.667 type value recall_at_1000 99.667 type value recall_at_3 68.378 type value recall_at_5 74.68299999999999 task dataset metrics type PairClassification type name config split revision mteb/sprintduplicatequestions-pairclassification MTEB SprintDuplicateQuestions default test 5a8256d0dff9c4bd3be3ba3e67e4e70173f802ea type value cos_sim_accuracy 99.76831683168317 type value cos_sim_ap 93.47124923047998 type value cos_sim_f1 88.06122448979592 type value cos_sim_precision 89.89583333333333 type value cos_sim_recall 86.3 type value dot_accuracy 99.57326732673268 type value dot_ap 84.06577868167207 type value dot_f1 77.82629791363416 type value dot_precision 75.58906691800189 type value dot_recall 80.2 type value euclidean_accuracy 99.74257425742574 type value euclidean_ap 92.1904681653555 type value euclidean_f1 86.74821610601427 type value euclidean_precision 88.46153846153845 type value euclidean_recall 85.1 type value manhattan_accuracy 99.74554455445545 type value manhattan_ap 92.4337790809948 type value manhattan_f1 86.86765457332653 type value manhattan_precision 88.81922675026124 type value manhattan_recall 85.0 type value max_accuracy 99.76831683168317 type value max_ap 93.47124923047998 type value max_f1 88.06122448979592 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering MTEB StackExchangeClustering default test 70a89468f6dccacc6aa2b12a6eac54e74328f235 type value v_measure 59.194098673976484 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering-p2p MTEB StackExchangeClusteringP2P default test d88009ab563dd0b16cfaf4436abaf97fa3550cf0 type value v_measure 32.5744032578115 task dataset metrics type Reranking type name config split revision mteb/stackoverflowdupquestions-reranking MTEB StackOverflowDupQuestions default test ef807ea29a75ec4f91b50fd4191cb4ee4589a9f9 type value map 49.61186384154483 type value mrr 50.55424253034547 task dataset metrics type Summarization type name config split revision mteb/summeval MTEB SummEval default test 8753c2788d36c01fc6f05d03fe3f7268d63f9122 type value cos_sim_pearson 30.027210161713946 type value cos_sim_spearman 31.030178065751735 type value dot_pearson 30.09179785685587 type value dot_spearman 30.408303252207813 task dataset metrics type Retrieval type name config split revision trec-covid MTEB TRECCOVID default test 2c8041b2c07a79b6f7ba8fe6acc72e5d9f92d217 type value map_at_1 0.22300000000000003 type value map_at_10 1.762 type value map_at_100 9.984 type value map_at_1000 24.265 type value map_at_3 0.631 type value map_at_5 0.9950000000000001 type value mrr_at_1 88.0 type value mrr_at_10 92.833 type value mrr_at_100 92.833 type value mrr_at_1000 92.833 type value mrr_at_3 92.333 type value mrr_at_5 92.833 type value ndcg_at_1 83.0 type value ndcg_at_10 75.17 type value ndcg_at_100 55.432 type value ndcg_at_1000 49.482 type value ndcg_at_3 82.184 type value ndcg_at_5 79.712 type value precision_at_1 88.0 type value precision_at_10 78.60000000000001 type value precision_at_100 56.56 type value precision_at_1000 22.334 type value precision_at_3 86.667 type value precision_at_5 83.6 type value recall_at_1 0.22300000000000003 type value recall_at_10 1.9879999999999998 type value recall_at_100 13.300999999999998 type value recall_at_1000 46.587 type value recall_at_3 0.6629999999999999 type value recall_at_5 1.079 task dataset metrics type Retrieval type name config split revision webis-touche2020 MTEB Touche2020 default test 527b7d77e16e343303e68cb6af11d6e18b9f7b3b type value map_at_1 3.047 type value map_at_10 8.792 type value map_at_100 14.631 type value map_at_1000 16.127 type value map_at_3 4.673 type value map_at_5 5.897 type value mrr_at_1 38.775999999999996 type value mrr_at_10 49.271 type value mrr_at_100 50.181 type value mrr_at_1000 50.2 type value mrr_at_3 44.558 type value mrr_at_5 47.925000000000004 type value ndcg_at_1 35.714 type value ndcg_at_10 23.44 type value ndcg_at_100 35.345 type value ndcg_at_1000 46.495 type value ndcg_at_3 26.146 type value ndcg_at_5 24.878 type value precision_at_1 38.775999999999996 type value precision_at_10 20.816000000000003 type value precision_at_100 7.428999999999999 type value precision_at_1000 1.494 type value precision_at_3 25.85 type value precision_at_5 24.082 type value recall_at_1 3.047 type value recall_at_10 14.975 type value recall_at_100 45.943 type value recall_at_1000 80.31099999999999 type value recall_at_3 5.478000000000001 type value recall_at_5 8.294 task dataset metrics type Classification type name config split revision mteb/toxic_conversations_50k MTEB ToxicConversationsClassification default test edfaf9da55d3dd50d43143d90c1ac476895ae6de type value accuracy 68.84080000000002 type value ap 13.135219251019848 type value f1 52.849999421995506 task dataset metrics type Classification type name config split revision mteb/tweet_sentiment_extraction MTEB TweetSentimentExtractionClassification default test 62146448f05be9e52a36b8ee9936447ea787eede type value accuracy 56.68647425014149 type value f1 56.97981427365949 task dataset metrics type Clustering type name config split revision mteb/twentynewsgroups-clustering MTEB TwentyNewsgroupsClustering default test 091a54f9a36281ce7d6590ec8c75dd485e7e01d4 type value v_measure 40.8911707239219 task dataset metrics type PairClassification type name config split revision mteb/twittersemeval2015-pairclassification MTEB TwitterSemEval2015 default test 70970daeab8776df92f5ea462b6173c0b46fd2d1 type value cos_sim_accuracy 83.04226023722954 type value cos_sim_ap 63.681339908301325 type value cos_sim_f1 60.349184470480125 type value cos_sim_precision 53.437754271765655 type value cos_sim_recall 69.31398416886545 type value dot_accuracy 81.46271681468677 type value dot_ap 57.78072296265885 type value dot_f1 56.28769265132901 type value dot_precision 48.7993803253292 type value dot_recall 66.49076517150397 type value euclidean_accuracy 82.16606067830959 type value euclidean_ap 59.974530371203514 type value euclidean_f1 56.856023506366306 type value euclidean_precision 53.037916857012334 type value euclidean_recall 61.2664907651715 type value manhattan_accuracy 82.16606067830959 type value manhattan_ap 59.98962379571767 type value manhattan_f1 56.98153158451947 type value manhattan_precision 51.41158989598811 type value manhattan_recall 63.90501319261214 type value max_accuracy 83.04226023722954 type value max_ap 63.681339908301325 type value max_f1 60.349184470480125 task dataset metrics type PairClassification type name config split revision mteb/twitterurlcorpus-pairclassification MTEB TwitterURLCorpus default test 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf type value cos_sim_accuracy 88.56871191834517 type value cos_sim_ap 84.80240716354544 type value cos_sim_f1 77.07765285922385 type value cos_sim_precision 74.84947406601378 type value cos_sim_recall 79.44256236526024 type value dot_accuracy 86.00923662048356 type value dot_ap 78.6556459012073 type value dot_f1 72.7583749109052 type value dot_precision 67.72823779193206 type value dot_recall 78.59562673236834 type value euclidean_accuracy 87.84103698529127 type value euclidean_ap 83.50424424952834 type value euclidean_f1 75.74496544549307 type value euclidean_precision 73.19402556369381 type value euclidean_recall 78.48013550970127 type value manhattan_accuracy 87.9225365777933 type value manhattan_ap 83.49479248597825 type value manhattan_f1 75.67748162447101 type value manhattan_precision 73.06810035842294 type value manhattan_recall 78.48013550970127 type value max_accuracy 88.56871191834517 type value max_ap 84.80240716354544 type value max_f1 77.07765285922385

SGPT-2.7B-weightedmean-msmarco-specb-bitfit

Usage

For usage instructions, refer to our codebase: https://github.com/Muennighoff/sgpt

Evaluation Results

For eval results, refer to the eval folder or our paper: https://arxiv.org/abs/2202.08904

Training

The model was trained with the parameters:

DataLoader:

torch.utils.data.dataloader.DataLoader of length 124796 with parameters:

{'batch_size': 4, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}

Loss:

sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss with parameters:

{'scale': 20.0, 'similarity_fct': 'cos_sim'}

Parameters of the fit()-Method:

{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 7.5e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 1000,
    "weight_decay": 0.01
}

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 300, 'do_lower_case': False}) with Transformer model: GPTNeoModel 
  (1): Pooling({'word_embedding_dimension': 2560, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': True, 'pooling_mode_lasttoken': False})
)

Citing & Authors

@article{muennighoff2022sgpt,
  title={SGPT: GPT Sentence Embeddings for Semantic Search},
  author={Muennighoff, Niklas},
  journal={arXiv preprint arXiv:2202.08904},
  year={2022}
}