SGPT-1.3B-weightedmean-msmarco-specb-bitfit/README.md

tags, model-index

tags

model-index

sentence-transformers feature-extraction sentence-similarity mteb

name results SGPT-1.3B-weightedmean-msmarco-specb-bitfit task dataset metrics type Classification type name config split revision mteb/amazon_counterfactual MTEB AmazonCounterfactualClassification (en) en test 2d8a100785abf0ae21420d2a55b0c56e3e1ea996 type value accuracy 65.20895522388061 type value ap 29.59212705444778 type value f1 59.97099864321921 task dataset metrics type Classification type name config split revision mteb/amazon_polarity MTEB AmazonPolarityClassification default test 80714f8dcf8cefc218ef4f8c5a966dd83f75a0e1 type value accuracy 73.20565 type value ap 67.36680643550963 type value f1 72.90420520325125 task dataset metrics type Classification type name config split revision mteb/amazon_reviews_multi MTEB AmazonReviewsClassification (en) en test c379a6705fec24a2493fa68e011692605f44e119 type value accuracy 34.955999999999996 type value f1 34.719324437696955 task dataset metrics type Retrieval type name config split revision arguana MTEB ArguAna default test 5b3e3697907184a9b77a3c99ee9ea1a9cbb1e4e3 type value map_at_1 26.101999999999997 type value map_at_10 40.958 type value map_at_100 42.033 type value map_at_1000 42.042 type value map_at_3 36.332 type value map_at_5 38.608 type value mrr_at_1 26.387 type value mrr_at_10 41.051 type value mrr_at_100 42.118 type value mrr_at_1000 42.126999999999995 type value mrr_at_3 36.415 type value mrr_at_5 38.72 type value ndcg_at_1 26.101999999999997 type value ndcg_at_10 49.68 type value ndcg_at_100 54.257999999999996 type value ndcg_at_1000 54.486000000000004 type value ndcg_at_3 39.864 type value ndcg_at_5 43.980000000000004 type value precision_at_1 26.101999999999997 type value precision_at_10 7.781000000000001 type value precision_at_100 0.979 type value precision_at_1000 0.1 type value precision_at_3 16.714000000000002 type value precision_at_5 12.034 type value recall_at_1 26.101999999999997 type value recall_at_10 77.809 type value recall_at_100 97.866 type value recall_at_1000 99.644 type value recall_at_3 50.141999999999996 type value recall_at_5 60.171 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-p2p MTEB ArxivClusteringP2P default test 0bbdb47bcbe3a90093699aefeed338a0f28a7ee8 type value v_measure 43.384194916953774 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-s2s MTEB ArxivClusteringS2S default test b73bd54100e5abfa6e3a23dcafb46fe4d2438dc3 type value v_measure 33.70962633433912 task dataset metrics type Reranking type name config split revision mteb/askubuntudupquestions-reranking MTEB AskUbuntuDupQuestions default test 4d853f94cd57d85ec13805aeeac3ae3e5eb4c49c type value map 58.133058996870076 type value mrr 72.10922041946972 task dataset metrics type STS type name config split revision mteb/biosses-sts MTEB BIOSSES default test 9ee918f184421b6bd48b78f6c714d86546106103 type value cos_sim_pearson 86.62153841660047 type value cos_sim_spearman 83.01514456843276 type value euclidean_pearson 86.00431518427241 type value euclidean_spearman 83.85552516285783 type value manhattan_pearson 85.83025803351181 type value manhattan_spearman 83.86636878343106 task dataset metrics type Classification type name config split revision mteb/banking77 MTEB Banking77Classification default test 44fa15921b4c889113cc5df03dd4901b49161ab7 type value accuracy 82.05844155844156 type value f1 82.0185837884764 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-p2p MTEB BiorxivClusteringP2P default test 11d0121201d1f1f280e8cc8f3d98fb9c4d9f9c55 type value v_measure 35.05918333141837 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-s2s MTEB BiorxivClusteringS2S default test c0fab014e1bcb8d3a5e31b2088972a1e01547dc1 type value v_measure 30.71055028830579 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackAndroidRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 26.519 type value map_at_10 35.634 type value map_at_100 36.961 type value map_at_1000 37.088 type value map_at_3 32.254 type value map_at_5 34.22 type value mrr_at_1 32.332 type value mrr_at_10 41.168 type value mrr_at_100 41.977 type value mrr_at_1000 42.028999999999996 type value mrr_at_3 38.196999999999996 type value mrr_at_5 40.036 type value ndcg_at_1 32.332 type value ndcg_at_10 41.471000000000004 type value ndcg_at_100 46.955999999999996 type value ndcg_at_1000 49.262 type value ndcg_at_3 35.937999999999995 type value ndcg_at_5 38.702999999999996 type value precision_at_1 32.332 type value precision_at_10 7.7829999999999995 type value precision_at_100 1.29 type value precision_at_1000 0.178 type value precision_at_3 16.834 type value precision_at_5 12.418 type value recall_at_1 26.519 type value recall_at_10 53.190000000000005 type value recall_at_100 76.56500000000001 type value recall_at_1000 91.47800000000001 type value recall_at_3 38.034 type value recall_at_5 45.245999999999995 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackEnglishRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 25.356 type value map_at_10 34.596 type value map_at_100 35.714 type value map_at_1000 35.839999999999996 type value map_at_3 32.073 type value map_at_5 33.475 type value mrr_at_1 31.274 type value mrr_at_10 39.592 type value mrr_at_100 40.284 type value mrr_at_1000 40.339999999999996 type value mrr_at_3 37.378 type value mrr_at_5 38.658 type value ndcg_at_1 31.274 type value ndcg_at_10 39.766 type value ndcg_at_100 44.028 type value ndcg_at_1000 46.445 type value ndcg_at_3 35.934 type value ndcg_at_5 37.751000000000005 type value precision_at_1 31.274 type value precision_at_10 7.452 type value precision_at_100 1.217 type value precision_at_1000 0.16999999999999998 type value precision_at_3 17.431 type value precision_at_5 12.306000000000001 type value recall_at_1 25.356 type value recall_at_10 49.344 type value recall_at_100 67.497 type value recall_at_1000 83.372 type value recall_at_3 38.227 type value recall_at_5 43.187999999999995 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGamingRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 32.759 type value map_at_10 43.937 type value map_at_100 45.004 type value map_at_1000 45.07 type value map_at_3 40.805 type value map_at_5 42.497 type value mrr_at_1 37.367 type value mrr_at_10 47.237 type value mrr_at_100 47.973 type value mrr_at_1000 48.010999999999996 type value mrr_at_3 44.65 type value mrr_at_5 46.050999999999995 type value ndcg_at_1 37.367 type value ndcg_at_10 49.659 type value ndcg_at_100 54.069 type value ndcg_at_1000 55.552 type value ndcg_at_3 44.169000000000004 type value ndcg_at_5 46.726 type value precision_at_1 37.367 type value precision_at_10 8.163 type value precision_at_100 1.133 type value precision_at_1000 0.131 type value precision_at_3 19.707 type value precision_at_5 13.718 type value recall_at_1 32.759 type value recall_at_10 63.341 type value recall_at_100 82.502 type value recall_at_1000 93.259 type value recall_at_3 48.796 type value recall_at_5 54.921 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGisRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 18.962 type value map_at_10 25.863000000000003 type value map_at_100 26.817999999999998 type value map_at_1000 26.918 type value map_at_3 23.043 type value map_at_5 24.599 type value mrr_at_1 20.452 type value mrr_at_10 27.301 type value mrr_at_100 28.233000000000004 type value mrr_at_1000 28.310000000000002 type value mrr_at_3 24.539 type value mrr_at_5 26.108999999999998 type value ndcg_at_1 20.452 type value ndcg_at_10 30.354999999999997 type value ndcg_at_100 35.336 type value ndcg_at_1000 37.927 type value ndcg_at_3 24.705 type value ndcg_at_5 27.42 type value precision_at_1 20.452 type value precision_at_10 4.949 type value precision_at_100 0.7799999999999999 type value precision_at_1000 0.104 type value precision_at_3 10.358 type value precision_at_5 7.774 type value recall_at_1 18.962 type value recall_at_10 43.056 type value recall_at_100 66.27300000000001 type value recall_at_1000 85.96000000000001 type value recall_at_3 27.776 type value recall_at_5 34.287 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackMathematicaRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 11.24 type value map_at_10 18.503 type value map_at_100 19.553 type value map_at_1000 19.689999999999998 type value map_at_3 16.150000000000002 type value map_at_5 17.254 type value mrr_at_1 13.806 type value mrr_at_10 21.939 type value mrr_at_100 22.827 type value mrr_at_1000 22.911 type value mrr_at_3 19.32 type value mrr_at_5 20.558 type value ndcg_at_1 13.806 type value ndcg_at_10 23.383000000000003 type value ndcg_at_100 28.834 type value ndcg_at_1000 32.175 type value ndcg_at_3 18.651999999999997 type value ndcg_at_5 20.505000000000003 type value precision_at_1 13.806 type value precision_at_10 4.714 type value precision_at_100 0.864 type value precision_at_1000 0.13 type value precision_at_3 9.328 type value precision_at_5 6.841 type value recall_at_1 11.24 type value recall_at_10 34.854 type value recall_at_100 59.50299999999999 type value recall_at_1000 83.25 type value recall_at_3 22.02 type value recall_at_5 26.715 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackPhysicsRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 23.012 type value map_at_10 33.048 type value map_at_100 34.371 type value map_at_1000 34.489 type value map_at_3 29.942999999999998 type value map_at_5 31.602000000000004 type value mrr_at_1 28.104000000000003 type value mrr_at_10 37.99 type value mrr_at_100 38.836 type value mrr_at_1000 38.891 type value mrr_at_3 35.226 type value mrr_at_5 36.693999999999996 type value ndcg_at_1 28.104000000000003 type value ndcg_at_10 39.037 type value ndcg_at_100 44.643 type value ndcg_at_1000 46.939 type value ndcg_at_3 33.784 type value ndcg_at_5 36.126000000000005 type value precision_at_1 28.104000000000003 type value precision_at_10 7.2669999999999995 type value precision_at_100 1.193 type value precision_at_1000 0.159 type value precision_at_3 16.298000000000002 type value precision_at_5 11.684 type value recall_at_1 23.012 type value recall_at_10 52.054 type value recall_at_100 75.622 type value recall_at_1000 90.675 type value recall_at_3 37.282 type value recall_at_5 43.307 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackProgrammersRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 21.624 type value map_at_10 30.209999999999997 type value map_at_100 31.52 type value map_at_1000 31.625999999999998 type value map_at_3 26.951000000000004 type value map_at_5 28.938999999999997 type value mrr_at_1 26.941 type value mrr_at_10 35.13 type value mrr_at_100 36.15 type value mrr_at_1000 36.204 type value mrr_at_3 32.42 type value mrr_at_5 34.155 type value ndcg_at_1 26.941 type value ndcg_at_10 35.726 type value ndcg_at_100 41.725 type value ndcg_at_1000 44.105 type value ndcg_at_3 30.184 type value ndcg_at_5 33.176 type value precision_at_1 26.941 type value precision_at_10 6.654999999999999 type value precision_at_100 1.1520000000000001 type value precision_at_1000 0.152 type value precision_at_3 14.346 type value precision_at_5 10.868 type value recall_at_1 21.624 type value recall_at_10 47.359 type value recall_at_100 73.436 type value recall_at_1000 89.988 type value recall_at_3 32.34 type value recall_at_5 39.856 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 20.67566666666667 type value map_at_10 28.479333333333333 type value map_at_100 29.612249999999996 type value map_at_1000 29.731166666666663 type value map_at_3 25.884 type value map_at_5 27.298916666666667 type value mrr_at_1 24.402583333333332 type value mrr_at_10 32.07041666666667 type value mrr_at_100 32.95841666666667 type value mrr_at_1000 33.025416666666665 type value mrr_at_3 29.677749999999996 type value mrr_at_5 31.02391666666667 type value ndcg_at_1 24.402583333333332 type value ndcg_at_10 33.326166666666666 type value ndcg_at_100 38.51566666666667 type value ndcg_at_1000 41.13791666666667 type value ndcg_at_3 28.687749999999994 type value ndcg_at_5 30.84766666666667 type value precision_at_1 24.402583333333332 type value precision_at_10 5.943749999999999 type value precision_at_100 1.0098333333333334 type value precision_at_1000 0.14183333333333334 type value precision_at_3 13.211500000000001 type value precision_at_5 9.548416666666668 type value recall_at_1 20.67566666666667 type value recall_at_10 44.245583333333336 type value recall_at_100 67.31116666666667 type value recall_at_1000 85.87841666666665 type value recall_at_3 31.49258333333333 type value recall_at_5 36.93241666666667 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackStatsRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 18.34 type value map_at_10 23.988 type value map_at_100 24.895 type value map_at_1000 24.992 type value map_at_3 21.831 type value map_at_5 23.0 type value mrr_at_1 20.399 type value mrr_at_10 26.186 type value mrr_at_100 27.017999999999997 type value mrr_at_1000 27.090999999999998 type value mrr_at_3 24.08 type value mrr_at_5 25.230000000000004 type value ndcg_at_1 20.399 type value ndcg_at_10 27.799000000000003 type value ndcg_at_100 32.579 type value ndcg_at_1000 35.209 type value ndcg_at_3 23.684 type value ndcg_at_5 25.521 type value precision_at_1 20.399 type value precision_at_10 4.585999999999999 type value precision_at_100 0.755 type value precision_at_1000 0.105 type value precision_at_3 10.276 type value precision_at_5 7.362 type value recall_at_1 18.34 type value recall_at_10 37.456 type value recall_at_100 59.86 type value recall_at_1000 79.703 type value recall_at_3 26.163999999999998 type value recall_at_5 30.652 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackTexRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 12.327 type value map_at_10 17.572 type value map_at_100 18.534 type value map_at_1000 18.653 type value map_at_3 15.703 type value map_at_5 16.752 type value mrr_at_1 15.038000000000002 type value mrr_at_10 20.726 type value mrr_at_100 21.61 type value mrr_at_1000 21.695 type value mrr_at_3 18.829 type value mrr_at_5 19.885 type value ndcg_at_1 15.038000000000002 type value ndcg_at_10 21.241 type value ndcg_at_100 26.179000000000002 type value ndcg_at_1000 29.316 type value ndcg_at_3 17.762 type value ndcg_at_5 19.413 type value precision_at_1 15.038000000000002 type value precision_at_10 3.8920000000000003 type value precision_at_100 0.75 type value precision_at_1000 0.11800000000000001 type value precision_at_3 8.351 type value precision_at_5 6.187 type value recall_at_1 12.327 type value recall_at_10 29.342000000000002 type value recall_at_100 51.854 type value recall_at_1000 74.648 type value recall_at_3 19.596 type value recall_at_5 23.899 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackUnixRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 20.594 type value map_at_10 27.878999999999998 type value map_at_100 28.926000000000002 type value map_at_1000 29.041 type value map_at_3 25.668999999999997 type value map_at_5 26.773999999999997 type value mrr_at_1 23.694000000000003 type value mrr_at_10 31.335 type value mrr_at_100 32.218 type value mrr_at_1000 32.298 type value mrr_at_3 29.26 type value mrr_at_5 30.328 type value ndcg_at_1 23.694000000000003 type value ndcg_at_10 32.456 type value ndcg_at_100 37.667 type value ndcg_at_1000 40.571 type value ndcg_at_3 28.283 type value ndcg_at_5 29.986 type value precision_at_1 23.694000000000003 type value precision_at_10 5.448 type value precision_at_100 0.9119999999999999 type value precision_at_1000 0.127 type value precision_at_3 12.717999999999998 type value precision_at_5 8.843 type value recall_at_1 20.594 type value recall_at_10 43.004999999999995 type value recall_at_100 66.228 type value recall_at_1000 87.17099999999999 type value recall_at_3 31.554 type value recall_at_5 35.838 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWebmastersRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 20.855999999999998 type value map_at_10 28.372000000000003 type value map_at_100 29.87 type value map_at_1000 30.075000000000003 type value map_at_3 26.054 type value map_at_5 27.128999999999998 type value mrr_at_1 25.494 type value mrr_at_10 32.735 type value mrr_at_100 33.794000000000004 type value mrr_at_1000 33.85 type value mrr_at_3 30.731 type value mrr_at_5 31.897 type value ndcg_at_1 25.494 type value ndcg_at_10 33.385 type value ndcg_at_100 39.436 type value ndcg_at_1000 42.313 type value ndcg_at_3 29.612 type value ndcg_at_5 31.186999999999998 type value precision_at_1 25.494 type value precision_at_10 6.422999999999999 type value precision_at_100 1.383 type value precision_at_1000 0.22399999999999998 type value precision_at_3 13.834 type value precision_at_5 10.0 type value recall_at_1 20.855999999999998 type value recall_at_10 42.678 type value recall_at_100 70.224 type value recall_at_1000 89.369 type value recall_at_3 31.957 type value recall_at_5 36.026 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWordpressRetrieval default test 2b9f5791698b5be7bc5e10535c8690f20043c3db type value map_at_1 16.519000000000002 type value map_at_10 22.15 type value map_at_100 23.180999999999997 type value map_at_1000 23.291999999999998 type value map_at_3 20.132 type value map_at_5 21.346 type value mrr_at_1 17.93 type value mrr_at_10 23.506 type value mrr_at_100 24.581 type value mrr_at_1000 24.675 type value mrr_at_3 21.503 type value mrr_at_5 22.686 type value ndcg_at_1 17.93 type value ndcg_at_10 25.636 type value ndcg_at_100 30.736 type value ndcg_at_1000 33.841 type value ndcg_at_3 21.546000000000003 type value ndcg_at_5 23.658 type value precision_at_1 17.93 type value precision_at_10 3.993 type value precision_at_100 0.6890000000000001 type value precision_at_1000 0.104 type value precision_at_3 9.057 type value precision_at_5 6.58 type value recall_at_1 16.519000000000002 type value recall_at_10 35.268 type value recall_at_100 58.17 type value recall_at_1000 81.66799999999999 type value recall_at_3 24.165 type value recall_at_5 29.254 task dataset metrics type Retrieval type name config split revision climate-fever MTEB ClimateFEVER default test 392b78eb68c07badcd7c2cd8f39af108375dfcce type value map_at_1 10.363 type value map_at_10 18.301000000000002 type value map_at_100 20.019000000000002 type value map_at_1000 20.207 type value map_at_3 14.877 type value map_at_5 16.544 type value mrr_at_1 22.866 type value mrr_at_10 34.935 type value mrr_at_100 35.802 type value mrr_at_1000 35.839999999999996 type value mrr_at_3 30.965999999999998 type value mrr_at_5 33.204 type value ndcg_at_1 22.866 type value ndcg_at_10 26.595000000000002 type value ndcg_at_100 33.513999999999996 type value ndcg_at_1000 36.872 type value ndcg_at_3 20.666999999999998 type value ndcg_at_5 22.728 type value precision_at_1 22.866 type value precision_at_10 8.632 type value precision_at_100 1.6119999999999999 type value precision_at_1000 0.22399999999999998 type value precision_at_3 15.504999999999999 type value precision_at_5 12.404 type value recall_at_1 10.363 type value recall_at_10 33.494 type value recall_at_100 57.593 type value recall_at_1000 76.342 type value recall_at_3 19.157 type value recall_at_5 24.637999999999998 task dataset metrics type Retrieval type name config split revision dbpedia-entity MTEB DBPedia default test f097057d03ed98220bc7309ddb10b71a54d667d6 type value map_at_1 7.436 type value map_at_10 14.760000000000002 type value map_at_100 19.206 type value map_at_1000 20.267 type value map_at_3 10.894 type value map_at_5 12.828999999999999 type value mrr_at_1 54.25 type value mrr_at_10 63.769 type value mrr_at_100 64.193 type value mrr_at_1000 64.211 type value mrr_at_3 61.458 type value mrr_at_5 63.096 type value ndcg_at_1 42.875 type value ndcg_at_10 31.507 type value ndcg_at_100 34.559 type value ndcg_at_1000 41.246 type value ndcg_at_3 35.058 type value ndcg_at_5 33.396 type value precision_at_1 54.25 type value precision_at_10 24.45 type value precision_at_100 7.383000000000001 type value precision_at_1000 1.582 type value precision_at_3 38.083 type value precision_at_5 32.6 type value recall_at_1 7.436 type value recall_at_10 19.862 type value recall_at_100 38.981 type value recall_at_1000 61.038000000000004 type value recall_at_3 11.949 type value recall_at_5 15.562000000000001 task dataset metrics type Classification type name config split revision mteb/emotion MTEB EmotionClassification default test 829147f8f75a25f005913200eb5ed41fae320aa1 type value accuracy 46.39 type value f1 42.26424885856703 task dataset metrics type Retrieval type name config split revision fever MTEB FEVER default test 1429cf27e393599b8b359b9b72c666f96b2525f9 type value map_at_1 50.916 type value map_at_10 62.258 type value map_at_100 62.741 type value map_at_1000 62.763000000000005 type value map_at_3 60.01800000000001 type value map_at_5 61.419999999999995 type value mrr_at_1 54.964999999999996 type value mrr_at_10 66.554 type value mrr_at_100 66.96600000000001 type value mrr_at_1000 66.97800000000001 type value mrr_at_3 64.414 type value mrr_at_5 65.77 type value ndcg_at_1 54.964999999999996 type value ndcg_at_10 68.12 type value ndcg_at_100 70.282 type value ndcg_at_1000 70.788 type value ndcg_at_3 63.861999999999995 type value ndcg_at_5 66.216 type value precision_at_1 54.964999999999996 type value precision_at_10 8.998000000000001 type value precision_at_100 1.016 type value precision_at_1000 0.107 type value precision_at_3 25.618000000000002 type value precision_at_5 16.676 type value recall_at_1 50.916 type value recall_at_10 82.04 type value recall_at_100 91.689 type value recall_at_1000 95.34899999999999 type value recall_at_3 70.512 type value recall_at_5 76.29899999999999 task dataset metrics type Retrieval type name config split revision fiqa MTEB FiQA2018 default test 41b686a7f28c59bcaaa5791efd47c67c8ebe28be type value map_at_1 13.568 type value map_at_10 23.264000000000003 type value map_at_100 24.823999999999998 type value map_at_1000 25.013999999999996 type value map_at_3 19.724 type value map_at_5 21.772 type value mrr_at_1 27.315 type value mrr_at_10 35.935 type value mrr_at_100 36.929 type value mrr_at_1000 36.985 type value mrr_at_3 33.591 type value mrr_at_5 34.848 type value ndcg_at_1 27.315 type value ndcg_at_10 29.988 type value ndcg_at_100 36.41 type value ndcg_at_1000 40.184999999999995 type value ndcg_at_3 26.342 type value ndcg_at_5 27.68 type value precision_at_1 27.315 type value precision_at_10 8.565000000000001 type value precision_at_100 1.508 type value precision_at_1000 0.219 type value precision_at_3 17.849999999999998 type value precision_at_5 13.672999999999998 type value recall_at_1 13.568 type value recall_at_10 37.133 type value recall_at_100 61.475 type value recall_at_1000 84.372 type value recall_at_3 24.112000000000002 type value recall_at_5 29.507 task dataset metrics type Retrieval type name config split revision hotpotqa MTEB HotpotQA default test 766870b35a1b9ca65e67a0d1913899973551fc6c type value map_at_1 30.878 type value map_at_10 40.868 type value map_at_100 41.693999999999996 type value map_at_1000 41.775 type value map_at_3 38.56 type value map_at_5 39.947 type value mrr_at_1 61.756 type value mrr_at_10 68.265 type value mrr_at_100 68.671 type value mrr_at_1000 68.694 type value mrr_at_3 66.78399999999999 type value mrr_at_5 67.704 type value ndcg_at_1 61.756 type value ndcg_at_10 49.931 type value ndcg_at_100 53.179 type value ndcg_at_1000 54.94799999999999 type value ndcg_at_3 46.103 type value ndcg_at_5 48.147 type value precision_at_1 61.756 type value precision_at_10 10.163 type value precision_at_100 1.2710000000000001 type value precision_at_1000 0.151 type value precision_at_3 28.179 type value precision_at_5 18.528 type value recall_at_1 30.878 type value recall_at_10 50.817 type value recall_at_100 63.544999999999995 type value recall_at_1000 75.361 type value recall_at_3 42.269 type value recall_at_5 46.32 task dataset metrics type Classification type name config split revision mteb/imdb MTEB ImdbClassification default test 8d743909f834c38949e8323a8a6ce8721ea6c7f4 type value accuracy 64.04799999999999 type value ap 59.185251455339284 type value f1 63.947123181349255 task dataset metrics type Retrieval type name config split revision msmarco MTEB MSMARCO default validation e6838a846e2408f22cf5cc337ebc83e0bcf77849 type value map_at_1 18.9 type value map_at_10 29.748 type value map_at_100 30.976 type value map_at_1000 31.041 type value map_at_3 26.112999999999996 type value map_at_5 28.197 type value mrr_at_1 19.413 type value mrr_at_10 30.322 type value mrr_at_100 31.497000000000003 type value mrr_at_1000 31.555 type value mrr_at_3 26.729000000000003 type value mrr_at_5 28.788999999999998 type value ndcg_at_1 19.413 type value ndcg_at_10 36.048 type value ndcg_at_100 42.152 type value ndcg_at_1000 43.772 type value ndcg_at_3 28.642 type value ndcg_at_5 32.358 type value precision_at_1 19.413 type value precision_at_10 5.785 type value precision_at_100 0.8869999999999999 type value precision_at_1000 0.10300000000000001 type value precision_at_3 12.192 type value precision_at_5 9.189 type value recall_at_1 18.9 type value recall_at_10 55.457 type value recall_at_100 84.09100000000001 type value recall_at_1000 96.482 type value recall_at_3 35.359 type value recall_at_5 44.275 task dataset metrics type Classification type name config split revision mteb/mtop_domain MTEB MTOPDomainClassification (en) en test a7e2a951126a26fc8c6a69f835f33a346ba259e3 type value accuracy 92.07706338349293 type value f1 91.56680443236652 task dataset metrics type Classification type name config split revision mteb/mtop_intent MTEB MTOPIntentClassification (en) en test 6299947a7777084cc2d4b64235bf7190381ce755 type value accuracy 71.18559051527589 type value f1 52.42887061726789 task dataset metrics type Classification type name config split revision mteb/amazon_massive_intent MTEB MassiveIntentClassification (en) en test 072a486a144adf7f4479a4a0dddb2152e161e1ea type value accuracy 68.64828513786148 type value f1 66.54281381596097 task dataset metrics type Classification type name config split revision mteb/amazon_massive_scenario MTEB MassiveScenarioClassification (en) en test 7d571f92784cd94a019292a1f45445077d0ef634 type value accuracy 76.04236718224612 type value f1 75.89170458655639 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-p2p MTEB MedrxivClusteringP2P default test dcefc037ef84348e49b0d29109e891c01067226b type value v_measure 32.0840369055247 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-s2s MTEB MedrxivClusteringS2S default test 3cd0e71dfbe09d4de0f9e5ecba43e7ce280959dc type value v_measure 29.448729560244537 task dataset metrics type Reranking type name config split revision mteb/mind_small MTEB MindSmallReranking default test 3bdac13927fdc888b903db93b2ffdbd90b295a69 type value map 31.340856463122375 type value mrr 32.398547669840916 task dataset metrics type Retrieval type name config split revision nfcorpus MTEB NFCorpus default test 7eb63cc0c1eb59324d709ebed25fcab851fa7610 type value map_at_1 5.526 type value map_at_10 11.745 type value map_at_100 14.831 type value map_at_1000 16.235 type value map_at_3 8.716 type value map_at_5 10.101 type value mrr_at_1 43.653 type value mrr_at_10 51.06699999999999 type value mrr_at_100 51.881 type value mrr_at_1000 51.912000000000006 type value mrr_at_3 49.02 type value mrr_at_5 50.288999999999994 type value ndcg_at_1 41.949999999999996 type value ndcg_at_10 32.083 type value ndcg_at_100 30.049999999999997 type value ndcg_at_1000 38.661 type value ndcg_at_3 37.940000000000005 type value ndcg_at_5 35.455999999999996 type value precision_at_1 43.344 type value precision_at_10 23.437 type value precision_at_100 7.829999999999999 type value precision_at_1000 2.053 type value precision_at_3 35.501 type value precision_at_5 30.464000000000002 type value recall_at_1 5.526 type value recall_at_10 15.445999999999998 type value recall_at_100 31.179000000000002 type value recall_at_1000 61.578 type value recall_at_3 9.71 type value recall_at_5 12.026 task dataset metrics type Retrieval type name config split revision nq MTEB NQ default test 6062aefc120bfe8ece5897809fb2e53bfe0d128c type value map_at_1 23.467 type value map_at_10 36.041000000000004 type value map_at_100 37.268 type value map_at_1000 37.322 type value map_at_3 32.09 type value map_at_5 34.414 type value mrr_at_1 26.738 type value mrr_at_10 38.665 type value mrr_at_100 39.64 type value mrr_at_1000 39.681 type value mrr_at_3 35.207 type value mrr_at_5 37.31 type value ndcg_at_1 26.709 type value ndcg_at_10 42.942 type value ndcg_at_100 48.296 type value ndcg_at_1000 49.651 type value ndcg_at_3 35.413 type value ndcg_at_5 39.367999999999995 type value precision_at_1 26.709 type value precision_at_10 7.306 type value precision_at_100 1.0290000000000001 type value precision_at_1000 0.116 type value precision_at_3 16.348 type value precision_at_5 12.068 type value recall_at_1 23.467 type value recall_at_10 61.492999999999995 type value recall_at_100 85.01100000000001 type value recall_at_1000 95.261 type value recall_at_3 41.952 type value recall_at_5 51.105999999999995 task dataset metrics type Retrieval type name config split revision quora MTEB QuoraRetrieval default test 6205996560df11e3a3da9ab4f926788fc30a7db4 type value map_at_1 67.51700000000001 type value map_at_10 81.054 type value map_at_100 81.727 type value map_at_1000 81.75200000000001 type value map_at_3 78.018 type value map_at_5 79.879 type value mrr_at_1 77.52 type value mrr_at_10 84.429 type value mrr_at_100 84.58200000000001 type value mrr_at_1000 84.584 type value mrr_at_3 83.268 type value mrr_at_5 84.013 type value ndcg_at_1 77.53 type value ndcg_at_10 85.277 type value ndcg_at_100 86.80499999999999 type value ndcg_at_1000 87.01 type value ndcg_at_3 81.975 type value ndcg_at_5 83.723 type value precision_at_1 77.53 type value precision_at_10 12.961 type value precision_at_100 1.502 type value precision_at_1000 0.156 type value precision_at_3 35.713 type value precision_at_5 23.574 type value recall_at_1 67.51700000000001 type value recall_at_10 93.486 type value recall_at_100 98.9 type value recall_at_1000 99.92999999999999 type value recall_at_3 84.17999999999999 type value recall_at_5 88.97500000000001 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering MTEB RedditClustering default test b2805658ae38990172679479369a78b86de8c390 type value v_measure 48.225994608749915 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering-p2p MTEB RedditClusteringP2P default test 385e3cb46b4cfa89021f56c4380204149d0efe33 type value v_measure 53.17635557157765 task dataset metrics type Retrieval type name config split revision scidocs MTEB SCIDOCS default test 5c59ef3e437a0a9651c8fe6fde943e7dce59fba5 type value map_at_1 3.988 type value map_at_10 9.4 type value map_at_100 10.968 type value map_at_1000 11.257 type value map_at_3 7.123 type value map_at_5 8.221 type value mrr_at_1 19.7 type value mrr_at_10 29.098000000000003 type value mrr_at_100 30.247 type value mrr_at_1000 30.318 type value mrr_at_3 26.55 type value mrr_at_5 27.915 type value ndcg_at_1 19.7 type value ndcg_at_10 16.176 type value ndcg_at_100 22.931 type value ndcg_at_1000 28.301 type value ndcg_at_3 16.142 type value ndcg_at_5 13.633999999999999 type value precision_at_1 19.7 type value precision_at_10 8.18 type value precision_at_100 1.8010000000000002 type value precision_at_1000 0.309 type value precision_at_3 15.1 type value precision_at_5 11.74 type value recall_at_1 3.988 type value recall_at_10 16.625 type value recall_at_100 36.61 type value recall_at_1000 62.805 type value recall_at_3 9.168 type value recall_at_5 11.902 task dataset metrics type STS type name config split revision mteb/sickr-sts MTEB SICK-R default test 20a6d6f312dd54037fe07a32d58e5e168867909d type value cos_sim_pearson 77.29330379162072 type value cos_sim_spearman 67.22953551111448 type value euclidean_pearson 71.44682700059415 type value euclidean_spearman 66.33178012153247 type value manhattan_pearson 71.46941734657887 type value manhattan_spearman 66.43234359835814 task dataset metrics type STS type name config split revision mteb/sts12-sts MTEB STS12 default test fdf84275bb8ce4b49c971d02e84dd1abc677a50f type value cos_sim_pearson 75.40943196466576 type value cos_sim_spearman 66.59241013465915 type value euclidean_pearson 71.32500540796616 type value euclidean_spearman 67.86667467202591 type value manhattan_pearson 71.48209832089134 type value manhattan_spearman 67.94511626964879 task dataset metrics type STS type name config split revision mteb/sts13-sts MTEB STS13 default test 1591bfcbe8c69d4bf7fe2a16e2451017832cafb9 type value cos_sim_pearson 77.08302398877518 type value cos_sim_spearman 77.33151317062642 type value euclidean_pearson 76.77020279715008 type value euclidean_spearman 77.13893776083225 type value manhattan_pearson 76.76732290707477 type value manhattan_spearman 77.14500877396631 task dataset metrics type STS type name config split revision mteb/sts14-sts MTEB STS14 default test e2125984e7df8b7871f6ae9949cf6b6795e7c54b type value cos_sim_pearson 77.46886184932168 type value cos_sim_spearman 71.82815265534886 type value euclidean_pearson 75.19783284299076 type value euclidean_spearman 71.36479611710412 type value manhattan_pearson 75.30375233959337 type value manhattan_spearman 71.46280266488021 task dataset metrics type STS type name config split revision mteb/sts15-sts MTEB STS15 default test 1cd7298cac12a96a373b6a2f18738bb3e739a9b6 type value cos_sim_pearson 80.093017609484 type value cos_sim_spearman 80.65931167868882 type value euclidean_pearson 80.36786337117047 type value euclidean_spearman 81.30521389642827 type value manhattan_pearson 80.37922433220973 type value manhattan_spearman 81.30496664496285 task dataset metrics type STS type name config split revision mteb/sts16-sts MTEB STS16 default test 360a0b2dff98700d09e634a01e1cc1624d3e42cd type value cos_sim_pearson 77.98998347238742 type value cos_sim_spearman 78.91151365939403 type value euclidean_pearson 76.40510899217841 type value euclidean_spearman 76.8551459824213 type value manhattan_pearson 76.3986079603294 type value manhattan_spearman 76.8848053254288 task dataset metrics type STS type name config split revision mteb/sts17-crosslingual-sts MTEB STS17 (en-en) en-en test 9fc37e8c632af1c87a3d23e685d49552a02582a0 type value cos_sim_pearson 85.63510653472044 type value cos_sim_spearman 86.98674844768605 type value euclidean_pearson 85.205080538809 type value euclidean_spearman 85.53630494151886 type value manhattan_pearson 85.48612469885626 type value manhattan_spearman 85.81741413931921 task dataset metrics type STS type name config split revision mteb/sts22-crosslingual-sts MTEB STS22 (en) en test 2de6ce8c1921b71a755b262c6b57fef195dd7906 type value cos_sim_pearson 66.7257987615171 type value cos_sim_spearman 67.30387805090024 type value euclidean_pearson 69.46877227885867 type value euclidean_spearman 69.33161798704344 type value manhattan_pearson 69.82773311626424 type value manhattan_spearman 69.57199940498796 task dataset metrics type STS type name config split revision mteb/stsbenchmark-sts MTEB STSBenchmark default test 8913289635987208e6e7c72789e4be2fe94b6abd type value cos_sim_pearson 79.37322139418472 type value cos_sim_spearman 77.5887175717799 type value euclidean_pearson 78.23006410562164 type value euclidean_spearman 77.18470385673044 type value manhattan_pearson 78.40868369362455 type value manhattan_spearman 77.36675823897656 task dataset metrics type Reranking type name config split revision mteb/scidocs-reranking MTEB SciDocsRR default test 56a6d0140cf6356659e2a7c1413286a774468d44 type value map 77.21233007730808 type value mrr 93.0502386139641 task dataset metrics type Retrieval type name config split revision scifact MTEB SciFact default test a75ae049398addde9b70f6b268875f5cbce99089 type value map_at_1 54.567 type value map_at_10 63.653000000000006 type value map_at_100 64.282 type value map_at_1000 64.31099999999999 type value map_at_3 60.478 type value map_at_5 62.322 type value mrr_at_1 56.99999999999999 type value mrr_at_10 64.759 type value mrr_at_100 65.274 type value mrr_at_1000 65.301 type value mrr_at_3 62.333000000000006 type value mrr_at_5 63.817 type value ndcg_at_1 56.99999999999999 type value ndcg_at_10 68.28699999999999 type value ndcg_at_100 70.98400000000001 type value ndcg_at_1000 71.695 type value ndcg_at_3 62.656 type value ndcg_at_5 65.523 type value precision_at_1 56.99999999999999 type value precision_at_10 9.232999999999999 type value precision_at_100 1.0630000000000002 type value precision_at_1000 0.11199999999999999 type value precision_at_3 24.221999999999998 type value precision_at_5 16.333000000000002 type value recall_at_1 54.567 type value recall_at_10 81.45599999999999 type value recall_at_100 93.5 type value recall_at_1000 99.0 type value recall_at_3 66.228 type value recall_at_5 73.489 task dataset metrics type PairClassification type name config split revision mteb/sprintduplicatequestions-pairclassification MTEB SprintDuplicateQuestions default test 5a8256d0dff9c4bd3be3ba3e67e4e70173f802ea type value cos_sim_accuracy 99.74455445544554 type value cos_sim_ap 92.57836032673468 type value cos_sim_f1 87.0471464019851 type value cos_sim_precision 86.4039408866995 type value cos_sim_recall 87.7 type value dot_accuracy 99.56039603960396 type value dot_ap 82.47233353407186 type value dot_f1 76.78207739307537 type value dot_precision 78.21576763485477 type value dot_recall 75.4 type value euclidean_accuracy 99.73069306930694 type value euclidean_ap 91.70507666665775 type value euclidean_f1 86.26262626262626 type value euclidean_precision 87.14285714285714 type value euclidean_recall 85.39999999999999 type value manhattan_accuracy 99.73861386138614 type value manhattan_ap 91.96809459281754 type value manhattan_f1 86.6 type value manhattan_precision 86.6 type value manhattan_recall 86.6 type value max_accuracy 99.74455445544554 type value max_ap 92.57836032673468 type value max_f1 87.0471464019851 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering MTEB StackExchangeClustering default test 70a89468f6dccacc6aa2b12a6eac54e74328f235 type value v_measure 60.85593925770172 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering-p2p MTEB StackExchangeClusteringP2P default test d88009ab563dd0b16cfaf4436abaf97fa3550cf0 type value v_measure 32.356772998237496 task dataset metrics type Reranking type name config split revision mteb/stackoverflowdupquestions-reranking MTEB StackOverflowDupQuestions default test ef807ea29a75ec4f91b50fd4191cb4ee4589a9f9 type value map 49.320607035290735 type value mrr 50.09196481622952 task dataset metrics type Summarization type name config split revision mteb/summeval MTEB SummEval default test 8753c2788d36c01fc6f05d03fe3f7268d63f9122 type value cos_sim_pearson 31.17573968015504 type value cos_sim_spearman 30.43371643155132 type value dot_pearson 30.164319483092744 type value dot_spearman 29.207082242868754 task dataset metrics type Retrieval type name config split revision trec-covid MTEB TRECCOVID default test 2c8041b2c07a79b6f7ba8fe6acc72e5d9f92d217 type value map_at_1 0.22100000000000003 type value map_at_10 1.7229999999999999 type value map_at_100 9.195 type value map_at_1000 21.999 type value map_at_3 0.6479999999999999 type value map_at_5 0.964 type value mrr_at_1 86.0 type value mrr_at_10 90.667 type value mrr_at_100 90.858 type value mrr_at_1000 90.858 type value mrr_at_3 90.667 type value mrr_at_5 90.667 type value ndcg_at_1 82.0 type value ndcg_at_10 72.98 type value ndcg_at_100 52.868 type value ndcg_at_1000 46.541 type value ndcg_at_3 80.39699999999999 type value ndcg_at_5 76.303 type value precision_at_1 86.0 type value precision_at_10 75.8 type value precision_at_100 53.5 type value precision_at_1000 20.946 type value precision_at_3 85.333 type value precision_at_5 79.2 type value recall_at_1 0.22100000000000003 type value recall_at_10 1.9109999999999998 type value recall_at_100 12.437 type value recall_at_1000 43.606 type value recall_at_3 0.681 type value recall_at_5 1.023 task dataset metrics type Retrieval type name config split revision webis-touche2020 MTEB Touche2020 default test 527b7d77e16e343303e68cb6af11d6e18b9f7b3b type value map_at_1 2.5 type value map_at_10 9.568999999999999 type value map_at_100 15.653 type value map_at_1000 17.188 type value map_at_3 5.335999999999999 type value map_at_5 6.522 type value mrr_at_1 34.694 type value mrr_at_10 49.184 type value mrr_at_100 50.512 type value mrr_at_1000 50.512 type value mrr_at_3 46.259 type value mrr_at_5 48.299 type value ndcg_at_1 30.612000000000002 type value ndcg_at_10 24.45 type value ndcg_at_100 35.870999999999995 type value ndcg_at_1000 47.272999999999996 type value ndcg_at_3 28.528 type value ndcg_at_5 25.768 type value precision_at_1 34.694 type value precision_at_10 21.429000000000002 type value precision_at_100 7.265000000000001 type value precision_at_1000 1.504 type value precision_at_3 29.252 type value precision_at_5 24.898 type value recall_at_1 2.5 type value recall_at_10 15.844 type value recall_at_100 45.469 type value recall_at_1000 81.148 type value recall_at_3 6.496 type value recall_at_5 8.790000000000001 task dataset metrics type Classification type name config split revision mteb/toxic_conversations_50k MTEB ToxicConversationsClassification default test edfaf9da55d3dd50d43143d90c1ac476895ae6de type value accuracy 68.7272 type value ap 13.156450706152686 type value f1 52.814703437064395 task dataset metrics type Classification type name config split revision mteb/tweet_sentiment_extraction MTEB TweetSentimentExtractionClassification default test 62146448f05be9e52a36b8ee9936447ea787eede type value accuracy 55.6677985285795 type value f1 55.9373937514999 task dataset metrics type Clustering type name config split revision mteb/twentynewsgroups-clustering MTEB TwentyNewsgroupsClustering default test 091a54f9a36281ce7d6590ec8c75dd485e7e01d4 type value v_measure 40.05809562275603 task dataset metrics type PairClassification type name config split revision mteb/twittersemeval2015-pairclassification MTEB TwitterSemEval2015 default test 70970daeab8776df92f5ea462b6173c0b46fd2d1 type value cos_sim_accuracy 82.76807534124099 type value cos_sim_ap 62.37052608803734 type value cos_sim_f1 59.077414934916646 type value cos_sim_precision 52.07326892109501 type value cos_sim_recall 68.25857519788919 type value dot_accuracy 80.56267509089825 type value dot_ap 54.75349561321037 type value dot_f1 54.75483794372552 type value dot_precision 49.77336499028707 type value dot_recall 60.844327176781 type value euclidean_accuracy 82.476008821601 type value euclidean_ap 61.17417554210511 type value euclidean_f1 57.80318696022382 type value euclidean_precision 53.622207176709544 type value euclidean_recall 62.69129287598945 type value manhattan_accuracy 82.48792990403528 type value manhattan_ap 61.044816292966544 type value manhattan_f1 58.03033951360462 type value manhattan_precision 53.36581045172719 type value manhattan_recall 63.58839050131926 type value max_accuracy 82.76807534124099 type value max_ap 62.37052608803734 type value max_f1 59.077414934916646 task dataset metrics type PairClassification type name config split revision mteb/twitterurlcorpus-pairclassification MTEB TwitterURLCorpus default test 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf type value cos_sim_accuracy 87.97881010594946 type value cos_sim_ap 83.78748636891035 type value cos_sim_f1 75.94113995691386 type value cos_sim_precision 72.22029307590805 type value cos_sim_recall 80.06621496766245 type value dot_accuracy 85.69294058291614 type value dot_ap 78.15363722278026 type value dot_f1 72.08894926888564 type value dot_precision 67.28959487419075 type value dot_recall 77.62550046196489 type value euclidean_accuracy 87.73625179493149 type value euclidean_ap 83.19012184470559 type value euclidean_f1 75.5148064623461 type value euclidean_precision 72.63352535381551 type value euclidean_recall 78.6341238065907 type value manhattan_accuracy 87.74013272790779 type value manhattan_ap 83.23305405113403 type value manhattan_f1 75.63960775639607 type value manhattan_precision 72.563304569246 type value manhattan_recall 78.9882968894364 type value max_accuracy 87.97881010594946 type value max_ap 83.78748636891035 type value max_f1 75.94113995691386

SGPT-1.3B-weightedmean-msmarco-specb-bitfit

Usage

For usage instructions, refer to our codebase: https://github.com/Muennighoff/sgpt

Evaluation Results

For eval results, refer to the eval folder or our paper: https://arxiv.org/abs/2202.08904

Training

The model was trained with the parameters:

DataLoader:

torch.utils.data.dataloader.DataLoader of length 62398 with parameters:

{'batch_size': 8, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}

Loss:

sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss with parameters:

{'scale': 20.0, 'similarity_fct': 'cos_sim'}

Parameters of the fit()-Method:

{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 0.0002
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 1000,
    "weight_decay": 0.01
}

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 300, 'do_lower_case': False}) with Transformer model: GPTNeoModel 
  (1): Pooling({'word_embedding_dimension': 2048, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': True, 'pooling_mode_lasttoken': False})
)

Citing & Authors

@article{muennighoff2022sgpt,
  title={SGPT: GPT Sentence Embeddings for Semantic Search},
  author={Muennighoff, Niklas},
  journal={arXiv preprint arXiv:2202.08904},
  year={2022}
}