jina-embeddings-v2-small-en/README.md

tags, datasets, language, inference, license, model-index

tags	datasets	language	inference	license	model-index
sentence-transformers feature-extraction sentence-similarity mteb	jinaai/negation-dataset	en	false	apache-2.0	name results jina-embedding-s-en-v2 task dataset metrics type Classification type name config split revision mteb/amazon_counterfactual MTEB AmazonCounterfactualClassification (en) en test e8379541af4e31359cca9fbcf4b00f2671dba205 type value accuracy 71.35820895522387 type value ap 33.99931933598115 type value f1 65.3853685535555 task dataset metrics type Classification type name config split revision mteb/amazon_polarity MTEB AmazonPolarityClassification default test e2d317d38cd51312af73b3d32a06d1a08b442046 type value accuracy 82.90140000000001 type value ap 78.01434597815617 type value f1 82.83357802722676 task dataset metrics type Classification type name config split revision mteb/amazon_reviews_multi MTEB AmazonReviewsClassification (en) en test 1399c76144fd37290681b995c656ef9b2e06e26d type value accuracy 40.88999999999999 type value f1 39.209432767163456 task dataset metrics type Retrieval type name config split revision arguana MTEB ArguAna default test None type value map_at_1 23.257 type value map_at_10 37.946000000000005 type value map_at_100 39.17 type value map_at_1000 39.181 type value map_at_3 32.99 type value map_at_5 35.467999999999996 type value mrr_at_1 23.541999999999998 type value mrr_at_10 38.057 type value mrr_at_100 39.289 type value mrr_at_1000 39.299 type value mrr_at_3 33.096 type value mrr_at_5 35.628 type value ndcg_at_1 23.257 type value ndcg_at_10 46.729 type value ndcg_at_100 51.900999999999996 type value ndcg_at_1000 52.16 type value ndcg_at_3 36.323 type value ndcg_at_5 40.766999999999996 type value precision_at_1 23.257 type value precision_at_10 7.510999999999999 type value precision_at_100 0.976 type value precision_at_1000 0.1 type value precision_at_3 15.339 type value precision_at_5 11.350999999999999 type value recall_at_1 23.257 type value recall_at_10 75.107 type value recall_at_100 97.58200000000001 type value recall_at_1000 99.57300000000001 type value recall_at_3 46.017 type value recall_at_5 56.757000000000005 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-p2p MTEB ArxivClusteringP2P default test a122ad7f3f0291bf49cc6f4d32aa80929df69d5d type value v_measure 44.02420878391967 task dataset metrics type Clustering type name config split revision mteb/arxiv-clustering-s2s MTEB ArxivClusteringS2S default test f910caf1a6075f7329cdf8c1a6135696f37dbd53 type value v_measure 35.16136856000258 task dataset metrics type Reranking type name config split revision mteb/askubuntudupquestions-reranking MTEB AskUbuntuDupQuestions default test 2000358ca161889fa9c082cb41daa8dcfb161a54 type value map 59.61809790513646 type value mrr 73.07215406938397 task dataset metrics type STS type name config split revision mteb/biosses-sts MTEB BIOSSES default test d3fb88f8f02e40887cd149695127462bbcf29b4a type value cos_sim_pearson 82.0167350090749 type value cos_sim_spearman 80.51569002630401 type value euclidean_pearson 81.46820525099726 type value euclidean_spearman 80.51569002630401 type value manhattan_pearson 81.35596555056757 type value manhattan_spearman 80.12592210903303 task dataset metrics type Classification type name config split revision mteb/banking77 MTEB Banking77Classification default test 0fd18e25b25c072e09e0d92ab615fda904d66300 type value accuracy 78.25 type value f1 77.34950913540605 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-p2p MTEB BiorxivClusteringP2P default test 65b79d1d13f80053f67aca9498d9402c2d9f1f40 type value v_measure 35.57238596005698 task dataset metrics type Clustering type name config split revision mteb/biorxiv-clustering-s2s MTEB BiorxivClusteringS2S default test 258694dd0231531bc1fd9de6ceb52a0853c6d908 type value v_measure 29.066444306196683 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackAndroidRetrieval default test None type value map_at_1 31.891000000000002 type value map_at_10 42.772 type value map_at_100 44.108999999999995 type value map_at_1000 44.236 type value map_at_3 39.289 type value map_at_5 41.113 type value mrr_at_1 39.342 type value mrr_at_10 48.852000000000004 type value mrr_at_100 49.534 type value mrr_at_1000 49.582 type value mrr_at_3 46.089999999999996 type value mrr_at_5 47.685 type value ndcg_at_1 39.342 type value ndcg_at_10 48.988 type value ndcg_at_100 53.854 type value ndcg_at_1000 55.955 type value ndcg_at_3 43.877 type value ndcg_at_5 46.027 type value precision_at_1 39.342 type value precision_at_10 9.285 type value precision_at_100 1.488 type value precision_at_1000 0.194 type value precision_at_3 20.696 type value precision_at_5 14.878 type value recall_at_1 31.891000000000002 type value recall_at_10 60.608 type value recall_at_100 81.025 type value recall_at_1000 94.883 type value recall_at_3 45.694 type value recall_at_5 51.684 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackEnglishRetrieval default test None type value map_at_1 28.778 type value map_at_10 37.632 type value map_at_100 38.800000000000004 type value map_at_1000 38.934999999999995 type value map_at_3 35.293 type value map_at_5 36.547000000000004 type value mrr_at_1 35.35 type value mrr_at_10 42.936 type value mrr_at_100 43.69 type value mrr_at_1000 43.739 type value mrr_at_3 41.062 type value mrr_at_5 42.097 type value ndcg_at_1 35.35 type value ndcg_at_10 42.528 type value ndcg_at_100 46.983000000000004 type value ndcg_at_1000 49.187999999999995 type value ndcg_at_3 39.271 type value ndcg_at_5 40.654 type value precision_at_1 35.35 type value precision_at_10 7.828 type value precision_at_100 1.3010000000000002 type value precision_at_1000 0.17700000000000002 type value precision_at_3 18.96 type value precision_at_5 13.120999999999999 type value recall_at_1 28.778 type value recall_at_10 50.775000000000006 type value recall_at_100 69.66799999999999 type value recall_at_1000 83.638 type value recall_at_3 40.757 type value recall_at_5 44.86 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGamingRetrieval default test None type value map_at_1 37.584 type value map_at_10 49.69 type value map_at_100 50.639 type value map_at_1000 50.702999999999996 type value map_at_3 46.61 type value map_at_5 48.486000000000004 type value mrr_at_1 43.009 type value mrr_at_10 52.949999999999996 type value mrr_at_100 53.618 type value mrr_at_1000 53.65299999999999 type value mrr_at_3 50.605999999999995 type value mrr_at_5 52.095 type value ndcg_at_1 43.009 type value ndcg_at_10 55.278000000000006 type value ndcg_at_100 59.134 type value ndcg_at_1000 60.528999999999996 type value ndcg_at_3 50.184 type value ndcg_at_5 52.919000000000004 type value precision_at_1 43.009 type value precision_at_10 8.821 type value precision_at_100 1.161 type value precision_at_1000 0.133 type value precision_at_3 22.424 type value precision_at_5 15.436 type value recall_at_1 37.584 type value recall_at_10 68.514 type value recall_at_100 85.099 type value recall_at_1000 95.123 type value recall_at_3 55.007 type value recall_at_5 61.714999999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackGisRetrieval default test None type value map_at_1 24.7 type value map_at_10 32.804 type value map_at_100 33.738 type value map_at_1000 33.825 type value map_at_3 30.639 type value map_at_5 31.781 type value mrr_at_1 26.328000000000003 type value mrr_at_10 34.679 type value mrr_at_100 35.510000000000005 type value mrr_at_1000 35.577999999999996 type value mrr_at_3 32.58 type value mrr_at_5 33.687 type value ndcg_at_1 26.328000000000003 type value ndcg_at_10 37.313 type value ndcg_at_100 42.004000000000005 type value ndcg_at_1000 44.232 type value ndcg_at_3 33.076 type value ndcg_at_5 34.966 type value precision_at_1 26.328000000000003 type value precision_at_10 5.627 type value precision_at_100 0.8410000000000001 type value precision_at_1000 0.106 type value precision_at_3 14.011000000000001 type value precision_at_5 9.582 type value recall_at_1 24.7 type value recall_at_10 49.324 type value recall_at_100 71.018 type value recall_at_1000 87.905 type value recall_at_3 37.7 type value recall_at_5 42.281 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackMathematicaRetrieval default test None type value map_at_1 14.350999999999999 type value map_at_10 21.745 type value map_at_100 22.731 type value map_at_1000 22.852 type value map_at_3 19.245 type value map_at_5 20.788 type value mrr_at_1 18.159 type value mrr_at_10 25.833000000000002 type value mrr_at_100 26.728 type value mrr_at_1000 26.802 type value mrr_at_3 23.383000000000003 type value mrr_at_5 24.887999999999998 type value ndcg_at_1 18.159 type value ndcg_at_10 26.518000000000004 type value ndcg_at_100 31.473000000000003 type value ndcg_at_1000 34.576 type value ndcg_at_3 21.907 type value ndcg_at_5 24.39 type value precision_at_1 18.159 type value precision_at_10 4.938 type value precision_at_100 0.853 type value precision_at_1000 0.125 type value precision_at_3 10.655000000000001 type value precision_at_5 7.985 type value recall_at_1 14.350999999999999 type value recall_at_10 37.284 type value recall_at_100 59.11300000000001 type value recall_at_1000 81.634 type value recall_at_3 24.753 type value recall_at_5 30.979 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackPhysicsRetrieval default test None type value map_at_1 26.978 type value map_at_10 36.276 type value map_at_100 37.547000000000004 type value map_at_1000 37.678 type value map_at_3 33.674 type value map_at_5 35.119 type value mrr_at_1 32.916000000000004 type value mrr_at_10 41.798 type value mrr_at_100 42.72 type value mrr_at_1000 42.778 type value mrr_at_3 39.493 type value mrr_at_5 40.927 type value ndcg_at_1 32.916000000000004 type value ndcg_at_10 41.81 type value ndcg_at_100 47.284 type value ndcg_at_1000 49.702 type value ndcg_at_3 37.486999999999995 type value ndcg_at_5 39.597 type value precision_at_1 32.916000000000004 type value precision_at_10 7.411 type value precision_at_100 1.189 type value precision_at_1000 0.158 type value precision_at_3 17.581 type value precision_at_5 12.397 type value recall_at_1 26.978 type value recall_at_10 52.869 type value recall_at_100 75.78399999999999 type value recall_at_1000 91.545 type value recall_at_3 40.717 type value recall_at_5 46.168 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackProgrammersRetrieval default test None type value map_at_1 24.641 type value map_at_10 32.916000000000004 type value map_at_100 34.165 type value map_at_1000 34.286 type value map_at_3 30.335 type value map_at_5 31.569000000000003 type value mrr_at_1 30.593999999999998 type value mrr_at_10 38.448 type value mrr_at_100 39.299 type value mrr_at_1000 39.362 type value mrr_at_3 36.244 type value mrr_at_5 37.232 type value ndcg_at_1 30.593999999999998 type value ndcg_at_10 38.2 type value ndcg_at_100 43.742 type value ndcg_at_1000 46.217000000000006 type value ndcg_at_3 33.925 type value ndcg_at_5 35.394 type value precision_at_1 30.593999999999998 type value precision_at_10 6.895 type value precision_at_100 1.1320000000000001 type value precision_at_1000 0.153 type value precision_at_3 16.096 type value precision_at_5 11.05 type value recall_at_1 24.641 type value recall_at_10 48.588 type value recall_at_100 72.841 type value recall_at_1000 89.535 type value recall_at_3 36.087 type value recall_at_5 40.346 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackRetrieval default test None type value map_at_1 24.79425 type value map_at_10 33.12033333333333 type value map_at_100 34.221333333333334 type value map_at_1000 34.3435 type value map_at_3 30.636583333333338 type value map_at_5 31.974083333333326 type value mrr_at_1 29.242416666666664 type value mrr_at_10 37.11675 type value mrr_at_100 37.93783333333334 type value mrr_at_1000 38.003083333333336 type value mrr_at_3 34.904666666666664 type value mrr_at_5 36.12916666666667 type value ndcg_at_1 29.242416666666664 type value ndcg_at_10 38.03416666666667 type value ndcg_at_100 42.86674999999999 type value ndcg_at_1000 45.34550000000001 type value ndcg_at_3 33.76466666666666 type value ndcg_at_5 35.668666666666674 type value precision_at_1 29.242416666666664 type value precision_at_10 6.589833333333334 type value precision_at_100 1.0693333333333332 type value precision_at_1000 0.14641666666666667 type value precision_at_3 15.430749999999998 type value precision_at_5 10.833833333333333 type value recall_at_1 24.79425 type value recall_at_10 48.582916666666655 type value recall_at_100 69.88499999999999 type value recall_at_1000 87.211 type value recall_at_3 36.625499999999995 type value recall_at_5 41.553999999999995 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackStatsRetrieval default test None type value map_at_1 22.767 type value map_at_10 28.450999999999997 type value map_at_100 29.332 type value map_at_1000 29.426000000000002 type value map_at_3 26.379 type value map_at_5 27.584999999999997 type value mrr_at_1 25.46 type value mrr_at_10 30.974 type value mrr_at_100 31.784000000000002 type value mrr_at_1000 31.857999999999997 type value mrr_at_3 28.962 type value mrr_at_5 30.066 type value ndcg_at_1 25.46 type value ndcg_at_10 32.041 type value ndcg_at_100 36.522 type value ndcg_at_1000 39.101 type value ndcg_at_3 28.152 type value ndcg_at_5 30.03 type value precision_at_1 25.46 type value precision_at_10 4.893 type value precision_at_100 0.77 type value precision_at_1000 0.107 type value precision_at_3 11.605 type value precision_at_5 8.19 type value recall_at_1 22.767 type value recall_at_10 40.71 type value recall_at_100 61.334999999999994 type value recall_at_1000 80.567 type value recall_at_3 30.198000000000004 type value recall_at_5 34.803 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackTexRetrieval default test None type value map_at_1 16.722 type value map_at_10 22.794 type value map_at_100 23.7 type value map_at_1000 23.822 type value map_at_3 20.781 type value map_at_5 22.024 type value mrr_at_1 20.061999999999998 type value mrr_at_10 26.346999999999998 type value mrr_at_100 27.153 type value mrr_at_1000 27.233 type value mrr_at_3 24.375 type value mrr_at_5 25.593 type value ndcg_at_1 20.061999999999998 type value ndcg_at_10 26.785999999999998 type value ndcg_at_100 31.319999999999997 type value ndcg_at_1000 34.346 type value ndcg_at_3 23.219 type value ndcg_at_5 25.107000000000003 type value precision_at_1 20.061999999999998 type value precision_at_10 4.78 type value precision_at_100 0.83 type value precision_at_1000 0.125 type value precision_at_3 10.874 type value precision_at_5 7.956 type value recall_at_1 16.722 type value recall_at_10 35.204 type value recall_at_100 55.797 type value recall_at_1000 77.689 type value recall_at_3 25.245 type value recall_at_5 30.115 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackUnixRetrieval default test None type value map_at_1 24.842 type value map_at_10 32.917 type value map_at_100 33.961000000000006 type value map_at_1000 34.069 type value map_at_3 30.595 type value map_at_5 31.837 type value mrr_at_1 29.011 type value mrr_at_10 36.977 type value mrr_at_100 37.814 type value mrr_at_1000 37.885999999999996 type value mrr_at_3 34.966 type value mrr_at_5 36.043 type value ndcg_at_1 29.011 type value ndcg_at_10 37.735 type value ndcg_at_100 42.683 type value ndcg_at_1000 45.198 type value ndcg_at_3 33.650000000000006 type value ndcg_at_5 35.386 type value precision_at_1 29.011 type value precision_at_10 6.259 type value precision_at_100 0.984 type value precision_at_1000 0.13 type value precision_at_3 15.329999999999998 type value precision_at_5 10.541 type value recall_at_1 24.842 type value recall_at_10 48.304 type value recall_at_100 70.04899999999999 type value recall_at_1000 87.82600000000001 type value recall_at_3 36.922 type value recall_at_5 41.449999999999996 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWebmastersRetrieval default test None type value map_at_1 24.252000000000002 type value map_at_10 32.293 type value map_at_100 33.816 type value map_at_1000 34.053 type value map_at_3 29.781999999999996 type value map_at_5 31.008000000000003 type value mrr_at_1 29.051 type value mrr_at_10 36.722 type value mrr_at_100 37.663000000000004 type value mrr_at_1000 37.734 type value mrr_at_3 34.354 type value mrr_at_5 35.609 type value ndcg_at_1 29.051 type value ndcg_at_10 37.775999999999996 type value ndcg_at_100 43.221 type value ndcg_at_1000 46.116 type value ndcg_at_3 33.403 type value ndcg_at_5 35.118 type value precision_at_1 29.051 type value precision_at_10 7.332 type value precision_at_100 1.49 type value precision_at_1000 0.23600000000000002 type value precision_at_3 15.415000000000001 type value precision_at_5 11.107 type value recall_at_1 24.252000000000002 type value recall_at_10 47.861 type value recall_at_100 72.21600000000001 type value recall_at_1000 90.886 type value recall_at_3 35.533 type value recall_at_5 39.959 task dataset metrics type Retrieval type name config split revision BeIR/cqadupstack MTEB CQADupstackWordpressRetrieval default test None type value map_at_1 20.025000000000002 type value map_at_10 27.154 type value map_at_100 28.118 type value map_at_1000 28.237000000000002 type value map_at_3 25.017 type value map_at_5 25.832 type value mrr_at_1 21.627 type value mrr_at_10 28.884999999999998 type value mrr_at_100 29.741 type value mrr_at_1000 29.831999999999997 type value mrr_at_3 26.741 type value mrr_at_5 27.628000000000004 type value ndcg_at_1 21.627 type value ndcg_at_10 31.436999999999998 type value ndcg_at_100 36.181000000000004 type value ndcg_at_1000 38.986 type value ndcg_at_3 27.025 type value ndcg_at_5 28.436 type value precision_at_1 21.627 type value precision_at_10 5.009 type value precision_at_100 0.7929999999999999 type value precision_at_1000 0.11299999999999999 type value precision_at_3 11.522 type value precision_at_5 7.763000000000001 type value recall_at_1 20.025000000000002 type value recall_at_10 42.954 type value recall_at_100 64.67500000000001 type value recall_at_1000 85.301 type value recall_at_3 30.892999999999997 type value recall_at_5 34.288000000000004 task dataset metrics type Retrieval type name config split revision climate-fever MTEB ClimateFEVER default test None type value map_at_1 10.079 type value map_at_10 16.930999999999997 type value map_at_100 18.398999999999997 type value map_at_1000 18.561 type value map_at_3 14.294 type value map_at_5 15.579 type value mrr_at_1 22.606 type value mrr_at_10 32.513 type value mrr_at_100 33.463 type value mrr_at_1000 33.513999999999996 type value mrr_at_3 29.479 type value mrr_at_5 31.3 type value ndcg_at_1 22.606 type value ndcg_at_10 24.053 type value ndcg_at_100 30.258000000000003 type value ndcg_at_1000 33.516 type value ndcg_at_3 19.721 type value ndcg_at_5 21.144 type value precision_at_1 22.606 type value precision_at_10 7.55 type value precision_at_100 1.399 type value precision_at_1000 0.2 type value precision_at_3 14.701 type value precision_at_5 11.192 type value recall_at_1 10.079 type value recall_at_10 28.970000000000002 type value recall_at_100 50.805 type value recall_at_1000 69.378 type value recall_at_3 18.199 type value recall_at_5 22.442 task dataset metrics type Retrieval type name config split revision dbpedia-entity MTEB DBPedia default test None type value map_at_1 7.794 type value map_at_10 15.165999999999999 type value map_at_100 20.508000000000003 type value map_at_1000 21.809 type value map_at_3 11.568000000000001 type value map_at_5 13.059000000000001 type value mrr_at_1 56.49999999999999 type value mrr_at_10 65.90899999999999 type value mrr_at_100 66.352 type value mrr_at_1000 66.369 type value mrr_at_3 64.0 type value mrr_at_5 65.10000000000001 type value ndcg_at_1 44.25 type value ndcg_at_10 32.649 type value ndcg_at_100 36.668 type value ndcg_at_1000 43.918 type value ndcg_at_3 37.096000000000004 type value ndcg_at_5 34.048 type value precision_at_1 56.49999999999999 type value precision_at_10 25.45 type value precision_at_100 8.055 type value precision_at_1000 1.7489999999999999 type value precision_at_3 41.0 type value precision_at_5 32.85 type value recall_at_1 7.794 type value recall_at_10 20.101 type value recall_at_100 42.448 type value recall_at_1000 65.88000000000001 type value recall_at_3 12.753 type value recall_at_5 15.307 task dataset metrics type Classification type name config split revision mteb/emotion MTEB EmotionClassification default test 4f58c6b202a23cf9a4da393831edf4f9183cad37 type value accuracy 44.01 type value f1 38.659680951114964 task dataset metrics type Retrieval type name config split revision fever MTEB FEVER default test None type value map_at_1 49.713 type value map_at_10 61.79 type value map_at_100 62.28 type value map_at_1000 62.297000000000004 type value map_at_3 59.361 type value map_at_5 60.92100000000001 type value mrr_at_1 53.405 type value mrr_at_10 65.79899999999999 type value mrr_at_100 66.219 type value mrr_at_1000 66.227 type value mrr_at_3 63.431000000000004 type value mrr_at_5 64.98 type value ndcg_at_1 53.405 type value ndcg_at_10 68.01899999999999 type value ndcg_at_100 70.197 type value ndcg_at_1000 70.571 type value ndcg_at_3 63.352 type value ndcg_at_5 66.018 type value precision_at_1 53.405 type value precision_at_10 9.119 type value precision_at_100 1.03 type value precision_at_1000 0.107 type value precision_at_3 25.602999999999998 type value precision_at_5 16.835 type value recall_at_1 49.713 type value recall_at_10 83.306 type value recall_at_100 92.92 type value recall_at_1000 95.577 type value recall_at_3 70.798 type value recall_at_5 77.254 task dataset metrics type Retrieval type name config split revision fiqa MTEB FiQA2018 default test None type value map_at_1 15.310000000000002 type value map_at_10 26.204 type value map_at_100 27.932000000000002 type value map_at_1000 28.121000000000002 type value map_at_3 22.481 type value map_at_5 24.678 type value mrr_at_1 29.784 type value mrr_at_10 39.582 type value mrr_at_100 40.52 type value mrr_at_1000 40.568 type value mrr_at_3 37.114000000000004 type value mrr_at_5 38.596000000000004 type value ndcg_at_1 29.784 type value ndcg_at_10 33.432 type value ndcg_at_100 40.281 type value ndcg_at_1000 43.653999999999996 type value ndcg_at_3 29.612 type value ndcg_at_5 31.223 type value precision_at_1 29.784 type value precision_at_10 9.645 type value precision_at_100 1.645 type value precision_at_1000 0.22499999999999998 type value precision_at_3 20.165 type value precision_at_5 15.401000000000002 type value recall_at_1 15.310000000000002 type value recall_at_10 40.499 type value recall_at_100 66.643 type value recall_at_1000 87.059 type value recall_at_3 27.492 type value recall_at_5 33.748 task dataset metrics type Retrieval type name config split revision hotpotqa MTEB HotpotQA default test None type value map_at_1 33.599000000000004 type value map_at_10 47.347 type value map_at_100 48.191 type value map_at_1000 48.263 type value map_at_3 44.698 type value map_at_5 46.278999999999996 type value mrr_at_1 67.19800000000001 type value mrr_at_10 74.054 type value mrr_at_100 74.376 type value mrr_at_1000 74.392 type value mrr_at_3 72.849 type value mrr_at_5 73.643 type value ndcg_at_1 67.19800000000001 type value ndcg_at_10 56.482 type value ndcg_at_100 59.694 type value ndcg_at_1000 61.204 type value ndcg_at_3 52.43299999999999 type value ndcg_at_5 54.608000000000004 type value precision_at_1 67.19800000000001 type value precision_at_10 11.613999999999999 type value precision_at_100 1.415 type value precision_at_1000 0.16199999999999998 type value precision_at_3 32.726 type value precision_at_5 21.349999999999998 type value recall_at_1 33.599000000000004 type value recall_at_10 58.069 type value recall_at_100 70.736 type value recall_at_1000 80.804 type value recall_at_3 49.088 type value recall_at_5 53.376000000000005 task dataset metrics type Classification type name config split revision mteb/imdb MTEB ImdbClassification default test 3d86128a09e091d6018b6d26cad27f2739fc2db7 type value accuracy 73.64359999999999 type value ap 67.54685976014599 type value f1 73.55148707559482 task dataset metrics type Retrieval type name config split revision msmarco MTEB MSMARCO default dev None type value map_at_1 19.502 type value map_at_10 30.816 type value map_at_100 32.007999999999996 type value map_at_1000 32.067 type value map_at_3 27.215 type value map_at_5 29.304000000000002 type value mrr_at_1 20.072000000000003 type value mrr_at_10 31.406 type value mrr_at_100 32.549 type value mrr_at_1000 32.602 type value mrr_at_3 27.839000000000002 type value mrr_at_5 29.926000000000002 type value ndcg_at_1 20.086000000000002 type value ndcg_at_10 37.282 type value ndcg_at_100 43.206 type value ndcg_at_1000 44.690000000000005 type value ndcg_at_3 29.932 type value ndcg_at_5 33.668 type value precision_at_1 20.086000000000002 type value precision_at_10 5.961 type value precision_at_100 0.898 type value precision_at_1000 0.10200000000000001 type value precision_at_3 12.856000000000002 type value precision_at_5 9.596 type value recall_at_1 19.502 type value recall_at_10 57.182 type value recall_at_100 84.952 type value recall_at_1000 96.34700000000001 type value recall_at_3 37.193 type value recall_at_5 46.157 task dataset metrics type Classification type name config split revision mteb/mtop_domain MTEB MTOPDomainClassification (en) en test d80d48c1eb48d3562165c59d59d0034df9fff0bf type value accuracy 93.96488828089375 type value f1 93.32119260543482 task dataset metrics type Classification type name config split revision mteb/mtop_intent MTEB MTOPIntentClassification (en) en test ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba type value accuracy 72.4965800273598 type value f1 49.34896217536082 task dataset metrics type Classification type name config split revision mteb/amazon_massive_intent MTEB MassiveIntentClassification (en) en test 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 type value accuracy 67.60928043039678 type value f1 64.34244712074538 task dataset metrics type Classification type name config split revision mteb/amazon_massive_scenario MTEB MassiveScenarioClassification (en) en test 7d571f92784cd94a019292a1f45445077d0ef634 type value accuracy 69.75453934095493 type value f1 68.39224867489249 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-p2p MTEB MedrxivClusteringP2P default test e7a26af6f3ae46b30dde8737f02c07b1505bcc73 type value v_measure 31.862573504920082 task dataset metrics type Clustering type name config split revision mteb/medrxiv-clustering-s2s MTEB MedrxivClusteringS2S default test 35191c8c0dca72d8ff3efcd72aa802307d469663 type value v_measure 27.511123551196803 task dataset metrics type Reranking type name config split revision mteb/mind_small MTEB MindSmallReranking default test 3bdac13927fdc888b903db93b2ffdbd90b295a69 type value map 30.99145104942086 type value mrr 32.03606480418627 task dataset metrics type Retrieval type name config split revision nfcorpus MTEB NFCorpus default test None type value map_at_1 5.015 type value map_at_10 11.054 type value map_at_100 13.773 type value map_at_1000 15.082999999999998 type value map_at_3 8.253 type value map_at_5 9.508999999999999 type value mrr_at_1 42.105 type value mrr_at_10 50.44499999999999 type value mrr_at_100 51.080000000000005 type value mrr_at_1000 51.129999999999995 type value mrr_at_3 48.555 type value mrr_at_5 49.84 type value ndcg_at_1 40.402 type value ndcg_at_10 30.403000000000002 type value ndcg_at_100 28.216 type value ndcg_at_1000 37.021 type value ndcg_at_3 35.53 type value ndcg_at_5 33.202999999999996 type value precision_at_1 42.105 type value precision_at_10 22.353 type value precision_at_100 7.266 type value precision_at_1000 2.011 type value precision_at_3 32.921 type value precision_at_5 28.297 type value recall_at_1 5.015 type value recall_at_10 14.393 type value recall_at_100 28.893 type value recall_at_1000 60.18 type value recall_at_3 9.184000000000001 type value recall_at_5 11.39 task dataset metrics type Retrieval type name config split revision nq MTEB NQ default test None type value map_at_1 29.524 type value map_at_10 44.182 type value map_at_100 45.228 type value map_at_1000 45.265 type value map_at_3 39.978 type value map_at_5 42.482 type value mrr_at_1 33.256 type value mrr_at_10 46.661 type value mrr_at_100 47.47 type value mrr_at_1000 47.496 type value mrr_at_3 43.187999999999995 type value mrr_at_5 45.330999999999996 type value ndcg_at_1 33.227000000000004 type value ndcg_at_10 51.589 type value ndcg_at_100 56.043 type value ndcg_at_1000 56.937000000000005 type value ndcg_at_3 43.751 type value ndcg_at_5 47.937000000000005 type value precision_at_1 33.227000000000004 type value precision_at_10 8.556999999999999 type value precision_at_100 1.103 type value precision_at_1000 0.11900000000000001 type value precision_at_3 19.921 type value precision_at_5 14.396999999999998 type value recall_at_1 29.524 type value recall_at_10 71.615 type value recall_at_100 91.056 type value recall_at_1000 97.72800000000001 type value recall_at_3 51.451 type value recall_at_5 61.119 task dataset metrics type Retrieval type name config split revision quora MTEB QuoraRetrieval default test None type value map_at_1 69.596 type value map_at_10 83.281 type value map_at_100 83.952 type value map_at_1000 83.97200000000001 type value map_at_3 80.315 type value map_at_5 82.223 type value mrr_at_1 80.17 type value mrr_at_10 86.522 type value mrr_at_100 86.644 type value mrr_at_1000 86.64500000000001 type value mrr_at_3 85.438 type value mrr_at_5 86.21799999999999 type value ndcg_at_1 80.19 type value ndcg_at_10 87.19 type value ndcg_at_100 88.567 type value ndcg_at_1000 88.70400000000001 type value ndcg_at_3 84.17999999999999 type value ndcg_at_5 85.931 type value precision_at_1 80.19 type value precision_at_10 13.209000000000001 type value precision_at_100 1.518 type value precision_at_1000 0.157 type value precision_at_3 36.717 type value precision_at_5 24.248 type value recall_at_1 69.596 type value recall_at_10 94.533 type value recall_at_100 99.322 type value recall_at_1000 99.965 type value recall_at_3 85.911 type value recall_at_5 90.809 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering MTEB RedditClustering default test 24640382cdbf8abc73003fb0fa6d111a705499eb type value v_measure 49.27650627571912 task dataset metrics type Clustering type name config split revision mteb/reddit-clustering-p2p MTEB RedditClusteringP2P default test 282350215ef01743dc01b456c7f5241fa8937f16 type value v_measure 57.08550946534183 task dataset metrics type Retrieval type name config split revision scidocs MTEB SCIDOCS default test None type value map_at_1 4.568 type value map_at_10 10.862 type value map_at_100 12.757 type value map_at_1000 13.031 type value map_at_3 7.960000000000001 type value map_at_5 9.337 type value mrr_at_1 22.5 type value mrr_at_10 32.6 type value mrr_at_100 33.603 type value mrr_at_1000 33.672000000000004 type value mrr_at_3 29.299999999999997 type value mrr_at_5 31.25 type value ndcg_at_1 22.5 type value ndcg_at_10 18.605 type value ndcg_at_100 26.029999999999998 type value ndcg_at_1000 31.256 type value ndcg_at_3 17.873 type value ndcg_at_5 15.511 type value precision_at_1 22.5 type value precision_at_10 9.58 type value precision_at_100 2.033 type value precision_at_1000 0.33 type value precision_at_3 16.633 type value precision_at_5 13.54 type value recall_at_1 4.568 type value recall_at_10 19.402 type value recall_at_100 41.277 type value recall_at_1000 66.963 type value recall_at_3 10.112 type value recall_at_5 13.712 task dataset metrics type STS type name config split revision mteb/sickr-sts MTEB SICK-R default test a6ea5a8cab320b040a23452cc28066d9beae2cee type value cos_sim_pearson 83.31992291680787 type value cos_sim_spearman 76.7212346922664 type value euclidean_pearson 80.42189271706478 type value euclidean_spearman 76.7212342532493 type value manhattan_pearson 80.33171093031578 type value manhattan_spearman 76.63192883074694 task dataset metrics type STS type name config split revision mteb/sts12-sts MTEB STS12 default test a0d554a64d88156834ff5ae9920b964011b16384 type value cos_sim_pearson 83.16654278886763 type value cos_sim_spearman 73.66390263429565 type value euclidean_pearson 79.7485360086639 type value euclidean_spearman 73.66389870373436 type value manhattan_pearson 79.73652237443706 type value manhattan_spearman 73.65296117151647 task dataset metrics type STS type name config split revision mteb/sts13-sts MTEB STS13 default test 7e90230a92c190f1bf69ae9002b8cea547a64cca type value cos_sim_pearson 82.40389689929246 type value cos_sim_spearman 83.29727595993955 type value euclidean_pearson 82.23970587854079 type value euclidean_spearman 83.29727595993955 type value manhattan_pearson 82.18823600831897 type value manhattan_spearman 83.20746192209594 task dataset metrics type STS type name config split revision mteb/sts14-sts MTEB STS14 default test 6031580fec1f6af667f0bd2da0a551cf4f0b2375 type value cos_sim_pearson 81.73505246913413 type value cos_sim_spearman 79.1686548248754 type value euclidean_pearson 80.48889135993412 type value euclidean_spearman 79.16864112930354 type value manhattan_pearson 80.40720651057302 type value manhattan_spearman 79.0640155089286 task dataset metrics type STS type name config split revision mteb/sts15-sts MTEB STS15 default test ae752c7c21bf194d8b67fd573edf7ae58183cbe3 type value cos_sim_pearson 86.3953512879065 type value cos_sim_spearman 87.29947322714338 type value euclidean_pearson 86.59759438529645 type value euclidean_spearman 87.29947511092824 type value manhattan_pearson 86.52097806169155 type value manhattan_spearman 87.22987242146534 task dataset metrics type STS type name config split revision mteb/sts16-sts MTEB STS16 default test 4d8694f8f0e0100860b497b999b3dbed754a0513 type value cos_sim_pearson 82.48565753792056 type value cos_sim_spearman 83.6049720319893 type value euclidean_pearson 82.56452023172913 type value euclidean_spearman 83.60490168191697 type value manhattan_pearson 82.58079941137872 type value manhattan_spearman 83.60975807374051 task dataset metrics type STS type name config split revision mteb/sts17-crosslingual-sts MTEB STS17 (en-en) en-en test af5e6fb845001ecf41f4c1e033ce921939a2a68d type value cos_sim_pearson 88.18239976618212 type value cos_sim_spearman 88.23061724730616 type value euclidean_pearson 87.78482472776658 type value euclidean_spearman 88.23061724730616 type value manhattan_pearson 87.75059641730239 type value manhattan_spearman 88.22527413524622 task dataset metrics type STS type name config split revision mteb/sts22-crosslingual-sts MTEB STS22 (en) en test 6d1ba47164174a496b7fa5d3569dae26a6813b80 type value cos_sim_pearson 63.42816418706765 type value cos_sim_spearman 63.4569864520124 type value euclidean_pearson 64.35405409953853 type value euclidean_spearman 63.4569864520124 type value manhattan_pearson 63.96649236073056 type value manhattan_spearman 63.01448583722708 task dataset metrics type STS type name config split revision mteb/stsbenchmark-sts MTEB STSBenchmark default test b0fddb56ed78048fa8b90373c8a3cfc37b684831 type value cos_sim_pearson 83.41659638047614 type value cos_sim_spearman 84.03893866106175 type value euclidean_pearson 84.2251203953798 type value euclidean_spearman 84.03893866106175 type value manhattan_pearson 84.22733643205514 type value manhattan_spearman 84.06504411263612 task dataset metrics type Reranking type name config split revision mteb/scidocs-reranking MTEB SciDocsRR default test d3c5e1fc0b855ab6097bf1cda04dd73947d7caab type value map 79.75608022582414 type value mrr 94.0947732369301 task dataset metrics type Retrieval type name config split revision scifact MTEB SciFact default test None type value map_at_1 50.161 type value map_at_10 59.458999999999996 type value map_at_100 60.156 type value map_at_1000 60.194 type value map_at_3 56.45400000000001 type value map_at_5 58.165 type value mrr_at_1 53.333 type value mrr_at_10 61.050000000000004 type value mrr_at_100 61.586 type value mrr_at_1000 61.624 type value mrr_at_3 58.889 type value mrr_at_5 60.122 type value ndcg_at_1 53.333 type value ndcg_at_10 63.888999999999996 type value ndcg_at_100 66.963 type value ndcg_at_1000 68.062 type value ndcg_at_3 59.01 type value ndcg_at_5 61.373999999999995 type value precision_at_1 53.333 type value precision_at_10 8.633000000000001 type value precision_at_100 1.027 type value precision_at_1000 0.11199999999999999 type value precision_at_3 23.111 type value precision_at_5 15.467 type value recall_at_1 50.161 type value recall_at_10 75.922 type value recall_at_100 90.0 type value recall_at_1000 98.667 type value recall_at_3 62.90599999999999 type value recall_at_5 68.828 task dataset metrics type PairClassification type name config split revision mteb/sprintduplicatequestions-pairclassification MTEB SprintDuplicateQuestions default test d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 type value cos_sim_accuracy 99.81188118811882 type value cos_sim_ap 95.11619225962413 type value cos_sim_f1 90.35840484603736 type value cos_sim_precision 91.23343527013252 type value cos_sim_recall 89.5 type value dot_accuracy 99.81188118811882 type value dot_ap 95.11619225962413 type value dot_f1 90.35840484603736 type value dot_precision 91.23343527013252 type value dot_recall 89.5 type value euclidean_accuracy 99.81188118811882 type value euclidean_ap 95.11619225962413 type value euclidean_f1 90.35840484603736 type value euclidean_precision 91.23343527013252 type value euclidean_recall 89.5 type value manhattan_accuracy 99.80891089108911 type value manhattan_ap 95.07294266220966 type value manhattan_f1 90.21794221996959 type value manhattan_precision 91.46968139773895 type value manhattan_recall 89.0 type value max_accuracy 99.81188118811882 type value max_ap 95.11619225962413 type value max_f1 90.35840484603736 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering MTEB StackExchangeClustering default test 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 type value v_measure 55.3481874105239 task dataset metrics type Clustering type name config split revision mteb/stackexchange-clustering-p2p MTEB StackExchangeClusteringP2P default test 815ca46b2622cec33ccafc3735d572c266efdb44 type value v_measure 34.421291695525 task dataset metrics type Reranking type name config split revision mteb/stackoverflowdupquestions-reranking MTEB StackOverflowDupQuestions default test e185fbe320c72810689fc5848eb6114e1ef5ec69 type value map 49.98746633276634 type value mrr 50.63143249724133 task dataset metrics type Summarization type name config split revision mteb/summeval MTEB SummEval default test cda12ad7615edc362dbf25a00fdd61d3b1eaf93c type value cos_sim_pearson 31.009961979844036 type value cos_sim_spearman 30.558416108881044 type value dot_pearson 31.009964941134253 type value dot_spearman 30.545760761761393 task dataset metrics type Retrieval type name config split revision trec-covid MTEB TRECCOVID default test None type value map_at_1 0.207 type value map_at_10 1.6 type value map_at_100 8.594 type value map_at_1000 20.213 type value map_at_3 0.585 type value map_at_5 0.9039999999999999 type value mrr_at_1 78.0 type value mrr_at_10 87.4 type value mrr_at_100 87.4 type value mrr_at_1000 87.4 type value mrr_at_3 86.667 type value mrr_at_5 87.06700000000001 type value ndcg_at_1 73.0 type value ndcg_at_10 65.18 type value ndcg_at_100 49.631 type value ndcg_at_1000 43.498999999999995 type value ndcg_at_3 71.83800000000001 type value ndcg_at_5 69.271 type value precision_at_1 78.0 type value precision_at_10 69.19999999999999 type value precision_at_100 50.980000000000004 type value precision_at_1000 19.426 type value precision_at_3 77.333 type value precision_at_5 74.0 type value recall_at_1 0.207 type value recall_at_10 1.822 type value recall_at_100 11.849 type value recall_at_1000 40.492 type value recall_at_3 0.622 type value recall_at_5 0.9809999999999999 task dataset metrics type Retrieval type name config split revision webis-touche2020 MTEB Touche2020 default test None type value map_at_1 2.001 type value map_at_10 10.376000000000001 type value map_at_100 16.936999999999998 type value map_at_1000 18.615000000000002 type value map_at_3 5.335999999999999 type value map_at_5 7.374 type value mrr_at_1 20.408 type value mrr_at_10 38.29 type value mrr_at_100 39.33 type value mrr_at_1000 39.347 type value mrr_at_3 32.993 type value mrr_at_5 36.973 type value ndcg_at_1 17.347 type value ndcg_at_10 23.515 type value ndcg_at_100 37.457 type value ndcg_at_1000 49.439 type value ndcg_at_3 22.762999999999998 type value ndcg_at_5 22.622 type value precision_at_1 20.408 type value precision_at_10 22.448999999999998 type value precision_at_100 8.184 type value precision_at_1000 1.608 type value precision_at_3 25.85 type value precision_at_5 25.306 type value recall_at_1 2.001 type value recall_at_10 17.422 type value recall_at_100 51.532999999999994 type value recall_at_1000 87.466 type value recall_at_3 6.861000000000001 type value recall_at_5 10.502 task dataset metrics type Classification type name config split revision mteb/toxic_conversations_50k MTEB ToxicConversationsClassification default test d7c0de2777da35d6aae2200a62c6e0e5af397c4c type value accuracy 71.54419999999999 type value ap 14.372170450843907 type value f1 54.94420257390529 task dataset metrics type Classification type name config split revision mteb/tweet_sentiment_extraction MTEB TweetSentimentExtractionClassification default test d604517c81ca91fe16a244d1248fc021f9ecee7a type value accuracy 59.402942840973395 type value f1 59.4166538875571 task dataset metrics type Clustering type name config split revision mteb/twentynewsgroups-clustering MTEB TwentyNewsgroupsClustering default test 6125ec4e24fa026cec8a478383ee943acfbd5449 type value v_measure 41.569064336457906 task dataset metrics type PairClassification type name config split revision mteb/twittersemeval2015-pairclassification MTEB TwitterSemEval2015 default test 70970daeab8776df92f5ea462b6173c0b46fd2d1 type value cos_sim_accuracy 85.31322644096085 type value cos_sim_ap 72.14518894837381 type value cos_sim_f1 66.67489813557229 type value cos_sim_precision 62.65954977953121 type value cos_sim_recall 71.2401055408971 type value dot_accuracy 85.31322644096085 type value dot_ap 72.14521480685293 type value dot_f1 66.67489813557229 type value dot_precision 62.65954977953121 type value dot_recall 71.2401055408971 type value euclidean_accuracy 85.31322644096085 type value euclidean_ap 72.14520820485349 type value euclidean_f1 66.67489813557229 type value euclidean_precision 62.65954977953121 type value euclidean_recall 71.2401055408971 type value manhattan_accuracy 85.21785778148656 type value manhattan_ap 72.01177147657364 type value manhattan_f1 66.62594673833374 type value manhattan_precision 62.0336669699727 type value manhattan_recall 71.95250659630607 type value max_accuracy 85.31322644096085 type value max_ap 72.14521480685293 type value max_f1 66.67489813557229 task dataset metrics type PairClassification type name config split revision mteb/twitterurlcorpus-pairclassification MTEB TwitterURLCorpus default test 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf type value cos_sim_accuracy 89.12756626693057 type value cos_sim_ap 86.05430786440826 type value cos_sim_f1 78.27759692216631 type value cos_sim_precision 75.33466248931929 type value cos_sim_recall 81.45980905451185 type value dot_accuracy 89.12950673341872 type value dot_ap 86.05431161145492 type value dot_f1 78.27759692216631 type value dot_precision 75.33466248931929 type value dot_recall 81.45980905451185 type value euclidean_accuracy 89.12756626693057 type value euclidean_ap 86.05431303247397 type value euclidean_f1 78.27759692216631 type value euclidean_precision 75.33466248931929 type value euclidean_recall 81.45980905451185 type value manhattan_accuracy 89.04994760740482 type value manhattan_ap 86.00860610892074 type value manhattan_f1 78.1846776005392 type value manhattan_precision 76.10438839480975 type value manhattan_recall 80.3818909762858 type value max_accuracy 89.12950673341872 type value max_ap 86.05431303247397 type value max_f1 78.27759692216631

The text embedding set trained by Jina AI.

Quick Start

The easiest way to starting using jina-embeddings-v2-small-en is to use Jina AI's Embedding API.

Intended Usage & Model Info

jina-embeddings-v2-small-en is an English, monolingual embedding model supporting 8192 sequence length. It is based on a BERT architecture (JinaBERT) that supports the symmetric bidirectional variant of ALiBi to allow longer sequence length. The backbone jina-bert-v2-small-en is pretrained on the C4 dataset. The model is further trained on Jina AI's collection of more than 400 millions of sentence pairs and hard negatives. These pairs were obtained from various domains and were carefully selected through a thorough cleaning process.

The embedding model was trained using 512 sequence length, but extrapolates to 8k sequence length (or even longer) thanks to ALiBi. This makes our model useful for a range of use cases, especially when processing long documents is needed, including long document retrieval, semantic textual similarity, text reranking, recommendation, RAG and LLM-based generative search, etc.

This model has 33 million parameters, which enables lightning-fast and memory efficient inference, while still delivering impressive performance. Additionally, we provide the following embedding models:

• jina-embeddings-v2-small-en: 33 million parameters (you are here).
• jina-embeddings-v2-base-en: 137 million parameters.
• jina-embeddings-v2-base-zh: 161 million parameters Chinese-English Bilingual embeddings.
• jina-embeddings-v2-base-de: 161 million parameters German-English Bilingual embeddings.
• jina-embeddings-v2-base-es: Spanish-English Bilingual embeddings (soon).

Data & Parameters

Jina Embeddings V2 technical report

Usage

Please apply mean pooling when integrating the model.

Why mean pooling?

mean poooling takes all token embeddings from model output and averaging them at sentence/paragraph level. It has been proved to be the most effective way to produce high-quality sentence embeddings. We offer an encode function to deal with this.

However, if you would like to do it without using the default encode function:

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0]
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

sentences = ['How is the weather today?', 'What is the current weather like today?']

tokenizer = AutoTokenizer.from_pretrained('jinaai/jina-embeddings-v2-small-en')
model = AutoModel.from_pretrained('jinaai/jina-embeddings-v2-small-en', trust_remote_code=True)

encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

with torch.no_grad():
    model_output = model(**encoded_input)

embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
embeddings = F.normalize(embeddings, p=2, dim=1)

You can use Jina Embedding models directly from transformers package.

!pip install transformers
from transformers import AutoModel
from numpy.linalg import norm

cos_sim = lambda a,b: (a @ b.T) / (norm(a)*norm(b))
model = AutoModel.from_pretrained('jinaai/jina-embeddings-v2-small-en', trust_remote_code=True) # trust_remote_code is needed to use the encode method
embeddings = model.encode(['How is the weather today?', 'What is the current weather like today?'])
print(cos_sim(embeddings[0], embeddings[1]))

If you only want to handle shorter sequence, such as 2k, pass the max_length parameter to the encode function:

embeddings = model.encode(
    ['Very long ... document'],
    max_length=2048
)

The latest sentence-transformers also supports Jina embeddings:

!pip install -U sentence-transformers
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

model = SentenceTransformer(
    "jinaai/jina-embeddings-v2-small-en", # switch to en/zh for English or Chinese
    trust_remote_code=True
)

# control your input sequence length up to 8192
model.max_seq_length = 1024

embeddings = model.encode([
    'How is the weather today?',
    'What is the current weather like today?'
])
print(cos_sim(embeddings[0], embeddings[1]))

Alternatives to Using Transformers Package

1. Managed SaaS: Get started with a free key on Jina AI's Embedding API.
2. Private and high-performance deployment: Get started by picking from our suite of models and deploy them on AWS Sagemaker.

RAG Performance

According to the latest blog post from LLamaIndex,

In summary, to achieve the peak performance in both hit rate and MRR, the combination of OpenAI or JinaAI-Base embeddings with the CohereRerank/bge-reranker-large reranker stands out.

Plans

1. Bilingual embedding models supporting more European & Asian languages, including Spanish, French, Italian and Japanese.
2. Multimodal embedding models enable Multimodal RAG applications.
3. High-performt rerankers.

Trouble Shooting

Loading of Model Code failed

If you forgot to pass the trust_remote_code=True flag when calling AutoModel.from_pretrained or initializing the model via the SentenceTransformer class, you will receive an error that the model weights could not be initialized. This is caused by tranformers falling back to creating a default BERT model, instead of a jina-embedding model:

Some weights of the model checkpoint at jinaai/jina-embeddings-v2-base-en were not used when initializing BertModel: ['encoder.layer.2.mlp.layernorm.weight', 'encoder.layer.3.mlp.layernorm.weight', 'encoder.layer.10.mlp.wo.bias', 'encoder.layer.5.mlp.wo.bias', 'encoder.layer.2.mlp.layernorm.bias', 'encoder.layer.1.mlp.gated_layers.weight', 'encoder.layer.5.mlp.gated_layers.weight', 'encoder.layer.8.mlp.layernorm.bias', ...

Contact

Join our Discord community and chat with other community members about ideas.

Citation

If you find Jina Embeddings useful in your research, please cite the following paper:

@misc{günther2023jina,
      title={Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents}, 
      author={Michael Günther and Jackmin Ong and Isabelle Mohr and Alaeddine Abdessalem and Tanguy Abel and Mohammad Kalim Akram and Susana Guzman and Georgios Mastrapas and Saba Sturua and Bo Wang and Maximilian Werk and Nan Wang and Han Xiao},
      year={2023},
      eprint={2310.19923},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}