e5-R-mistral-7b/README.md

library_name, license, datasets, language, model-index, tags

library_name	license	datasets	language	model-index	tags
transformers	apache-2.0	BeastyZ/E5-R	en	name results e5-R-mistral-7b dataset metrics task config name revision split type default MTEB ArguAna None test mteb/arguana type value map_at_1 33.57 type value map_at_10 49.952000000000005 type value map_at_100 50.673 type value map_at_1000 50.674 type value map_at_3 44.915 type value map_at_5 47.876999999999995 type value mrr_at_1 34.211000000000006 type value mrr_at_10 50.19 type value mrr_at_100 50.905 type value mrr_at_1000 50.906 type value mrr_at_3 45.128 type value mrr_at_5 48.097 type value ndcg_at_1 33.57 type value ndcg_at_10 58.994 type value ndcg_at_100 61.806000000000004 type value ndcg_at_1000 61.824999999999996 type value ndcg_at_3 48.681000000000004 type value ndcg_at_5 54.001 type value precision_at_1 33.57 type value precision_at_10 8.784 type value precision_at_100 0.9950000000000001 type value precision_at_1000 0.1 type value precision_at_3 19.867 type value precision_at_5 14.495 type value recall_at_1 33.57 type value recall_at_10 87.83800000000001 type value recall_at_100 99.502 type value recall_at_1000 99.644 type value recall_at_3 59.602 type value recall_at_5 72.475 type value main_score 58.994 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackRetrieval None test mteb/cqadupstack type value map_at_1 24.75 type value map_at_10 34.025 type value map_at_100 35.126000000000005 type value map_at_1000 35.219 type value map_at_3 31.607000000000003 type value map_at_5 32.962 type value mrr_at_1 27.357 type value mrr_at_10 36.370999999999995 type value mrr_at_100 37.364000000000004 type value mrr_at_1000 37.423 type value mrr_at_3 34.288000000000004 type value mrr_at_5 35.434 type value ndcg_at_1 27.357 type value ndcg_at_10 46.593999999999997 type value ndcg_at_100 44.317 type value ndcg_at_1000 46.475 type value ndcg_at_3 34.473 type value ndcg_at_5 36.561 type value precision_at_1 27.357 type value precision_at_10 6.081 type value precision_at_100 0.9299999999999999 type value precision_at_1000 0.124 type value precision_at_3 14.911 type value precision_at_5 10.24 type value recall_at_1 24.75 type value recall_at_10 51.856 type value recall_at_100 76.44300000000001 type value recall_at_1000 92.078 type value recall_at_3 39.427 type value recall_at_5 44.639 type value main_score 46.593999999999997 type Retrieval dataset metrics task config name revision split type default MTEB ClimateFEVER None test mteb/climate-fever type value map_at_1 16.436 type value map_at_10 29.693 type value map_at_100 32.179 type value map_at_1000 32.353 type value map_at_3 24.556 type value map_at_5 27.105 type value mrr_at_1 37.524 type value mrr_at_10 51.475 type value mrr_at_100 52.107000000000006 type value mrr_at_1000 52.123 type value mrr_at_3 48.35 type value mrr_at_5 50.249 type value ndcg_at_1 37.524 type value ndcg_at_10 40.258 type value ndcg_at_100 48.364000000000004 type value ndcg_at_1000 51.031000000000006 type value ndcg_at_3 33.359 type value ndcg_at_5 35.573 type value precision_at_1 37.524 type value precision_at_10 12.886000000000001 type value precision_at_100 2.169 type value precision_at_1000 0.268 type value precision_at_3 25.624000000000002 type value precision_at_5 19.453 type value recall_at_1 16.436 type value recall_at_10 47.77 type value recall_at_100 74.762 type value recall_at_1000 89.316 type value recall_at_3 30.508000000000003 type value recall_at_5 37.346000000000004 type value main_score 40.258 type Retrieval dataset metrics task config name revision split type default MTEB DBPedia None test mteb/dbpedia type value map_at_1 10.147 type value map_at_10 24.631 type value map_at_100 35.657 type value map_at_1000 37.824999999999996 type value map_at_3 16.423 type value map_at_5 19.666 type value mrr_at_1 76.5 type value mrr_at_10 82.793 type value mrr_at_100 83.015 type value mrr_at_1000 83.021 type value mrr_at_3 81.75 type value mrr_at_5 82.375 type value ndcg_at_1 64.75 type value ndcg_at_10 51.031000000000006 type value ndcg_at_100 56.005 type value ndcg_at_1000 63.068000000000005 type value ndcg_at_3 54.571999999999996 type value ndcg_at_5 52.66499999999999 type value precision_at_1 76.5 type value precision_at_10 42.15 type value precision_at_100 13.22 type value precision_at_1000 2.5989999999999998 type value precision_at_3 58.416999999999994 type value precision_at_5 52.2 type value recall_at_1 10.147 type value recall_at_10 30.786 type value recall_at_100 62.873000000000005 type value recall_at_1000 85.358 type value recall_at_3 17.665 type value recall_at_5 22.088 type value main_score 51.031000000000006 type Retrieval dataset metrics task config name revision split type default MTEB FEVER None test mteb/fever type value map_at_1 78.52900000000001 type value map_at_10 87.24199999999999 type value map_at_100 87.446 type value map_at_1000 87.457 type value map_at_3 86.193 type value map_at_5 86.898 type value mrr_at_1 84.518 type value mrr_at_10 90.686 type value mrr_at_100 90.73 type value mrr_at_1000 90.731 type value mrr_at_3 90.227 type value mrr_at_5 90.575 type value ndcg_at_1 84.518 type value ndcg_at_10 90.324 type value ndcg_at_100 90.96300000000001 type value ndcg_at_1000 91.134 type value ndcg_at_3 88.937 type value ndcg_at_5 89.788 type value precision_at_1 84.518 type value precision_at_10 10.872 type value precision_at_100 1.1440000000000001 type value precision_at_1000 0.117 type value precision_at_3 34.108 type value precision_at_5 21.154999999999998 type value recall_at_1 78.52900000000001 type value recall_at_10 96.123 type value recall_at_100 98.503 type value recall_at_1000 99.518 type value recall_at_3 92.444 type value recall_at_5 94.609 type value main_score 90.324 type Retrieval dataset metrics task config name revision split type default MTEB FiQA2018 None test mteb/fiqa type value map_at_1 29.38 type value map_at_10 50.28 type value map_at_100 52.532999999999994 type value map_at_1000 52.641000000000005 type value map_at_3 43.556 type value map_at_5 47.617 type value mrr_at_1 56.79 type value mrr_at_10 65.666 type value mrr_at_100 66.211 type value mrr_at_1000 66.226 type value mrr_at_3 63.452 type value mrr_at_5 64.895 type value ndcg_at_1 56.79 type value ndcg_at_10 58.68 type value ndcg_at_100 65.22 type value ndcg_at_1000 66.645 type value ndcg_at_3 53.981 type value ndcg_at_5 55.95 type value precision_at_1 56.79 type value precision_at_10 16.311999999999998 type value precision_at_100 2.316 type value precision_at_1000 0.258 type value precision_at_3 36.214 type value precision_at_5 27.067999999999998 type value recall_at_1 29.38 type value recall_at_10 66.503 type value recall_at_100 89.885 type value recall_at_1000 97.954 type value recall_at_3 48.866 type value recall_at_5 57.60999999999999 type value main_score 58.68 type Retrieval dataset metrics task config name revision split type default MTEB HotpotQA None test mteb/hotpotqa type value map_at_1 42.134 type value map_at_10 73.412 type value map_at_100 74.144 type value map_at_1000 74.181 type value map_at_3 70.016 type value map_at_5 72.174 type value mrr_at_1 84.267 type value mrr_at_10 89.18599999999999 type value mrr_at_100 89.29599999999999 type value mrr_at_1000 89.298 type value mrr_at_3 88.616 type value mrr_at_5 88.957 type value ndcg_at_1 84.267 type value ndcg_at_10 80.164 type value ndcg_at_100 82.52199999999999 type value ndcg_at_1000 83.176 type value ndcg_at_3 75.616 type value ndcg_at_5 78.184 type value precision_at_1 84.267 type value precision_at_10 16.916 type value precision_at_100 1.872 type value precision_at_1000 0.196 type value precision_at_3 49.71 type value precision_at_5 31.854 type value recall_at_1 42.134 type value recall_at_10 84.578 type value recall_at_100 93.606 type value recall_at_1000 97.86 type value recall_at_3 74.564 type value recall_at_5 79.635 type value main_score 80.164 type Retrieval dataset metrics task config name revision split type default MTEB MSMARCO None dev mteb/msmarco type value map_at_1 22.276 type value map_at_10 35.493 type value map_at_100 36.656 type value map_at_1000 36.699 type value map_at_3 31.320999999999998 type value map_at_5 33.772999999999996 type value mrr_at_1 22.966 type value mrr_at_10 36.074 type value mrr_at_100 37.183 type value mrr_at_1000 37.219 type value mrr_at_3 31.984 type value mrr_at_5 34.419 type value ndcg_at_1 22.966 type value ndcg_at_10 42.895 type value ndcg_at_100 48.453 type value ndcg_at_1000 49.464999999999996 type value ndcg_at_3 34.410000000000004 type value ndcg_at_5 38.78 type value precision_at_1 22.966 type value precision_at_10 6.88 type value precision_at_100 0.966 type value precision_at_1000 0.105 type value precision_at_3 14.785 type value precision_at_5 11.074 type value recall_at_1 22.276 type value recall_at_10 65.756 type value recall_at_100 91.34100000000001 type value recall_at_1000 98.957 type value recall_at_3 42.67 type value recall_at_5 53.161 type value main_score 42.895 type Retrieval dataset metrics task config name revision split type default MTEB NFCorpus None test mteb/nfcorpus type value map_at_1 7.188999999999999 type value map_at_10 16.176 type value map_at_100 20.504 type value map_at_1000 22.203999999999997 type value map_at_3 11.766 type value map_at_5 13.655999999999999 type value mrr_at_1 55.418 type value mrr_at_10 62.791 type value mrr_at_100 63.339 type value mrr_at_1000 63.369 type value mrr_at_3 60.99099999999999 type value mrr_at_5 62.059 type value ndcg_at_1 53.715 type value ndcg_at_10 41.377 type value ndcg_at_100 37.999 type value ndcg_at_1000 46.726 type value ndcg_at_3 47.262 type value ndcg_at_5 44.708999999999996 type value precision_at_1 55.108000000000004 type value precision_at_10 30.154999999999998 type value precision_at_100 9.582 type value precision_at_1000 2.2720000000000002 type value precision_at_3 43.55 type value precision_at_5 38.204 type value recall_at_1 7.188999999999999 type value recall_at_10 20.655 type value recall_at_100 38.068000000000005 type value recall_at_1000 70.208 type value recall_at_3 12.601 type value recall_at_5 15.573999999999998 type value main_score 41.377 type Retrieval dataset metrics task config name revision split type default MTEB NQ None test mteb/nq type value map_at_1 46.017 type value map_at_10 62.910999999999994 type value map_at_100 63.526 type value map_at_1000 63.536 type value map_at_3 59.077999999999996 type value map_at_5 61.521 type value mrr_at_1 51.68000000000001 type value mrr_at_10 65.149 type value mrr_at_100 65.542 type value mrr_at_1000 65.55 type value mrr_at_3 62.49 type value mrr_at_5 64.178 type value ndcg_at_1 51.651 type value ndcg_at_10 69.83500000000001 type value ndcg_at_100 72.18 type value ndcg_at_1000 72.393 type value ndcg_at_3 63.168 type value ndcg_at_5 66.958 type value precision_at_1 51.651 type value precision_at_10 10.626 type value precision_at_100 1.195 type value precision_at_1000 0.121 type value precision_at_3 28.012999999999998 type value precision_at_5 19.09 type value recall_at_1 46.017 type value recall_at_10 88.345 type value recall_at_100 98.129 type value recall_at_1000 99.696 type value recall_at_3 71.531 type value recall_at_5 80.108 type value main_score 69.83500000000001 type Retrieval dataset metrics task config name revision split type default MTEB QuoraRetrieval None test mteb/quora type value map_at_1 72.473 type value map_at_10 86.72800000000001 type value map_at_100 87.323 type value map_at_1000 87.332 type value map_at_3 83.753 type value map_at_5 85.627 type value mrr_at_1 83.39 type value mrr_at_10 89.149 type value mrr_at_100 89.228 type value mrr_at_1000 89.229 type value mrr_at_3 88.335 type value mrr_at_5 88.895 type value ndcg_at_1 83.39 type value ndcg_at_10 90.109 type value ndcg_at_100 91.09 type value ndcg_at_1000 91.13900000000001 type value ndcg_at_3 87.483 type value ndcg_at_5 88.942 type value precision_at_1 83.39 type value precision_at_10 13.711 type value precision_at_100 1.549 type value precision_at_1000 0.157 type value precision_at_3 38.342999999999996 type value precision_at_5 25.188 type value recall_at_1 72.473 type value recall_at_10 96.57 type value recall_at_100 99.792 type value recall_at_1000 99.99900000000001 type value recall_at_3 88.979 type value recall_at_5 93.163 type value main_score 90.109 type Retrieval dataset metrics task config name revision split type default MTEB SCIDOCS None test mteb/scidocs type value map_at_1 4.598 type value map_at_10 11.405999999999999 type value map_at_100 13.447999999999999 type value map_at_1000 13.758999999999999 type value map_at_3 8.332 type value map_at_5 9.709 type value mrr_at_1 22.6 type value mrr_at_10 32.978 type value mrr_at_100 34.149 type value mrr_at_1000 34.213 type value mrr_at_3 29.7 type value mrr_at_5 31.485000000000003 type value ndcg_at_1 22.6 type value ndcg_at_10 19.259999999999998 type value ndcg_at_100 27.21 type value ndcg_at_1000 32.7 type value ndcg_at_3 18.445 type value ndcg_at_5 15.812000000000001 type value precision_at_1 22.6 type value precision_at_10 9.959999999999999 type value precision_at_100 2.139 type value precision_at_1000 0.345 type value precision_at_3 17.299999999999997 type value precision_at_5 13.719999999999999 type value recall_at_1 4.598 type value recall_at_10 20.186999999999998 type value recall_at_100 43.362 type value recall_at_1000 70.11800000000001 type value recall_at_3 10.543 type value recall_at_5 13.923 type value main_score 19.259999999999998 type Retrieval dataset metrics task config name revision split type default MTEB SciFact None test mteb/scifact type value map_at_1 65.467 type value map_at_10 74.935 type value map_at_100 75.395 type value map_at_1000 75.412 type value map_at_3 72.436 type value map_at_5 73.978 type value mrr_at_1 68.667 type value mrr_at_10 76.236 type value mrr_at_100 76.537 type value mrr_at_1000 76.55499999999999 type value mrr_at_3 74.722 type value mrr_at_5 75.639 type value ndcg_at_1 68.667 type value ndcg_at_10 78.92099999999999 type value ndcg_at_100 80.645 type value ndcg_at_1000 81.045 type value ndcg_at_3 75.19500000000001 type value ndcg_at_5 77.114 type value precision_at_1 68.667 type value precision_at_10 10.133000000000001 type value precision_at_100 1.0999999999999999 type value precision_at_1000 0.11299999999999999 type value precision_at_3 28.889 type value precision_at_5 18.8 type value recall_at_1 65.467 type value recall_at_10 89.517 type value recall_at_100 97 type value recall_at_1000 100 type value recall_at_3 79.72200000000001 type value recall_at_5 84.511 type value main_score 78.92099999999999 type Retrieval dataset metrics task config name revision split type default MTEB TRECCOVID None test mteb/trec-covid type value map_at_1 0.244 type value map_at_10 2.183 type value map_at_100 13.712 type value map_at_1000 33.147 type value map_at_3 0.7270000000000001 type value map_at_5 1.199 type value mrr_at_1 94 type value mrr_at_10 97 type value mrr_at_100 97 type value mrr_at_1000 97 type value mrr_at_3 97 type value mrr_at_5 97 type value ndcg_at_1 92 type value ndcg_at_10 84.399 type value ndcg_at_100 66.771 type value ndcg_at_1000 59.092 type value ndcg_at_3 89.173 type value ndcg_at_5 88.52600000000001 type value precision_at_1 94 type value precision_at_10 86.8 type value precision_at_100 68.24 type value precision_at_1000 26.003999999999998 type value precision_at_3 92.667 type value precision_at_5 92.4 type value recall_at_1 0.244 type value recall_at_10 2.302 type value recall_at_100 16.622 type value recall_at_1000 55.175 type value recall_at_3 0.748 type value recall_at_5 1.247 type value main_score 84.399 type Retrieval dataset metrics task config name revision split type default MTEB Touche2020 None test mteb/touche2020 type value map_at_1 2.707 type value map_at_10 10.917 type value map_at_100 16.308 type value map_at_1000 17.953 type value map_at_3 5.65 type value map_at_5 7.379 type value mrr_at_1 34.694 type value mrr_at_10 49.745 type value mrr_at_100 50.309000000000005 type value mrr_at_1000 50.32 type value mrr_at_3 44.897999999999996 type value mrr_at_5 48.061 type value ndcg_at_1 33.672999999999995 type value ndcg_at_10 26.894000000000002 type value ndcg_at_100 37.423 type value ndcg_at_1000 49.376999999999995 type value ndcg_at_3 30.456 type value ndcg_at_5 27.772000000000002 type value precision_at_1 34.694 type value precision_at_10 23.878 type value precision_at_100 7.489999999999999 type value precision_at_1000 1.555 type value precision_at_3 31.293 type value precision_at_5 26.939 type value recall_at_1 2.707 type value recall_at_10 18.104 type value recall_at_100 46.93 type value recall_at_1000 83.512 type value recall_at_3 6.622999999999999 type value recall_at_5 10.051 type value main_score 26.894000000000002 type Retrieval	mteb

Model Card for e5-R-mistral-7b

Model Description

e5-R-mistral-7b is a LLM retriever fine-tuned from mistralai/Mistral-7B-v0.1.

• Model type: CausalLM
• Repository: Welcome to our GitHub repository to obtain code
• Training dataset: Dataset used for fine-tuning e5-R-mistral-7b is available here.