---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
- mteb
model-index:
- name: mmlw-e5-large
  results:
  - task:
      type: Clustering
    dataset:
      type: PL-MTEB/8tags-clustering
      name: MTEB 8TagsClustering
      config: default
      split: test
      revision: None
    metrics:
    - type: v_measure
      value: 30.623921415441725
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/allegro-reviews
      name: MTEB AllegroReviews
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 37.683896620278325
    - type: f1
      value: 34.19193027014284
  - task:
      type: Retrieval
    dataset:
      type: arguana-pl
      name: MTEB ArguAna-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 38.407000000000004
    - type: map_at_10
      value: 55.147
    - type: map_at_100
      value: 55.757
    - type: map_at_1000
      value: 55.761
    - type: map_at_3
      value: 51.268
    - type: map_at_5
      value: 53.696999999999996
    - type: mrr_at_1
      value: 40.043
    - type: mrr_at_10
      value: 55.840999999999994
    - type: mrr_at_100
      value: 56.459
    - type: mrr_at_1000
      value: 56.462999999999994
    - type: mrr_at_3
      value: 52.074
    - type: mrr_at_5
      value: 54.364999999999995
    - type: ndcg_at_1
      value: 38.407000000000004
    - type: ndcg_at_10
      value: 63.248000000000005
    - type: ndcg_at_100
      value: 65.717
    - type: ndcg_at_1000
      value: 65.79
    - type: ndcg_at_3
      value: 55.403999999999996
    - type: ndcg_at_5
      value: 59.760000000000005
    - type: precision_at_1
      value: 38.407000000000004
    - type: precision_at_10
      value: 8.862
    - type: precision_at_100
      value: 0.991
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 22.451
    - type: precision_at_5
      value: 15.576
    - type: recall_at_1
      value: 38.407000000000004
    - type: recall_at_10
      value: 88.62
    - type: recall_at_100
      value: 99.075
    - type: recall_at_1000
      value: 99.57300000000001
    - type: recall_at_3
      value: 67.354
    - type: recall_at_5
      value: 77.881
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/cbd
      name: MTEB CBD
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 66.14999999999999
    - type: ap
      value: 21.69513674684204
    - type: f1
      value: 56.48142830893528
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/cdsce-pairclassification
      name: MTEB CDSC-E
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 89.4
    - type: cos_sim_ap
      value: 76.83228768203222
    - type: cos_sim_f1
      value: 65.3658536585366
    - type: cos_sim_precision
      value: 60.909090909090914
    - type: cos_sim_recall
      value: 70.52631578947368
    - type: dot_accuracy
      value: 84.1
    - type: dot_ap
      value: 57.26072201751864
    - type: dot_f1
      value: 62.75395033860045
    - type: dot_precision
      value: 54.9407114624506
    - type: dot_recall
      value: 73.15789473684211
    - type: euclidean_accuracy
      value: 89.4
    - type: euclidean_ap
      value: 76.59095263388942
    - type: euclidean_f1
      value: 65.21739130434783
    - type: euclidean_precision
      value: 60.26785714285714
    - type: euclidean_recall
      value: 71.05263157894737
    - type: manhattan_accuracy
      value: 89.4
    - type: manhattan_ap
      value: 76.58825999753456
    - type: manhattan_f1
      value: 64.72019464720195
    - type: manhattan_precision
      value: 60.18099547511312
    - type: manhattan_recall
      value: 70.0
    - type: max_accuracy
      value: 89.4
    - type: max_ap
      value: 76.83228768203222
    - type: max_f1
      value: 65.3658536585366
  - task:
      type: STS
    dataset:
      type: PL-MTEB/cdscr-sts
      name: MTEB CDSC-R
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 93.73949495291659
    - type: cos_sim_spearman
      value: 93.50397366192922
    - type: euclidean_pearson
      value: 92.47498888987636
    - type: euclidean_spearman
      value: 93.39315936230747
    - type: manhattan_pearson
      value: 92.47250250777654
    - type: manhattan_spearman
      value: 93.36739690549109
  - task:
      type: Retrieval
    dataset:
      type: dbpedia-pl
      name: MTEB DBPedia-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 8.434
    - type: map_at_10
      value: 18.424
    - type: map_at_100
      value: 26.428
    - type: map_at_1000
      value: 28.002
    - type: map_at_3
      value: 13.502
    - type: map_at_5
      value: 15.577
    - type: mrr_at_1
      value: 63.0
    - type: mrr_at_10
      value: 72.714
    - type: mrr_at_100
      value: 73.021
    - type: mrr_at_1000
      value: 73.028
    - type: mrr_at_3
      value: 70.75
    - type: mrr_at_5
      value: 72.3
    - type: ndcg_at_1
      value: 52.75
    - type: ndcg_at_10
      value: 39.839999999999996
    - type: ndcg_at_100
      value: 44.989000000000004
    - type: ndcg_at_1000
      value: 52.532999999999994
    - type: ndcg_at_3
      value: 45.198
    - type: ndcg_at_5
      value: 42.015
    - type: precision_at_1
      value: 63.0
    - type: precision_at_10
      value: 31.05
    - type: precision_at_100
      value: 10.26
    - type: precision_at_1000
      value: 1.9879999999999998
    - type: precision_at_3
      value: 48.25
    - type: precision_at_5
      value: 40.45
    - type: recall_at_1
      value: 8.434
    - type: recall_at_10
      value: 24.004
    - type: recall_at_100
      value: 51.428
    - type: recall_at_1000
      value: 75.712
    - type: recall_at_3
      value: 15.015
    - type: recall_at_5
      value: 18.282999999999998
  - task:
      type: Retrieval
    dataset:
      type: fiqa-pl
      name: MTEB FiQA-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 19.088
    - type: map_at_10
      value: 31.818
    - type: map_at_100
      value: 33.689
    - type: map_at_1000
      value: 33.86
    - type: map_at_3
      value: 27.399
    - type: map_at_5
      value: 29.945
    - type: mrr_at_1
      value: 38.117000000000004
    - type: mrr_at_10
      value: 47.668
    - type: mrr_at_100
      value: 48.428
    - type: mrr_at_1000
      value: 48.475
    - type: mrr_at_3
      value: 45.242
    - type: mrr_at_5
      value: 46.716
    - type: ndcg_at_1
      value: 38.272
    - type: ndcg_at_10
      value: 39.903
    - type: ndcg_at_100
      value: 46.661
    - type: ndcg_at_1000
      value: 49.625
    - type: ndcg_at_3
      value: 35.921
    - type: ndcg_at_5
      value: 37.558
    - type: precision_at_1
      value: 38.272
    - type: precision_at_10
      value: 11.358
    - type: precision_at_100
      value: 1.8190000000000002
    - type: precision_at_1000
      value: 0.23500000000000001
    - type: precision_at_3
      value: 24.434
    - type: precision_at_5
      value: 18.395
    - type: recall_at_1
      value: 19.088
    - type: recall_at_10
      value: 47.355999999999995
    - type: recall_at_100
      value: 72.451
    - type: recall_at_1000
      value: 90.257
    - type: recall_at_3
      value: 32.931
    - type: recall_at_5
      value: 39.878
  - task:
      type: Retrieval
    dataset:
      type: hotpotqa-pl
      name: MTEB HotpotQA-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 39.095
    - type: map_at_10
      value: 62.529
    - type: map_at_100
      value: 63.425
    - type: map_at_1000
      value: 63.483000000000004
    - type: map_at_3
      value: 58.887
    - type: map_at_5
      value: 61.18599999999999
    - type: mrr_at_1
      value: 78.123
    - type: mrr_at_10
      value: 84.231
    - type: mrr_at_100
      value: 84.408
    - type: mrr_at_1000
      value: 84.414
    - type: mrr_at_3
      value: 83.286
    - type: mrr_at_5
      value: 83.94
    - type: ndcg_at_1
      value: 78.19
    - type: ndcg_at_10
      value: 70.938
    - type: ndcg_at_100
      value: 73.992
    - type: ndcg_at_1000
      value: 75.1
    - type: ndcg_at_3
      value: 65.863
    - type: ndcg_at_5
      value: 68.755
    - type: precision_at_1
      value: 78.19
    - type: precision_at_10
      value: 14.949000000000002
    - type: precision_at_100
      value: 1.733
    - type: precision_at_1000
      value: 0.188
    - type: precision_at_3
      value: 42.381
    - type: precision_at_5
      value: 27.711000000000002
    - type: recall_at_1
      value: 39.095
    - type: recall_at_10
      value: 74.747
    - type: recall_at_100
      value: 86.631
    - type: recall_at_1000
      value: 93.923
    - type: recall_at_3
      value: 63.571999999999996
    - type: recall_at_5
      value: 69.27799999999999
  - task:
      type: Retrieval
    dataset:
      type: msmarco-pl
      name: MTEB MSMARCO-PL
      config: default
      split: validation
      revision: None
    metrics:
    - type: map_at_1
      value: 19.439999999999998
    - type: map_at_10
      value: 30.264000000000003
    - type: map_at_100
      value: 31.438
    - type: map_at_1000
      value: 31.495
    - type: map_at_3
      value: 26.735
    - type: map_at_5
      value: 28.716
    - type: mrr_at_1
      value: 19.914
    - type: mrr_at_10
      value: 30.753999999999998
    - type: mrr_at_100
      value: 31.877
    - type: mrr_at_1000
      value: 31.929000000000002
    - type: mrr_at_3
      value: 27.299
    - type: mrr_at_5
      value: 29.254
    - type: ndcg_at_1
      value: 20.014000000000003
    - type: ndcg_at_10
      value: 36.472
    - type: ndcg_at_100
      value: 42.231
    - type: ndcg_at_1000
      value: 43.744
    - type: ndcg_at_3
      value: 29.268
    - type: ndcg_at_5
      value: 32.79
    - type: precision_at_1
      value: 20.014000000000003
    - type: precision_at_10
      value: 5.814
    - type: precision_at_100
      value: 0.8710000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 12.426
    - type: precision_at_5
      value: 9.238
    - type: recall_at_1
      value: 19.439999999999998
    - type: recall_at_10
      value: 55.535000000000004
    - type: recall_at_100
      value: 82.44399999999999
    - type: recall_at_1000
      value: 94.217
    - type: recall_at_3
      value: 35.963
    - type: recall_at_5
      value: 44.367000000000004
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (pl)
      config: pl
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 72.01412239408205
    - type: f1
      value: 70.04544187503352
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (pl)
      config: pl
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 75.26899798251513
    - type: f1
      value: 75.55876166863844
  - task:
      type: Retrieval
    dataset:
      type: nfcorpus-pl
      name: MTEB NFCorpus-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 5.772
    - type: map_at_10
      value: 12.708
    - type: map_at_100
      value: 16.194
    - type: map_at_1000
      value: 17.630000000000003
    - type: map_at_3
      value: 9.34
    - type: map_at_5
      value: 10.741
    - type: mrr_at_1
      value: 43.344
    - type: mrr_at_10
      value: 53.429
    - type: mrr_at_100
      value: 53.88699999999999
    - type: mrr_at_1000
      value: 53.925
    - type: mrr_at_3
      value: 51.342
    - type: mrr_at_5
      value: 52.456
    - type: ndcg_at_1
      value: 41.641
    - type: ndcg_at_10
      value: 34.028000000000006
    - type: ndcg_at_100
      value: 31.613000000000003
    - type: ndcg_at_1000
      value: 40.428
    - type: ndcg_at_3
      value: 38.991
    - type: ndcg_at_5
      value: 36.704
    - type: precision_at_1
      value: 43.034
    - type: precision_at_10
      value: 25.324999999999996
    - type: precision_at_100
      value: 7.889
    - type: precision_at_1000
      value: 2.069
    - type: precision_at_3
      value: 36.739
    - type: precision_at_5
      value: 32.074000000000005
    - type: recall_at_1
      value: 5.772
    - type: recall_at_10
      value: 16.827
    - type: recall_at_100
      value: 32.346000000000004
    - type: recall_at_1000
      value: 62.739
    - type: recall_at_3
      value: 10.56
    - type: recall_at_5
      value: 12.655
  - task:
      type: Retrieval
    dataset:
      type: nq-pl
      name: MTEB NQ-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 26.101000000000003
    - type: map_at_10
      value: 39.912
    - type: map_at_100
      value: 41.037
    - type: map_at_1000
      value: 41.077000000000005
    - type: map_at_3
      value: 35.691
    - type: map_at_5
      value: 38.155
    - type: mrr_at_1
      value: 29.403000000000002
    - type: mrr_at_10
      value: 42.376999999999995
    - type: mrr_at_100
      value: 43.248999999999995
    - type: mrr_at_1000
      value: 43.277
    - type: mrr_at_3
      value: 38.794000000000004
    - type: mrr_at_5
      value: 40.933
    - type: ndcg_at_1
      value: 29.519000000000002
    - type: ndcg_at_10
      value: 47.33
    - type: ndcg_at_100
      value: 52.171
    - type: ndcg_at_1000
      value: 53.125
    - type: ndcg_at_3
      value: 39.316
    - type: ndcg_at_5
      value: 43.457
    - type: precision_at_1
      value: 29.519000000000002
    - type: precision_at_10
      value: 8.03
    - type: precision_at_100
      value: 1.075
    - type: precision_at_1000
      value: 0.117
    - type: precision_at_3
      value: 18.009
    - type: precision_at_5
      value: 13.221
    - type: recall_at_1
      value: 26.101000000000003
    - type: recall_at_10
      value: 67.50399999999999
    - type: recall_at_100
      value: 88.64699999999999
    - type: recall_at_1000
      value: 95.771
    - type: recall_at_3
      value: 46.669
    - type: recall_at_5
      value: 56.24
  - task:
      type: Classification
    dataset:
      type: laugustyniak/abusive-clauses-pl
      name: MTEB PAC
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 63.76773819866782
    - type: ap
      value: 74.87896817642536
    - type: f1
      value: 61.420506092721425
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/ppc-pairclassification
      name: MTEB PPC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 82.1
    - type: cos_sim_ap
      value: 91.09417013497443
    - type: cos_sim_f1
      value: 84.78437754271766
    - type: cos_sim_precision
      value: 83.36
    - type: cos_sim_recall
      value: 86.25827814569537
    - type: dot_accuracy
      value: 75.9
    - type: dot_ap
      value: 86.82680649789796
    - type: dot_f1
      value: 80.5379746835443
    - type: dot_precision
      value: 77.12121212121212
    - type: dot_recall
      value: 84.27152317880795
    - type: euclidean_accuracy
      value: 81.6
    - type: euclidean_ap
      value: 90.81248760600693
    - type: euclidean_f1
      value: 84.35374149659863
    - type: euclidean_precision
      value: 86.7132867132867
    - type: euclidean_recall
      value: 82.11920529801324
    - type: manhattan_accuracy
      value: 81.6
    - type: manhattan_ap
      value: 90.81272803548767
    - type: manhattan_f1
      value: 84.33530906011855
    - type: manhattan_precision
      value: 86.30849220103987
    - type: manhattan_recall
      value: 82.45033112582782
    - type: max_accuracy
      value: 82.1
    - type: max_ap
      value: 91.09417013497443
    - type: max_f1
      value: 84.78437754271766
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/psc-pairclassification
      name: MTEB PSC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 98.05194805194806
    - type: cos_sim_ap
      value: 99.52709687103496
    - type: cos_sim_f1
      value: 96.83257918552036
    - type: cos_sim_precision
      value: 95.82089552238806
    - type: cos_sim_recall
      value: 97.86585365853658
    - type: dot_accuracy
      value: 92.30055658627087
    - type: dot_ap
      value: 94.12759311032353
    - type: dot_f1
      value: 87.00906344410878
    - type: dot_precision
      value: 86.22754491017965
    - type: dot_recall
      value: 87.8048780487805
    - type: euclidean_accuracy
      value: 98.05194805194806
    - type: euclidean_ap
      value: 99.49402675624125
    - type: euclidean_f1
      value: 96.8133535660091
    - type: euclidean_precision
      value: 96.37462235649546
    - type: euclidean_recall
      value: 97.2560975609756
    - type: manhattan_accuracy
      value: 98.05194805194806
    - type: manhattan_ap
      value: 99.50120505935962
    - type: manhattan_f1
      value: 96.8133535660091
    - type: manhattan_precision
      value: 96.37462235649546
    - type: manhattan_recall
      value: 97.2560975609756
    - type: max_accuracy
      value: 98.05194805194806
    - type: max_ap
      value: 99.52709687103496
    - type: max_f1
      value: 96.83257918552036
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/polemo2_in
      name: MTEB PolEmo2.0-IN
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 69.45983379501385
    - type: f1
      value: 68.60917948426784
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/polemo2_out
      name: MTEB PolEmo2.0-OUT
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 43.13765182186235
    - type: f1
      value: 36.15557441785656
  - task:
      type: Retrieval
    dataset:
      type: quora-pl
      name: MTEB Quora-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 67.448
    - type: map_at_10
      value: 81.566
    - type: map_at_100
      value: 82.284
    - type: map_at_1000
      value: 82.301
    - type: map_at_3
      value: 78.425
    - type: map_at_5
      value: 80.43400000000001
    - type: mrr_at_1
      value: 77.61
    - type: mrr_at_10
      value: 84.467
    - type: mrr_at_100
      value: 84.63199999999999
    - type: mrr_at_1000
      value: 84.634
    - type: mrr_at_3
      value: 83.288
    - type: mrr_at_5
      value: 84.095
    - type: ndcg_at_1
      value: 77.66
    - type: ndcg_at_10
      value: 85.63199999999999
    - type: ndcg_at_100
      value: 87.166
    - type: ndcg_at_1000
      value: 87.306
    - type: ndcg_at_3
      value: 82.32300000000001
    - type: ndcg_at_5
      value: 84.22
    - type: precision_at_1
      value: 77.66
    - type: precision_at_10
      value: 13.136000000000001
    - type: precision_at_100
      value: 1.522
    - type: precision_at_1000
      value: 0.156
    - type: precision_at_3
      value: 36.153
    - type: precision_at_5
      value: 23.982
    - type: recall_at_1
      value: 67.448
    - type: recall_at_10
      value: 93.83200000000001
    - type: recall_at_100
      value: 99.212
    - type: recall_at_1000
      value: 99.94
    - type: recall_at_3
      value: 84.539
    - type: recall_at_5
      value: 89.71000000000001
  - task:
      type: Retrieval
    dataset:
      type: scidocs-pl
      name: MTEB SCIDOCS-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 4.393
    - type: map_at_10
      value: 11.472
    - type: map_at_100
      value: 13.584999999999999
    - type: map_at_1000
      value: 13.918
    - type: map_at_3
      value: 8.212
    - type: map_at_5
      value: 9.864
    - type: mrr_at_1
      value: 21.7
    - type: mrr_at_10
      value: 32.268
    - type: mrr_at_100
      value: 33.495000000000005
    - type: mrr_at_1000
      value: 33.548
    - type: mrr_at_3
      value: 29.15
    - type: mrr_at_5
      value: 30.91
    - type: ndcg_at_1
      value: 21.6
    - type: ndcg_at_10
      value: 19.126
    - type: ndcg_at_100
      value: 27.496
    - type: ndcg_at_1000
      value: 33.274
    - type: ndcg_at_3
      value: 18.196
    - type: ndcg_at_5
      value: 15.945
    - type: precision_at_1
      value: 21.6
    - type: precision_at_10
      value: 9.94
    - type: precision_at_100
      value: 2.1999999999999997
    - type: precision_at_1000
      value: 0.359
    - type: precision_at_3
      value: 17.2
    - type: precision_at_5
      value: 14.12
    - type: recall_at_1
      value: 4.393
    - type: recall_at_10
      value: 20.166999999999998
    - type: recall_at_100
      value: 44.678000000000004
    - type: recall_at_1000
      value: 72.868
    - type: recall_at_3
      value: 10.473
    - type: recall_at_5
      value: 14.313
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/sicke-pl-pairclassification
      name: MTEB SICK-E-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 82.65389319200979
    - type: cos_sim_ap
      value: 76.13749398520014
    - type: cos_sim_f1
      value: 66.64355062413314
    - type: cos_sim_precision
      value: 64.93243243243244
    - type: cos_sim_recall
      value: 68.44729344729345
    - type: dot_accuracy
      value: 76.0905014268243
    - type: dot_ap
      value: 58.058968583382494
    - type: dot_f1
      value: 61.181080324657145
    - type: dot_precision
      value: 50.391885661595204
    - type: dot_recall
      value: 77.84900284900284
    - type: euclidean_accuracy
      value: 82.61312678353036
    - type: euclidean_ap
      value: 76.10290283033221
    - type: euclidean_f1
      value: 66.50782845473111
    - type: euclidean_precision
      value: 63.6897001303781
    - type: euclidean_recall
      value: 69.58689458689459
    - type: manhattan_accuracy
      value: 82.6742763962495
    - type: manhattan_ap
      value: 76.12712309700966
    - type: manhattan_f1
      value: 66.59700452803902
    - type: manhattan_precision
      value: 65.16700749829583
    - type: manhattan_recall
      value: 68.09116809116809
    - type: max_accuracy
      value: 82.6742763962495
    - type: max_ap
      value: 76.13749398520014
    - type: max_f1
      value: 66.64355062413314
  - task:
      type: STS
    dataset:
      type: PL-MTEB/sickr-pl-sts
      name: MTEB SICK-R-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 81.23898481255246
    - type: cos_sim_spearman
      value: 76.0416957474899
    - type: euclidean_pearson
      value: 78.96475496102107
    - type: euclidean_spearman
      value: 76.07208683063504
    - type: manhattan_pearson
      value: 78.92666424673251
    - type: manhattan_spearman
      value: 76.04968227583831
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (pl)
      config: pl
      split: test
      revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
    metrics:
    - type: cos_sim_pearson
      value: 39.13987124398541
    - type: cos_sim_spearman
      value: 40.40194528288759
    - type: euclidean_pearson
      value: 29.14566247168167
    - type: euclidean_spearman
      value: 39.97389932591777
    - type: manhattan_pearson
      value: 29.172993134388935
    - type: manhattan_spearman
      value: 39.85681935287037
  - task:
      type: Retrieval
    dataset:
      type: scifact-pl
      name: MTEB SciFact-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 57.260999999999996
    - type: map_at_10
      value: 66.92399999999999
    - type: map_at_100
      value: 67.443
    - type: map_at_1000
      value: 67.47800000000001
    - type: map_at_3
      value: 64.859
    - type: map_at_5
      value: 65.71900000000001
    - type: mrr_at_1
      value: 60.333000000000006
    - type: mrr_at_10
      value: 67.95400000000001
    - type: mrr_at_100
      value: 68.42
    - type: mrr_at_1000
      value: 68.45
    - type: mrr_at_3
      value: 66.444
    - type: mrr_at_5
      value: 67.128
    - type: ndcg_at_1
      value: 60.333000000000006
    - type: ndcg_at_10
      value: 71.209
    - type: ndcg_at_100
      value: 73.37
    - type: ndcg_at_1000
      value: 74.287
    - type: ndcg_at_3
      value: 67.66799999999999
    - type: ndcg_at_5
      value: 68.644
    - type: precision_at_1
      value: 60.333000000000006
    - type: precision_at_10
      value: 9.467
    - type: precision_at_100
      value: 1.053
    - type: precision_at_1000
      value: 0.11299999999999999
    - type: precision_at_3
      value: 26.778000000000002
    - type: precision_at_5
      value: 16.933
    - type: recall_at_1
      value: 57.260999999999996
    - type: recall_at_10
      value: 83.256
    - type: recall_at_100
      value: 92.767
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_3
      value: 72.933
    - type: recall_at_5
      value: 75.744
  - task:
      type: Retrieval
    dataset:
      type: trec-covid-pl
      name: MTEB TRECCOVID-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.22
    - type: map_at_10
      value: 1.693
    - type: map_at_100
      value: 9.281
    - type: map_at_1000
      value: 21.462999999999997
    - type: map_at_3
      value: 0.609
    - type: map_at_5
      value: 0.9570000000000001
    - type: mrr_at_1
      value: 80.0
    - type: mrr_at_10
      value: 88.73299999999999
    - type: mrr_at_100
      value: 88.73299999999999
    - type: mrr_at_1000
      value: 88.73299999999999
    - type: mrr_at_3
      value: 88.333
    - type: mrr_at_5
      value: 88.73299999999999
    - type: ndcg_at_1
      value: 79.0
    - type: ndcg_at_10
      value: 71.177
    - type: ndcg_at_100
      value: 52.479
    - type: ndcg_at_1000
      value: 45.333
    - type: ndcg_at_3
      value: 77.48
    - type: ndcg_at_5
      value: 76.137
    - type: precision_at_1
      value: 82.0
    - type: precision_at_10
      value: 74.0
    - type: precision_at_100
      value: 53.68000000000001
    - type: precision_at_1000
      value: 19.954
    - type: precision_at_3
      value: 80.667
    - type: precision_at_5
      value: 80.80000000000001
    - type: recall_at_1
      value: 0.22
    - type: recall_at_10
      value: 1.934
    - type: recall_at_100
      value: 12.728
    - type: recall_at_1000
      value: 41.869
    - type: recall_at_3
      value: 0.637
    - type: recall_at_5
      value: 1.042
language: pl
license: apache-2.0
widget:
- source_sentence: "query: Jak dożyć 100 lat?"
  sentences:
    - "passage: Trzeba zdrowo się odżywiać i uprawiać sport."
    - "passage: Trzeba pić alkohol, imprezować i jeździć szybkimi autami."
    - "passage: Gdy trwała kampania politycy zapewniali, że rozprawią się z zakazem niedzielnego handlu."

---

<h1 align="center">MMLW-e5-large</h1>

MMLW (muszę mieć lepszą wiadomość) are neural text encoders for Polish.
This is a distilled model that can be used to generate embeddings applicable to many tasks such as semantic similarity, clustering, information retrieval. The model can also serve as a base for further fine-tuning. 
It transforms texts to 1024 dimensional vectors. 
The model was initialized with multilingual E5 checkpoint, and then trained with [multilingual knowledge distillation method](https://aclanthology.org/2020.emnlp-main.365/) on a diverse corpus of 60 million Polish-English text pairs. We utilised [English FlagEmbeddings (BGE)](https://huggingface.co/BAAI/bge-base-en) as teacher models for distillation. 

## Usage (Sentence-Transformers)

⚠️ Our embedding models require the use of specific prefixes and suffixes when encoding texts. For this model, queries should be prefixed with **"query: "** and passages with **"passage: "** ⚠️

You can use the model like this with [sentence-transformers](https://www.SBERT.net):

```python
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

query_prefix = "query: "
answer_prefix = "passage: "
queries = [query_prefix + "Jak dożyć 100 lat?"]
answers = [
    answer_prefix + "Trzeba zdrowo się odżywiać i uprawiać sport.",
    answer_prefix + "Trzeba pić alkohol, imprezować i jeździć szybkimi autami.",
    answer_prefix + "Gdy trwała kampania politycy zapewniali, że rozprawią się z zakazem niedzielnego handlu."
]
model = SentenceTransformer("sdadas/mmlw-e5-large")
queries_emb = model.encode(queries, convert_to_tensor=True, show_progress_bar=False)
answers_emb = model.encode(answers, convert_to_tensor=True, show_progress_bar=False)

best_answer = cos_sim(queries_emb, answers_emb).argmax().item()
print(answers[best_answer])
# Trzeba zdrowo się odżywiać i uprawiać sport.
```

## Evaluation Results

- The model achieves an **Average Score** of **61.17** on the Polish Massive Text Embedding Benchmark (MTEB). See [MTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) for detailed results.
- The model achieves **NDCG@10** of **56.09** on the Polish Information Retrieval Benchmark. See [PIRB Leaderboard](https://huggingface.co/spaces/sdadas/pirb) for detailed results.

## Acknowledgements
This model was trained with the A100 GPU cluster support delivered by the Gdansk University of Technology within the TASK center initiative.

## Citation

```bibtex
@article{dadas2024pirb,
  title={{PIRB}: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methods}, 
  author={Sławomir Dadas and Michał Perełkiewicz and Rafał Poświata},
  year={2024},
  eprint={2402.13350},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
```