Hugging Face's logo

jinaai
/
jina-embeddings-v3

Feature Extraction
Transformers
PyTorch
ONNX
Safetensors
sentence-transformers
sentence-similarity
mteb
custom_code
Eval Results
🇪🇺 Region: EU
Model card Files Community
89
bwang0911 commited on
Commit
a408c93
•
1 Parent(s): 58c59c2

chore: patch multi-class classification rus scores

Browse files
Files changed (1) hide show
  1. README.md +269 -0
README.md CHANGED
@@ -131,6 +131,275 @@ model-index:
131
  value: 43.4726782647566
132
  task:
133
  type: STS
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
134
  - dataset:
135
  config: default
136
  name: MTEB ATEC (default)
 
131
  value: 43.4726782647566
132
  task:
133
  type: STS
134
+ - dataset:
135
+ config: default
136
+ name: MTEB CEDRClassification (default)
137
+ revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
138
+ split: test
139
+ type: ai-forever/cedr-classification
140
+ metrics:
141
+ - type: accuracy
142
+ value: 47.30605738575983
143
+ - type: f1
144
+ value: 41.26091043925065
145
+ - type: lrap
146
+ value: 72.89452709883206
147
+ - type: main_score
148
+ value: 47.30605738575983
149
+ task:
150
+ type: MultilabelClassification
151
+ - dataset:
152
+ config: ru
153
+ name: MTEB MIRACLReranking (ru)
154
+ revision: 6d1962c527217f8927fca80f890f14f36b2802af
155
+ split: dev
156
+ type: miracl/mmteb-miracl-reranking
157
+ metrics:
158
+ - type: MAP@1(MIRACL)
159
+ value: 20.721999999999998
160
+ - type: MAP@10(MIRACL)
161
+ value: 33.900999999999996
162
+ - type: MAP@100(MIRACL)
163
+ value: 36.813
164
+ - type: MAP@1000(MIRACL)
165
+ value: 36.813
166
+ - type: MAP@20(MIRACL)
167
+ value: 35.684
168
+ - type: MAP@3(MIRACL)
169
+ value: 28.141
170
+ - type: MAP@5(MIRACL)
171
+ value: 31.075000000000003
172
+ - type: NDCG@1(MIRACL)
173
+ value: 32.799
174
+ - type: NDCG@10(MIRACL)
175
+ value: 42.065000000000005
176
+ - type: NDCG@100(MIRACL)
177
+ value: 49.730999999999995
178
+ - type: NDCG@1000(MIRACL)
179
+ value: 49.730999999999995
180
+ - type: NDCG@20(MIRACL)
181
+ value: 46.0
182
+ - type: NDCG@3(MIRACL)
183
+ value: 34.481
184
+ - type: NDCG@5(MIRACL)
185
+ value: 37.452999999999996
186
+ - type: P@1(MIRACL)
187
+ value: 32.799
188
+ - type: P@10(MIRACL)
189
+ value: 11.668000000000001
190
+ - type: P@100(MIRACL)
191
+ value: 1.9529999999999998
192
+ - type: P@1000(MIRACL)
193
+ value: 0.19499999999999998
194
+ - type: P@20(MIRACL)
195
+ value: 7.51
196
+ - type: P@3(MIRACL)
197
+ value: 20.823
198
+ - type: P@5(MIRACL)
199
+ value: 16.728
200
+ - type: Recall@1(MIRACL)
201
+ value: 20.721999999999998
202
+ - type: Recall@10(MIRACL)
203
+ value: 54.762
204
+ - type: Recall@100(MIRACL)
205
+ value: 79.952
206
+ - type: Recall@1000(MIRACL)
207
+ value: 79.952
208
+ - type: Recall@20(MIRACL)
209
+ value: 66.26100000000001
210
+ - type: Recall@3(MIRACL)
211
+ value: 34.410000000000004
212
+ - type: Recall@5(MIRACL)
213
+ value: 42.659000000000006
214
+ - type: main_score
215
+ value: 42.065000000000005
216
+ - type: nAUC_MAP@1000_diff1(MIRACL)
217
+ value: 14.33534992502818
218
+ - type: nAUC_MAP@1000_max(MIRACL)
219
+ value: 12.367998764646115
220
+ - type: nAUC_MAP@1000_std(MIRACL)
221
+ value: 4.569686002935006
222
+ - type: nAUC_MAP@100_diff1(MIRACL)
223
+ value: 14.33534992502818
224
+ - type: nAUC_MAP@100_max(MIRACL)
225
+ value: 12.367998764646115
226
+ - type: nAUC_MAP@100_std(MIRACL)
227
+ value: 4.569686002935006
228
+ - type: nAUC_MAP@10_diff1(MIRACL)
229
+ value: 16.920323975680027
230
+ - type: nAUC_MAP@10_max(MIRACL)
231
+ value: 9.327171297204082
232
+ - type: nAUC_MAP@10_std(MIRACL)
233
+ value: 3.2039133783079015
234
+ - type: nAUC_MAP@1_diff1(MIRACL)
235
+ value: 28.698973487482206
236
+ - type: nAUC_MAP@1_max(MIRACL)
237
+ value: 2.9217687660885034
238
+ - type: nAUC_MAP@1_std(MIRACL)
239
+ value: -1.1247408800976524
240
+ - type: nAUC_MAP@20_diff1(MIRACL)
241
+ value: 15.359083081640476
242
+ - type: nAUC_MAP@20_max(MIRACL)
243
+ value: 11.310494233946345
244
+ - type: nAUC_MAP@20_std(MIRACL)
245
+ value: 4.4171898386022885
246
+ - type: nAUC_MAP@3_diff1(MIRACL)
247
+ value: 22.27430591851617
248
+ - type: nAUC_MAP@3_max(MIRACL)
249
+ value: 6.407438291284658
250
+ - type: nAUC_MAP@3_std(MIRACL)
251
+ value: 0.9799184530397409
252
+ - type: nAUC_MAP@5_diff1(MIRACL)
253
+ value: 19.20571689941054
254
+ - type: nAUC_MAP@5_max(MIRACL)
255
+ value: 7.987468654026893
256
+ - type: nAUC_MAP@5_std(MIRACL)
257
+ value: 1.8324246565938962
258
+ - type: nAUC_NDCG@1000_diff1(MIRACL)
259
+ value: 3.7537669018914768
260
+ - type: nAUC_NDCG@1000_max(MIRACL)
261
+ value: 20.7944707840533
262
+ - type: nAUC_NDCG@1000_std(MIRACL)
263
+ value: 8.444837055303063
264
+ - type: nAUC_NDCG@100_diff1(MIRACL)
265
+ value: 3.7537669018914768
266
+ - type: nAUC_NDCG@100_max(MIRACL)
267
+ value: 20.7944707840533
268
+ - type: nAUC_NDCG@100_std(MIRACL)
269
+ value: 8.444837055303063
270
+ - type: nAUC_NDCG@10_diff1(MIRACL)
271
+ value: 10.829575656103888
272
+ - type: nAUC_NDCG@10_max(MIRACL)
273
+ value: 13.0445496498929
274
+ - type: nAUC_NDCG@10_std(MIRACL)
275
+ value: 6.050412212625362
276
+ - type: nAUC_NDCG@1_diff1(MIRACL)
277
+ value: 19.1388712233292
278
+ - type: nAUC_NDCG@1_max(MIRACL)
279
+ value: 10.871900994781642
280
+ - type: nAUC_NDCG@1_std(MIRACL)
281
+ value: 3.218568248751811
282
+ - type: nAUC_NDCG@20_diff1(MIRACL)
283
+ value: 7.093172181746442
284
+ - type: nAUC_NDCG@20_max(MIRACL)
285
+ value: 16.955238078958836
286
+ - type: nAUC_NDCG@20_std(MIRACL)
287
+ value: 8.325656379573035
288
+ - type: nAUC_NDCG@3_diff1(MIRACL)
289
+ value: 17.134437303330802
290
+ - type: nAUC_NDCG@3_max(MIRACL)
291
+ value: 10.235328822955793
292
+ - type: nAUC_NDCG@3_std(MIRACL)
293
+ value: 3.2341358691084814
294
+ - type: nAUC_NDCG@5_diff1(MIRACL)
295
+ value: 14.733664618337636
296
+ - type: nAUC_NDCG@5_max(MIRACL)
297
+ value: 11.181897412035282
298
+ - type: nAUC_NDCG@5_std(MIRACL)
299
+ value: 3.642277088791985
300
+ - type: nAUC_P@1000_diff1(MIRACL)
301
+ value: -26.330038284867573
302
+ - type: nAUC_P@1000_max(MIRACL)
303
+ value: 28.450694137240458
304
+ - type: nAUC_P@1000_std(MIRACL)
305
+ value: 9.892993775474912
306
+ - type: nAUC_P@100_diff1(MIRACL)
307
+ value: -26.330038284867552
308
+ - type: nAUC_P@100_max(MIRACL)
309
+ value: 28.45069413724051
310
+ - type: nAUC_P@100_std(MIRACL)
311
+ value: 9.892993775474928
312
+ - type: nAUC_P@10_diff1(MIRACL)
313
+ value: -17.436937353231112
314
+ - type: nAUC_P@10_max(MIRACL)
315
+ value: 24.327018012947857
316
+ - type: nAUC_P@10_std(MIRACL)
317
+ value: 11.78803527706634
318
+ - type: nAUC_P@1_diff1(MIRACL)
319
+ value: 19.1388712233292
320
+ - type: nAUC_P@1_max(MIRACL)
321
+ value: 10.871900994781642
322
+ - type: nAUC_P@1_std(MIRACL)
323
+ value: 3.218568248751811
324
+ - type: nAUC_P@20_diff1(MIRACL)
325
+ value: -22.947528755272426
326
+ - type: nAUC_P@20_max(MIRACL)
327
+ value: 27.773093471902538
328
+ - type: nAUC_P@20_std(MIRACL)
329
+ value: 14.898619107087221
330
+ - type: nAUC_P@3_diff1(MIRACL)
331
+ value: 1.4100426412400944
332
+ - type: nAUC_P@3_max(MIRACL)
333
+ value: 17.397472872058845
334
+ - type: nAUC_P@3_std(MIRACL)
335
+ value: 8.240008229861875
336
+ - type: nAUC_P@5_diff1(MIRACL)
337
+ value: -7.971349332207021
338
+ - type: nAUC_P@5_max(MIRACL)
339
+ value: 22.198441167940963
340
+ - type: nAUC_P@5_std(MIRACL)
341
+ value: 9.00265164460082
342
+ - type: nAUC_Recall@1000_diff1(MIRACL)
343
+ value: -38.69835271863148
344
+ - type: nAUC_Recall@1000_max(MIRACL)
345
+ value: 50.9545152809108
346
+ - type: nAUC_Recall@1000_std(MIRACL)
347
+ value: 20.44270887092116
348
+ - type: nAUC_Recall@100_diff1(MIRACL)
349
+ value: -38.69835271863148
350
+ - type: nAUC_Recall@100_max(MIRACL)
351
+ value: 50.9545152809108
352
+ - type: nAUC_Recall@100_std(MIRACL)
353
+ value: 20.44270887092116
354
+ - type: nAUC_Recall@10_diff1(MIRACL)
355
+ value: -0.08109036309433801
356
+ - type: nAUC_Recall@10_max(MIRACL)
357
+ value: 12.696619907773568
358
+ - type: nAUC_Recall@10_std(MIRACL)
359
+ value: 8.791982704261589
360
+ - type: nAUC_Recall@1_diff1(MIRACL)
361
+ value: 28.698973487482206
362
+ - type: nAUC_Recall@1_max(MIRACL)
363
+ value: 2.9217687660885034
364
+ - type: nAUC_Recall@1_std(MIRACL)
365
+ value: -1.1247408800976524
366
+ - type: nAUC_Recall@20_diff1(MIRACL)
367
+ value: -13.312171017942623
368
+ - type: nAUC_Recall@20_max(MIRACL)
369
+ value: 24.19847346821666
370
+ - type: nAUC_Recall@20_std(MIRACL)
371
+ value: 15.8157702609797
372
+ - type: nAUC_Recall@3_diff1(MIRACL)
373
+ value: 16.909128321353343
374
+ - type: nAUC_Recall@3_max(MIRACL)
375
+ value: 6.552122731902991
376
+ - type: nAUC_Recall@3_std(MIRACL)
377
+ value: 1.9963898223457228
378
+ - type: nAUC_Recall@5_diff1(MIRACL)
379
+ value: 9.990292655247721
380
+ - type: nAUC_Recall@5_max(MIRACL)
381
+ value: 9.361722273507574
382
+ - type: nAUC_Recall@5_std(MIRACL)
383
+ value: 3.270918827854495
384
+ task:
385
+ type: Reranking
386
+ - dataset:
387
+ config: default
388
+ name: MTEB SensitiveTopicsClassification (default)
389
+ revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
390
+ split: test
391
+ type: ai-forever/sensitive-topics-classification
392
+ metrics:
393
+ - type: accuracy
394
+ value: 30.634765625
395
+ - type: f1
396
+ value: 32.647559808678665
397
+ - type: lrap
398
+ value: 45.94319661458259
399
+ - type: main_score
400
+ value: 30.634765625
401
+ task:
402
+ type: MultilabelClassification
403
  - dataset:
404
  config: default
405
  name: MTEB ATEC (default)