Update README.md
Browse files
README.md
CHANGED
|
@@ -156,14 +156,16 @@ You can finetune this model on your own dataset.
|
|
| 156 |
|
| 157 |
## Benchmarks
|
| 158 |
|
|
|
|
| 159 |
### Retieval
|
| 160 |
-
Evaluated with [MIRACL-ja](https://huggingface.co/datasets/miracl/miracl), [JQARA](https://huggingface.co/datasets/hotchpotch/JQaRA) and [MLDR-ja](https://huggingface.co/datasets/Shitao/MLDR).
|
| 161 |
|
| 162 |
-
| model | size | MIRACL<br>Recall@5 | JQaRA<br>nDCG@10 | MLDR<br>nDCG@10 |
|
| 163 |
-
|
| 164 |
-
|
|
| 165 |
-
| GLuCoSE | 0.1B | 53.3 | 30.8 | 25.2 |
|
| 166 |
-
|
|
|
|
|
| 167 |
|
| 168 |
|
| 169 |
### JMTEB
|
|
@@ -173,10 +175,11 @@ Evaluated with [JMTEB](https://github.com/sbintuitions/JMTEB).
|
|
| 173 |
|
| 174 |
| model | size | Class. | Ret. | STS. | Clus. | Pair. | Avg. |
|
| 175 |
|:--:|:--:|:--:|:--:|:----:|:-------:|:-------:|:------:|
|
| 176 |
-
|
|
| 177 |
-
| GLuCoSE | 0.1B | **82.6** | 69.8 | 78.2 | 51.5 | **66.2** | 69.7 |
|
| 178 |
| RoSEtta | 0.2B | 79.0 | **84.3** | **81.4** | **53.2** | 61.7 | **71.9** |
|
| 179 |
|
|
|
|
| 180 |
## Authors
|
| 181 |
Chihiro Yano, Mocho Go, Hideyuki Tachibana, Hiroto Takegawa, Yotaro Watanabe
|
| 182 |
|
|
|
|
| 156 |
|
| 157 |
## Benchmarks
|
| 158 |
|
| 159 |
+
|
| 160 |
### Retieval
|
| 161 |
+
Evaluated with [MIRACL-ja](https://huggingface.co/datasets/miracl/miracl), [JQARA](https://huggingface.co/datasets/hotchpotch/JQaRA) , [JaCWIR](https://huggingface.co/datasets/hotchpotch/JaCWIR) and [MLDR-ja](https://huggingface.co/datasets/Shitao/MLDR).
|
| 162 |
|
| 163 |
+
| model | size | MIRACL<br>Recall@5 | JQaRA<br>nDCG@10 | JaCWIR<br>MAP@10 | MLDR<br>nDCG@10 |
|
| 164 |
+
|:--:|:--:|:--:|:--:|:--:|:----:|
|
| 165 |
+
| [mE5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 0.3B | **84.2** | 47.2 | **85.3** | 25.4 |
|
| 166 |
+
| [GLuCoSE](https://huggingface.co/pkshatech/GLuCoSE-base-ja) | 0.1B | 53.3 | 30.8 | 68.6 | 25.2 |
|
| 167 |
+
|[ruri-base](https://huggingface.co/cl-nagoya/ruri-base) | 0.1B | 74.3 | **58.1** | 84.6 | **35.3** |
|
| 168 |
+
| RoSEtta | 0.2B | 79.3 | 57.7 | 83.8 | 32.3 |
|
| 169 |
|
| 170 |
|
| 171 |
### JMTEB
|
|
|
|
| 175 |
|
| 176 |
| model | size | Class. | Ret. | STS. | Clus. | Pair. | Avg. |
|
| 177 |
|:--:|:--:|:--:|:--:|:----:|:-------:|:-------:|:------:|
|
| 178 |
+
| [mE5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 0.3B | 75.1 | 80.6 | 80.5 | **52.6** | 62.4 | 70.2 |
|
| 179 |
+
| [GLuCoSE](https://huggingface.co/pkshatech/GLuCoSE-base-ja) | 0.1B | **82.6** | 69.8 | 78.2 | 51.5 | **66.2** | 69.7 |
|
| 180 |
| RoSEtta | 0.2B | 79.0 | **84.3** | **81.4** | **53.2** | 61.7 | **71.9** |
|
| 181 |
|
| 182 |
+
|
| 183 |
## Authors
|
| 184 |
Chihiro Yano, Mocho Go, Hideyuki Tachibana, Hiroto Takegawa, Yotaro Watanabe
|
| 185 |
|