Update README.md
Browse files
README.md
CHANGED
@@ -11,12 +11,26 @@ GitHub: https://github.com/RAIVNLab/AdANNS
|
|
11 |
|
12 |
Arxiv: https://arxiv.org/abs/2305.19435
|
13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
We provide four BERT-Base models finetuned on Natural Questions with [Matryoshka Representation Learning](https://github.com/RAIVNLab/MRL) (MRL).
|
15 |
|
16 |
A vanilla pretrained BERT-Base has a 768-d representation (information bottleneck). As we train with MRL, we enforce the network to learn representations at
|
17 |
multiple granularities nested within a 768-d embedding. The granularities at which we finetune BERT-Base with Matroyshka Loss are specified in the folder name,
|
18 |
e.g. for `dpr-nq-d768_384_192_96_48`, we have d=[48, 96, 192, 384, 768].
|
19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
## Citation
|
21 |
If you find this project useful in your research, please consider citing:
|
22 |
```
|
@@ -28,4 +42,4 @@ If you find this project useful in your research, please consider citing:
|
|
28 |
month = {December},
|
29 |
year = {2023},
|
30 |
}
|
31 |
-
```
|
|
|
11 |
|
12 |
Arxiv: https://arxiv.org/abs/2305.19435
|
13 |
|
14 |
+
<p align="center">
|
15 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/64af72d4a609b29cc7b5919b/QYOqZ25qc9pTWlFR0D0VF.png" alt="drawing" width="600"/>
|
16 |
+
Adaptive representations can be utilized effectively in the decoupled components of clustering and
|
17 |
+
searching for a better accuracy-compute trade-off (AdANNS-IVF).
|
18 |
+
</p>
|
19 |
+
|
20 |
We provide four BERT-Base models finetuned on Natural Questions with [Matryoshka Representation Learning](https://github.com/RAIVNLab/MRL) (MRL).
|
21 |
|
22 |
A vanilla pretrained BERT-Base has a 768-d representation (information bottleneck). As we train with MRL, we enforce the network to learn representations at
|
23 |
multiple granularities nested within a 768-d embedding. The granularities at which we finetune BERT-Base with Matroyshka Loss are specified in the folder name,
|
24 |
e.g. for `dpr-nq-d768_384_192_96_48`, we have d=[48, 96, 192, 384, 768].
|
25 |
|
26 |
+
You can easily load an mrl-nq model as follows:
|
27 |
+
```
|
28 |
+
from transformers import BertModel
|
29 |
+
import torch
|
30 |
+
|
31 |
+
model = BertModel.from_pretrained('dpr-nq-d768_384_192_96_48')
|
32 |
+
```
|
33 |
+
|
34 |
## Citation
|
35 |
If you find this project useful in your research, please consider citing:
|
36 |
```
|
|
|
42 |
month = {December},
|
43 |
year = {2023},
|
44 |
}
|
45 |
+
```
|