ai-forever
/

RUDOLPH-1.3B

Model card Files Files and versions Community

sberbank-ai commited on Oct 6, 2022

Commit

99b265c

•

1 Parent(s): e36d5a0

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -43,7 +43,7 @@ RUDOLPH 1.3B is a Transformer-based decoder model with the following parameters:
 The primary proposed method is to modify the sparse transformer's attention mask to better control multi-modalities and up to the next level with "hyper-modality". It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
-<img src="https://raw.githubusercontent.com/ai-forever/ru-dolph/master/pics/attention_mask_13b.png" height="20" border="2"/>
 # Authors

 The primary proposed method is to modify the sparse transformer's attention mask to better control multi-modalities and up to the next level with "hyper-modality". It allows us to calculate the transitions of modalities in both directions, unlike another similar work DALL-E Transformer, which used only one direction, "text to image". The proposed "image to right text" direction is achieved by extension sparse attention mask to the right for auto-repressively text generation with both image and left text condition.
+<img src="https://raw.githubusercontent.com/lizagonch/ru-dolph/develop_v1/pics/attention_masks_1300m.png" height="20" border="2"/>
 # Authors