ukr-models commited on
Commit
6cf5dfd
1 Parent(s): 3acf6c4

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - uk
4
+ tags:
5
+ - ukrainian
6
+ widget:
7
+ - text: "Тарас Шевченко – великий український <mask>."
8
+ license: mit
9
+ ---
10
+ This is a smaller version of the [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) model with only Ukrainian and some English embeddings left.
11
+
12
+ * The original model has 470M parameters, with 384M of them being input and output embeddings.
13
+ * After shrinking the `sentencepiece` vocabulary from 250K to 31K (top 25K Ukrainian tokens and top English tokens) the number of model parameters reduced to 134M parameters, and model size reduced from 1GB to 400MB.