Suchinthana commited on
Commit
8b78cf6
1 Parent(s): 1790d8a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -1,3 +1,39 @@
1
  ---
2
  license: apache-2.0
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - si
5
  ---
6
+ ### Fine tuned MT5 base model with Sinhala English Wikipedia Dataset (Experimentally continues training)
7
+
8
+ This model is fine tuned with articles from Sinhala Wikipedia for article generation. Used around 10,000 articles for training and fine tuned arround 95 times.
9
+
10
+ ### How to use
11
+
12
+ We have to use **"writeWiki: "** part at the begining of each prompt.
13
+
14
+ You can use this model with a pipeline for text generation.
15
+
16
+ First you might need to install required libraries and import them.
17
+ ```py
18
+ !pip uninstall transformers -y
19
+ !pip install transformers
20
+
21
+ pip install tokenizers sentencepiece
22
+ ```
23
+
24
+ Then we might need to restart the runtime either manually or use the below code to end it.
25
+ ```py
26
+ import os
27
+ os.kill(os.getpid(), 9)
28
+ ```
29
+
30
+ Then we just have to import the tokenizer and run the pipeline:
31
+
32
+ ```py
33
+ from transformers import AutoTokenizer
34
+ tokenizer = AutoTokenizer.from_pretrained('google/mt5-small')
35
+
36
+ from transformers import pipeline
37
+ generator = pipeline(model='Suchinthana/MT5-Sinhala-Wikigen-Experimental', tokenizer=tokenizer)
38
+ generator("writeWiki: මානව ආහාර", do_sample=True, max_length=180)
39
+ ```