bourdoiscatie commited on
Commit
d714cb7
1 Parent(s): a0e4338

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -1,3 +1,15 @@
1
  ---
 
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language: en
3
  license: apache-2.0
4
  ---
5
+
6
+ ### Description
7
+ Adaptation of the [flan-t5-base](https://huggingface.co/google/flan-t5-base) weights to make it compatible with the [FAT5](https://github.com/catie-aq/flashT5) framework (Flash Attention T5).
8
+ This adaptation should enable the user to efficiently continue the pre-training of the flan-t5 to adapt it to more recent data, or to specialize it in a specific domain, for example.
9
+
10
+ ### Usage
11
+ ```
12
+ from transformers import AutoModel, AutoTokenizer
13
+ model = AutoModel.from_pretrained("CATIE-AQ/FAT5-base-flan-en", trust_remote_code=True)
14
+ tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-base")
15
+ ```