Sharathhebbar24
commited on
Commit
•
4b47c40
1
Parent(s):
f6df300
Update README.md
Browse files
README.md
CHANGED
@@ -18,13 +18,14 @@ of publicly available data) with an automatic process to generate inputs and lab
|
|
18 |
it was trained to guess the next word in sentences.
|
19 |
|
20 |
More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,
|
21 |
-
|
22 |
-
predictions for the token `i` only
|
23 |
|
24 |
This way, the model learns an inner representation of the English language that can then be used to extract features
|
25 |
useful for downstream tasks. The model is best at what it was trained for, however, which is generating texts from a
|
26 |
prompt.
|
27 |
|
|
|
28 |
|
29 |
```python
|
30 |
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
18 |
it was trained to guess the next word in sentences.
|
19 |
|
20 |
More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,
|
21 |
+
shifting one token (word or piece of word) to the right. The model uses a mask mechanism to make sure the
|
22 |
+
predictions for the token `i` only use the inputs from `1` to `i` but not the future tokens.
|
23 |
|
24 |
This way, the model learns an inner representation of the English language that can then be used to extract features
|
25 |
useful for downstream tasks. The model is best at what it was trained for, however, which is generating texts from a
|
26 |
prompt.
|
27 |
|
28 |
+
### To use this model
|
29 |
|
30 |
```python
|
31 |
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
|