Update README.md
Browse files
README.md
CHANGED
@@ -82,6 +82,13 @@ Encoder:
|
|
82 |
|
83 |
## How to Get Started with the Model
|
84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
85 |
```python
|
86 |
from transformers import AutoProcessor, VisionEncoderDecoderModel
|
87 |
import requests
|
@@ -110,6 +117,9 @@ loss = outputs.loss
|
|
110 |
generated_ids = model.generate(pixel_values)
|
111 |
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
112 |
```
|
|
|
|
|
|
|
113 |
|
114 |
[More Information Needed]
|
115 |
|
|
|
82 |
|
83 |
## How to Get Started with the Model
|
84 |
|
85 |
+
|
86 |
+
### VisionEncoderDecoderModel
|
87 |
+
#### As a vision encoder model :
|
88 |
+
|
89 |
+
the tensors are combined into the original mistral model so it can be accessed by intaciating the correct model which is the VisionEncoderDecoderModel
|
90 |
+
|
91 |
+
|
92 |
```python
|
93 |
from transformers import AutoProcessor, VisionEncoderDecoderModel
|
94 |
import requests
|
|
|
117 |
generated_ids = model.generate(pixel_values)
|
118 |
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
119 |
```
|
120 |
+
### As a standard LLM:
|
121 |
+
|
122 |
+
it can still also be used as a normal AutoModelForCausalLM or MistralModelForCausalLM !
|
123 |
|
124 |
[More Information Needed]
|
125 |
|