| The easiest way we recommend doing this is by using an iterator: | |
| def data(): | |
| for i in range(1000): | |
| yield f"My example {i}" | |
| pipe = pipeline(model="openai-community/gpt2", device=0) | |
| generated_characters = 0 | |
| for out in pipe(data()): | |
| generated_characters += len(out[0]["generated_text"]) | |
| The iterator data() yields each result, and the pipeline automatically | |
| recognizes the input is iterable and will start fetching the data while | |
| it continues to process it on the GPU (this uses DataLoader under the hood). |