Is there any tuturial for inference with batch_size > 1?
#29
by
virtueai-mz
- opened
The model.chat function only supports bs=1. I want to know if there is a code example for bs>1.
+1
+1
This comment has been hidden