mistralai

/

Mistral-7B-Instruct-v0.2

like 2.58k

Follow

Mistral AI_ 3,082

Hi Team,

While using Quantized version on a GCP instance with cpu it is taking ~10 min for each Question which is taking only few seconds with API. Can someone please suggest if we have an alternative here either to use GPU or use a Full version instead. Some articles or references are appreciated.

Thanks