Hello, can the rtx 3050 8g run this model?

#1
by win10 - opened

May I ask if the rtx 3050 8g can run this model?

No, this model requires about 14Gbs just to load, and will consume as much VRAM you can possibly feed it to store context.
I'd say that a 24Gb card is the minimum & that'd let you load about 5k tokens in memory.

TehVenom changed discussion status to closed

Sign up or log in to comment