Weight size VS VRAM requirements

#8
by mindkrypted - opened

Hello, I'd be interested to hear about what makes the model's inference require the large amount of VRAM ( 4 x 80GB )?
Weight is about 40GB and logically fits into 48GB of VRAM with enough leftover for a short* context.

Very promising model, great job from the team.
Thanks!

Genmo org

The model requires a huge sequence length for generating videos (44.5K tokens) -- which takes memory.
Also the VAE is a massive memory-hog.
But, we've reduced the requirements, so it is now possible on a single 4090.

if you are looking for insight or alternate routes - eyeball the following: it can work on a 3090 - takes about 17-18 Gb IIRC
(check out the https://github.com/victorchall/genmoai-smol repo or https://github.com/kijai/ComfyUI-MochiWrapper which provides a gguf and an f8 of the weights)
Huge thanks to the genmo team

@ved-genmo great work folks! It would be super helpful if you guys can dedicate a section in Readme about hardware & time requirements to run this.

The model requires a huge sequence length for generating videos (44.5K tokens) -- which takes memory.
Also the VAE is a massive memory-hog.
But, we've reduced the requirements, so it is now possible on a single 4090.

That's awesome, thanks for sharing the details.
44.5k tokens, impressive!

if you are looking for insight or alternate routes - eyeball the following: it can work on a 3090 - takes about 17-18 Gb IIRC
(check out the https://github.com/victorchall/genmoai-smol repo or https://github.com/kijai/ComfyUI-MochiWrapper which provides a gguf and an f8 of the weights)
Huge thanks to the genmo team

Excellent does this mean this will not be able to run on a m2 mac 32gb?

if you are looking for insight or alternate routes - eyeball the following: it can work on a 3090 - takes about 17-18 Gb IIRC
(check out the https://github.com/victorchall/genmoai-smol repo or https://github.com/kijai/ComfyUI-MochiWrapper which provides a gguf and an f8 of the weights)
Huge thanks to the genmo team

Excellent does this mean this will not be able to run on a m2 mac 32gb?

not sure. sorry. It will prolly depend on the the base footprint of the OS.

Sign up or log in to comment