Should this work with llama2.c from karpathy?
See title. Thank you!
It is compatible with llama.cpp. I have not played with llama2.c yet but my best guess is Yes. We use the same arc. & tokenizer with Llama 2. Maybe you need to modify some config files a bit to fit our model shape.
See title. Thank you!
Appreciate if you share your test result here if you plan to test it out.
Thanks.
I tried it before and got “segmentation fault”.
I should have some time later tonight for tinkering, if I get a better result I’ll write an update.
I tried it before and got “segmentation fault”.
I should have some time later tonight for tinkering, if I get a better result I’ll write an update.
I'm hitting the following error when running export.py:
RuntimeError: shape '[32, 2, 32, 2048]' is invalid for input of size 524288
May I know any update from your side? It will be great if you can share your findings so far, I think it will benefit everyone here.
Thank you.
I tried it before and got “segmentation fault”.
I should have some time later tonight for tinkering, if I get a better result I’ll write an update.
I'm hitting the following error when running export.py:
RuntimeError: shape '[32, 2, 32, 2048]' is invalid for input of size 524288May I know any update from your side? It will be great if you can share your findings so far, I think it will benefit everyone here.
Thank you.
Unfortunately that's as far as I got as well.