Can't merge files with gguf
When I try to merge the two parts of iq4_xs I get an error: gguf_init_from_file: invalid magic characters '¿≤:°' and then it destroys the file
MINGW64 ~/.cache/lm-studio/models/dranger003/c4ai-command-r-plus-iMat.GGUF/join
$ ls -l
total 54877592
-rw-r--r-- 1 J 197121 48653614944 Apr 14 03:50 ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf
-rw-r--r-- 1 J 197121 7547577984 Apr 16 20:20 ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf
MINGW64 ~/.cache/lm-studio/models/dranger003/c4ai-command-r-plus-iMat.GGUF/join
$ ../../../../../../LLM/llama-b2687-bin-win-cuda-cu11.7.1-x64/gguf-split.exe --merge ggml-c4ai-command-r-plus-104b-iq4_xs
-00001-of-00002.gguf ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf
gguf_merge: ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf -> ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf
gguf_merge: reading metadata ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf done
gguf_merge: reading metadata ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf ...gguf_init_from_file: invalid magic characters '¿≤:°'
gguf_merge: failed to load input GGUF from ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf
MINGW64 ~/.cache/lm-studio/models/dranger003/c4ai-command-r-plus-iMat.GGUF/join
$ ls -l
total 47507140
-rw-r--r-- 1 J 197121 48653614944 Apr 14 03:50 ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf
-rw-r--r-- 1 J 197121 0 Apr 16 20:37 ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf
These other bugs are possibly related or caused by this:
https://huggingface.co/dranger003/c4ai-command-r-plus-iMat.GGUF/discussions/12
https://huggingface.co/dranger003/c4ai-command-r-plus-iMat.GGUF/discussions/11
@zedmango
You need to use gguf-split, do you know how lm studio is merging?
@zedmango You need to use
gguf-split, do you know how lm studio is merging?
That was me on the command line, not lm studio. I used gguf-split --merge. See what I put above:
$ ../../../../../../LLM/llama-b2687-bin-win-cuda-cu11.7.1-x64/gguf-split.exe --merge ggml-c4ai-command-r-plus-104b-iq4_xs -00001-of-00002.gguf ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf
I entered that command into the command line and got the above error. Using gguf-split --merge.
I don't think you're supposed to specify all the parts, just the first one but I can't test this right now. Also, which commit hash are you on? I think they may recently have changed how gguf-split works.
Thank you! That worked:
j@LAPTOP-QD5K4715:/mnt/c/Users/J/.cache/lm-studio/models/dranger003/c4ai-command-r-plus-iMat.GGUF/join
$ ../../../../../../LLM/llama-b2687-bin-win-cuda-cu11.7.1-x64/gguf-split.exe --merge ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf ggml-c4ai-command-r-plus-104b-iq4_xs.gguf
gguf_merge: ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf -> ggml-c4ai-command-r-plus-104b-iq4_xs.gguf
gguf_merge: reading metadata ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf done
gguf_merge: reading metadata ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf done
gguf_merge: writing tensors ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf done
gguf_merge: writing tensors ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf done
gguf_merge: ggml-c4ai-command-r-plus-104b-iq4_xs.gguf merged from 2 split with 642 tensors.
Oops, left out the command - here it is
gguf_merge: ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf -> ggml-c4ai-command-r-plus-104b-iq4_xs.gguf
gguf_merge: reading metadata ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf done
gguf_merge: reading metadata ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf done
gguf_merge: writing tensors ggml-c4ai-command-r-plus-104b-iq4_xs-00001-of-00002.gguf done
gguf_merge: writing tensors ggml-c4ai-command-r-plus-104b-iq4_xs-00002-of-00002.gguf done
gguf_merge: ggml-c4ai-command-r-plus-104b-iq4_xs.gguf merged from 2 split with 642 tensors.```
Anyway, thank you!