bartowski

/

Meta-Llama-3.1-8B-Instruct-GGUF

like 85

I've never seen that before :o can you show me? Are you sure that doesn't only apply to .bin files?

Hmm, I'm looking back at the old model I used, I can't seem to find the virus scan indicator I was reffering to. I haven't used it in awhile so I forgot where it stated this. Is there a small script we can write to check the model info which should include the virus scanning status?

I'm not aware of any sadly, BUT, you can see that HF scans GGUF files and shows all the metadata and layers in the browser natively (https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/tree/main?show_file_info=Meta-Llama-3.1-8B-Instruct-Q6_K.gguf)

I'm not sure if that's comprehensive enough though. I think the bigger concern is making sure you're running them with a trusted tool (llama.cpp directly, koboldcpp, lmstudio, text gen webui come to mind)

I'm not aware of any existing virus scanning for GGUF at this time but if you become aware of any please let me know :)

Will do! I'm currently using llama.cpp. Also, yes, me too, I'm using modelscan() for models in general but it skips over GGUF files because it assumes they are safe. Will keep you posted!

i think this may be caused due to file size

large files are not automatically scanned by hugging face for viruses and dont have any "No Virus" or "Virus Detected" marker

Ooooo you know what, I DO not see that "no virus" shield on the smaller files, so yes that must be it.. I don't know if there's a sophisticated enough way for huggingface to actually "scan" uploaded GGUF files, short of attempting to run them all in a VM, but even then a highly advanced virus could run the model just fine but then execute some extra code.. So i think it sadly still comes back to making sure the tool is updated and you're downloading from someone you trust (whether that's me or not), or if you can't trust anyone make the conversions locally