To use this model with llama-cli correctly

#1
by b1u3m3 - opened
Shrewd Security org
edited 4 days ago
llama-cli --hf-repo AcuteShrewdSecurity/Llama-Phishsense-1B-Q8_0-GGUF --hf-file llama-phishsense-1b-q8_0.gguf -p "Classify the following text as phishing or not. Respond with 'TRUE' or 'FALSE', followed by detailed reasoning:\n\nSubject: Reminder: USPS Holiday Schedule\n\nDear John,\n\nWe hope this message finds you well! As the holiday season approaches, we want to remind you that USPS will operate on a modified schedule on federal holidays. To ensure your packages arrive on time, please plan your shipments accordingly. For more information, visit our official website at https://usps.com/holiday-schedule. Thank you for choosing USPS, and we wish you a wonderful holiday seasonregards,\nMary Johnson\nCustomer Support Team\nUnited States Postal Service\nAnswer:" -n 1 --lora ~/Llama-Phishsense-1B-f32.gguf 2>/dev/null

Returns FALSE

llama-cli --hf-repo AcuteShrewdSecurity/Llama-Phishsense-1B-Q8_0-GGUF --hf-file llama-phishsense-1b-q8_0.gguf -p "Classify the following text as phishing or not. Respond with 'TRUE' or 'FALSE', followed by detailed reasoning:\n\nSubject: Action Required: Verify Your USPS Package Delivery\n\nDear John,\n\nWe are writing to inform you of an issue with your recent package delivery attempt. To avoid delays or a possible return to the sender, please confirm your shipping address and delivery preferences by clicking the secure link below: https://usps-secure-tracking.com/verify. Immediate action is required, as unverified packages will be held for only 24 hours. Thank you for choosing USPS, and we appreciate your prompt attention to this matter.\n\nBest regards,\nMary Johnson\nCustomer Support Team\nUnited States Postal Service\nAnswer:" -n 1 --lora ~/Llama-Phishsense-1B-f32.gguf 2>/dev/null

Returns TRUE

Things to note:

  1. Gguf format for lora is available here
b1u3m3 changed discussion title from To use this model with llama-cli to To use this model with llama-cli correctly

Sign up or log in to comment