Gabriel C's picture

Gabriel C

gabrielchua

·

https://gabrielchua.me

AI & ML interests

None yet

Recent Activity

updated a Space 1 day ago

govtech/off-topic-demo

posted an update 2 days ago

Sharing my first paper! == Large Language Models (LLMs) are powerful, but they're prone to off-topic misuse, where users push them beyond their intended scope. Think harmful prompts, jailbreaks, and misuse. So how do we build better guardrails? Traditional guardrails rely on curated examples or classifiers. The problem? ⚠️ High false-positive rates ⚠️ Poor adaptability to new misuse types ⚠️ Require real-world data, which is often unavailable during pre-production Our method skips the need for real-world misuse examples. Instead, we: 1️⃣ Define the problem space qualitatively 2️⃣ Use an LLM to generate synthetic misuse prompts 3️⃣ Train and test guardrails on this dataset We apply this to the off-topic prompt detection problem, and fine-tune simple bi- and cross-encoder classifiers that outperform heuristics based on cosine similarity or prompt engineering. Additionally, framing the problem as prompt relevance allows these fine-tuned classifiers to generalise to other risk categories (e.g., jailbreak, toxic prompts). Through this work, we also open-source our dataset (2M examples, ~50M+ tokens) and models. paper: https://huggingface.co/papers/2411.12946 artifacts: https://huggingface.co/collections/govtech/off-topic-guardrail-673838a62e4c661f248e81a4

upvoted a paper 2 days ago

Harnessing the Potential of Gen-AI Coding Assistants in Public Sector Software Development

View all activity

Organizations

gabrielchua's activity

commented a paper 3 days ago

A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection

Paper • 2411.12946 • Published 8 days ago • 19 •

New activity in gabrielchua/open-notebooklm about 1 month ago

Swedish Motors

#11 opened about 2 months ago by

Could you explain the "Use advanced audio generation? (Experimental)" option?

#12 opened about 2 months ago by

New activity in gabrielchua/open-notebooklm about 2 months ago

What features would you like?

#13 opened about 2 months ago by

The accent makes it charming :-)

#10 opened about 2 months ago by

Súper Neurona test - Castigos y consecencias

#9 opened about 2 months ago by

only support one 'ZH' speaker

#8 opened about 2 months ago by

More voices

#7 opened about 2 months ago by

interview Reza

#6 opened about 2 months ago by

Apply for community grant: Personal project (gpu)

#1 opened about 2 months ago by

Yay lot break up’s

#3 opened about 2 months ago by

runtime error

#4 opened about 2 months ago by

Update app.py

#2 opened about 2 months ago by

New activity in gabrielchua/off-topic 3 months ago

Librarian Bot: Add language metadata for dataset

#2 opened 3 months ago by

New activity in answerdotai/answerai-colbert-small-v1 3 months ago

Fine-tuning example

#9 opened 3 months ago by

New activity in meta-llama/Prompt-Guard-86M 3 months ago

Model classifies everything as unsafe

#15 opened 4 months ago by

New activity in gabrielchua/hey-gemma 9 months ago

Apply for community grant: Personal project (gpu)

#1 opened 9 months ago by

New activity in gabrielchua/catch-me-if-you-can 9 months ago

Quick hack to disable show_progress

#3 opened 9 months ago by

Output to Markdown (replaces #1)

#2 opened 9 months ago by