Metrics
14 Record Views
Preprint
Proactive Adversarial Defense: Harnessing Prompt Tuning in Vision-Language Models to Detect Unseen Backdoored Images
Cornell University, v.2
12/11/2024
Abstract
Backdoor attacks pose a critical threat by embedding hidden triggers into inputs, causing models to misclassify them into target labels. While extensive
research has focused on mitigating these attacks in object recognition models through weight fine-tuning, much less attention has been given to detecting
backdoored samples directly. Given the vast datasets used in training, manual inspection for backdoor triggers is impractical, and even state-of-the-art
defense mechanisms fail to fully neutralize their impact. To address this gap, we introduce a groundbreaking method to detect unseen backdoored images during both training and inference. Leveraging the transformative success of prompt tuning in Vision Language Models (VLMs), our approach trains learnable text prompts to differentiate clean images from those with hidden backdoor triggers. Experiments demonstrate the exceptional efficacy of this method, achieving an impressive average accuracy of 86% across two renowned datasets for detecting unseen backdoor triggers, establishing a new standard in backdoor defense.
Files and links (1)
Details
- Title
- Proactive Adversarial Defense
- Edition
- v.2
- Resource Type
- Preprint
- Publisher
- Cornell University; arXiv
- Format
- link
- Number of pages
- 5
- Identifiers
- 99380592374806600
- Academic Unit
- Center for Cybersecurity; Division of Academic Affairs
- Language
- English