Metrics
15 Record Views
Conference proceeding
Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection
Proceedings: 2025 IEEE 49th Annual Computers, Software, and Applications Conference COMPSAC 2025, pp.1033-1039
IEEE Annual International Computer Software and Applications Conference
Annual Computers, Software, and Applications Conference (COMPSAC), 49th (Toronto, Ontario, Canada, 07/08/2025–07/11/2025)
08/26/2025
Web of Science ID: WOS:001575960000127
Abstract
Large language models (LLMs) are becoming a popular tool as they have significantly advanced in their capability to tackle a wide range of language-based tasks. However, LLMs applications are highly vulnerable to prompt injection attacks, which poses a critical problem. These attacks target LLMs applications through using carefully designed input prompts to divert the model from adhering to original instruction, thereby it could execute unintended actions. These manipulations pose serious security threats which potentially results in data leaks, biased outputs, or harmful responses. This project explores the security vulnerabilities in relation to prompt injection attacks. To detect whether a prompt is vulnerable or not, we follows two approaches: 1) a pre-trained LLM, and 2) a fine-tuned LLM. Then, we conduct a thorough analysis and comparison of the classification performance. Firstly, we use pre-trained XLM-RoBERTa model to detect prompt injections using test dataset without any fine-tuning and evaluate it by zero-shot classification. Then, this proposed work will apply supervised fine-tuning to this pre-trained LLM using a task-specific labeled dataset from deepset in huggingface, and this fine-tuned model achieves impressive results with 99.13% accuracy, 100% precision, 98.33% recall and 99.15% F1-score thorough rigorous experimentation and evaluation. We observe that our approach is highly efficient in detecting prompt injection attacks.
Related links
Details
- Title
- Fine-tuned Large Language Models (LLMs)
- Publication Details
- Proceedings: 2025 IEEE 49th Annual Computers, Software, and Applications Conference COMPSAC 2025, pp.1033-1039
- Resource Type
- Conference proceeding
- Conference
- Annual Computers, Software, and Applications Conference (COMPSAC), 49th (Toronto, Ontario, Canada, 07/08/2025–07/11/2025)
- Publisher
- IEEE; ieeexplore
- Series
- IEEE Annual International Computer Software and Applications Conference
- Number of pages
- 7
- Grant note
- National Science Foundation (NSF): 1946442, 2433800 National Institutes of Health (NIH): 9R42LM014356-02A1
This research is funded by the National Science Foundation (NSF) and the National Institutes of Health (NIH) (Award #9R42LM014356-02A1). Also, this work is supported by the National Science Foundation under NSF Awards #1946442 and #2433800. Any opinions, findings, recommendations, expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
- Copyright
- © 2025, IEEE
- Identifiers
- WOS:001575960000127; 99381483999306600
- Academic Unit
- Center for Cybersecurity
- Language
- English