Metrics
2 File views/ downloads
20 Record Views
Journal article
Model Retraining upon Concept Drift Detection in Network Traffic Big Data
Future internet, Vol.17(8), p.328
07/24/2025
Web of Science ID: WOS:001558323000001
Abstract
This paper presents a comprehensive model for detecting and addressing concept drift in network security data using the Isolation Forest algorithm. The approach leverages Isolation Forest’s inherent ability to efficiently isolate anomalies in high-dimensional data, making it suitable for adapting to shifting data distributions in dynamic environments.Anomalies in network attack data may not occur in large numbers, so it is important to be able to detect anomalies even with small batch sizes. The novelty of this work lies in successfully detecting anomalies even with small batch sizes and identifying the point at which incremental retraining needs to be started. Triggering retraining early also keeps the model in sync with the latest data, reducing the chance for attacks to be successfully conducted. Our methodology implements an end-to-end workflow that continuously monitors incoming data and detects distribution changes using Isolation Forest, then manages model retraining using Random Forest to maintain optimal performance. We evaluate our approach using UWF-ZeekDataFall22, a newly created dataset that analyzes Zeek’s Connection Logs collected through Security Onion 2 network security monitor and labeled using the MITRE ATT&CK framework. Incremental as well as full retraining are analyzed using Random Forest. There was a steady increase in the model’s performance with incremental retraining and a positive impact on the model’s performance with full model retraining.
Files and links (2)
Related links
Details
- Title
- Model Retraining upon Concept Drift Detection in Network Traffic Big Data
- Publication Details
- Future internet, Vol.17(8), p.328
- Resource Type
- Journal article
- Publisher
- MDPI
- Number of pages
- 23
- Grant note
- US National Center for Academic Excellence in Cybersecurity (NCAE): 2021 NCAE-C-002 US National Center for Academic Excellence in Cybersecurity: H98230-21-1-0170 Cyber Research Innovation Grant ProgramAskew Institute at the University of West Florida, USA
The research was funded by the US National Center for Academic Excellence in Cybersecurity (NCAE), 2021 NCAE-C-002: Cyber Research Innovation Grant Program, Grant Number: H98230-21-1-0170. This research was also partially supported by the Askew Institute at the University of West Florida, USA.
- Copyright
- © 2025 by the authors.
- Identifiers
- WOS:001558323000001; 99381463211206600
- Academic Unit
- Mathematics and Statistics; Computer Science; Hal Marcus College of Science and Engineering
- Language
- English