HREF-IDS: a hybrid recursive feature elimination model for enhanced feature selection in network intrusion detection systems

A surge in Internet usage due to recent developments in emerging technologies (i.e., artificial intelligence, blockchain, cloud computing, and IoT) has significantly contributed to various sectors, especially in the healthcare industry. This high usage of the Internet and widespread adoption of these emerging technologies have significantly increased the network traffic volume. The increased volume of network traffic resulted in various cybersecurity challenges. The examples of such challenges include Denial-of-Service (DoS) attacks and malware propagation. These attacks motivated researchers and industry practitioners to propose solutions for network intrusion detection. Existing intrusion detection systems (IDS) for network security have addressed these challenges by monitoring and identifying malicious activities within networks or systems. Existing IDSs use machine learning-based methods to predict network traffic patterns. However, in the realm of network security, this study introduces a novel feature selection and extraction approach for intrusion detection systems. It proposes a hybrid recursive feature elimination (HREF) method, utilizing Random Forest (RF) and Extreme Gradient Boosting (XGB) models, to identify the most significant features that distinguish attacks from benign activities within a network. It considers the basic set theory operations to determine optimal common features. It trains and evaluates six established machine learning models, including Stochastic Gradient Descent (SGD), Naïve Bayes (NB), Decision Tree (DT), AdaBoost, Multi-Layer Perceptron (MLP), and Logistic Regression (LR), using selected features from the UNSW-NB15 dataset. The intersection approach assisted in identifying nine important features out of 42. The proposed enhanced network intrusion detection system (NIDS) achieves high precision in network traffic prediction. with an accuracy of 95.36%, using only nine selected features. In comparison to existing IDS methodologies, the model maintains competitive performance while reducing the feature set by approximately 79%, demonstrating its high efficiency and effectiveness.