Real-Time Information Security Situational Awareness in Big Data Networks Using an Improved C4.5 Decision Tree with Dynamic Feature Weighting and Hybrid Pruning

Abstract

With the rapid development of big data technology, network attacks are characterized by scale, concealment and intelligence. In this paper, an improved C4.5 decision tree algorithm (DW-C4.5) is proposed, and a real-time detection model is constructed by dynamic feature weighting (integrating random forest feature importance and information gain ratio optimization) and mixed pruning strategy (pre-pruning error rate threshold of 0.05+pruning cost complexity after pruning). Twelve kinds of attacks, such as DDoS, APT and zero-day exploitation, are tested on four public data sets (NSL-KDD, CIC-IDS2017 and UNSW-NB15) and one enterprise intranet log data set. The results show that the detection accuracy is 96.71%, which is 10.3 percentage points higher than that of traditional C4.5. The integrated Spark Streaming framework achieves a log throughput of 280,000 logs per second, and the false alarm rate is controlled below 3.12%. This method provides an efficient technical path for the dynamic security protection of massive network data.