RMAM: A Rule-based Filter Method with Modified Huffman Ranking for Variable Selection in Supervised Learning

Abstract

Most existing filter-based variable selection methods compute variable scores using static mathematical formulations over fixed contingency tables and do not provide principled cut-off mechanisms to distinguish influential from redundant variables thus leaving final subset selection to end users. To address this limitation, this paper proposes a new filter-based variable selection method called Rule Mining Analysis Method (RMAM). For classification tasks involving supervised learning, this method dynamically generates rules from the training data by iteratively selecting variable values that maximize information gain with respect to the target class. Each generated rule is assigned a weight based on rule coverage and uncovered training data instances, and variable scores are subsequently computed by aggregating the weights of all rules in which they appear. To automate variable subset determination, RMAM integrates a new Modified Huffman Ranker (MHR) that computes an adaptive cut-off threshold to distinguish essential from redundant variables. Experimental evaluation across 27 UCI benchmark datasets demonstrates that RMAM reduced variable search space by an average of 75.2% relative to the original datasets thereby outperforming Information Gain and Chi-Square methods which achieved average reductions of 44.7% and 50.0%, respectively. When evaluated using Naïve Bayes classification algorithm, RMAM achieved comparable or superior predictive performance on the majority of datasets while offering substantially smaller and more interpretable variable subsets. These results demonstrate RMAM’s effectiveness as a practical variable selection approach for supervised learning applications.