Ensemble decision forest of RBF networks via hybrid feature clustering approach for high-dimensional data classification

By: Shadi Abpeykar, Mehdi Ghatee, & Hadi Zare

Published: Computational Statistics & Data Analysis Volume 131, March 2019, Pages 12-36.


Classification of the high-dimensional data is challenging due to the curse of dimensionality, heavy computational burden and decreasing precision of algorithms. In order to mitigate these effects, feature selection approaches that can determine an efficient subset of features are utilized in the processing. However, most of these techniques attain just one subset of non-redundant features including the best ones. Alternatively, clustering approaches can be used to find the most informative clusters of features instead of generating just a single subset. So called, Hybrid Feature Clustering (HFC) method is capable of maximizing the classification accuracy while keeping the amount of redundant features in each cluster low. The patterns of each cluster are classified by a neural tree that employs Radial Basis Function (RBF) for the nodes. Within each neural tree, a hierarchical approach is proposed to transfer the knowledge of synaptic weights from a parent RBF node to each child. A gating network is applied on the forest of these neural trees in order to aggregate the results. By assessing the classification accuracy and the computational complexity on high-dimensional datasets it can be shown that the proposed solution has outperformed the state of the art classifiers. Furthermore, the computational complexity and the convergence of this method are theoretically proven and the robustness analysis under noisy conditions is conducted.


  • Proposing a new ensemble method to classify high-dimensional datasets in a reasonable time.
  • Using HFC method to cluster features by maximizing accuracy and minimizing redundancy, based on a modified GA.
  • Developing a forest of neural trees with RBF nodes on the features of all of the clusters.
  • Transferring the knowledge of RBF synaptic weights to accelerate training of next RBF nodes.
  • Constructing a gating network to aggregate results of neural trees to improve accuracy.

Leave a comment

Your email address will not be published. Required fields are marked *