Combining localized fusion and dynamic selection for high-performance SVM

Abstract

To resolve class-ambiguity in real world problems, we previously presented two different ensemble approaches with support vector machines (SVMs): multiple decision templates (MuDTs) and dynamic ordering of one-vs.-all SVMs (DO-SVMs). MuDTs is a classifier fusion method, which models intra-class variations as subclass templates. On the other hand, DO-SVMs is an ensemble method that dynamically selects proper SVMs to classify an input sample based on its class probability. In this paper, we newly propose a hybrid scheme of those two approaches to utilize their complementary properties. The localized fusion approach of MuDTs increases variance of the classification models while the dynamic selection scheme of DO-SVMs reduces the unbiased-variance, which causes incorrect prediction. We show the complementary properties of MuDTs and DO-SVMs with several benchmark datasets and verify the performance of the proposed method. We also test how much our method could increase its baseline accuracy by comparing with other combinatorial ensemble approaches. (C) 2014 Elsevier Ltd. All rights reserved.