Multiple Classifier Systems

A Multiple Classifier System (MCS) is a pattern classification system made up of an ensemble of individual classifiers whose outputs or decisions on an input sample are combined in some way to get a final decision on its classification.

The idea of classifier combination has been investigated (under several names) in the pattern recognition field since the late 70s, with the main motivation that MCSs can allow to overcome some known limitations of the traditional approach to classifier design (namely, using a monolithic classifier chosen as the best one for the application at hand, among a given set of available classification algorithms). To be very short, one of the key ideas is that it is often very difficult to find the real best classifier for the task at hand, while different classifiers designed for the same task (for instance, using the same base classifier trained on different training sets, or on different features, or using different kind of classifiers) can exhibit complementary strengths and weaknesses: a proper combination of an ensemble of different classifiers could therefore be more effective than using a single, monolithic classifier (even though the latter appears to be the best classifier among the available ones).

Relevant contributions to MCSs have been provided by the machine learning, neural networks, and statistics fields. Both theoretical and empirical evidence acquired in the past fifteen years led MCSs to become to date one of the main tools for the design of classification systems. Despite this, MCSs still exhibit several open issues, and therefore are still one of the main research topics in the pattern recognition field. A very good introduction to MCSs and a comprehensive overview of their state of the art can be found in the book by Ludmila I. Kuncheva, "Combining Pattern Classifiers: Methods and Algorithms," Hoboken (N.J.), Wiley, 2004.

The PRA Group works on MCSs since its foundation in 1996, and is the co-organizer (together with the research group of Prof. Josef Kittler [link sul nome alla pagina di Josef della sua università: prendilo dalla pagina dei VP] of the University of Surrey) of the International Workshop on Multiple Classifier Systems (see the Events section of this site).

Under the methodological viewpoint our work is related to two complementary approaches in the design of a MCS, namely how to construct an ensemble of classifiers to be used with a given combining rule (known in the literature as coverage optimization approach), and how to find a proper combining rule for a given ensemble of classifiers (decision optimization). In particular, our contributions on the latter issue are focused on the development of dynamic classifier selection rules, and to the analysis of the linear combination rule (see the pages under this section).

The main application of MCSs we addressed are the classification of remote-sensing images, and personal identity verification systems based on biometrics (see the corresponding pages under the Research section of this web site).

People working on this topic:

• Battista Biggio
• Giorgio Fumera
• Giorgio Giacinto
• Fabio Roli
• Roberto Tronci

Publications on Multiple Classifier Systems