Combining rules

One of our contributions was the proposal of combining rules based on the dynamic classifier selection (DCS) approach. DCS rules consists in assigning an input sample to the class provided by one of the classifiers of a given ensemble, instead of combining the outputs of all the individual classifiers. Such classifier is chosen at operation phase ("dynamically") as the one exhibiting the highest local accuracy in a properly selected neighborhood of the input sample.

Another contribution was a theoretical analysis of the linear combination rule (simple and weighted averaging of classifier outputs), based on an analytical framework developed in works by K. Tumer and J. Ghosh and further extended by us. Our analysis improved the understanding of the linear combining rule and provided some practical guidelines for the design of linear combiners. In particular, these guidelines refer to the choice between the simple and the weighted averaging rule, and to the choice of the ensemble size in Bagging and in similar methods based on randomization for the construction of classifier ensembles. Our results were also applied to the development of an ensemble learning method for linearly combined neural network classifiers, based on the negative correlation learning approach previously proposed by other authors for regression problems.

We also proposed a method for improving the effectiveness of the Behavior Knowledge Space combining rule in the case of small training sample size, and provided some experimental comparisons between fixed and trained rules, with applications to biometric multimodal personal identity verification systems.

People working on this topic:

• Battista Biggio
• Luca Didaci
• Giorgio Fumera
• Giorgio Giacinto
• Fabio Roli
• Roberto Tronci