Classifier Ensembles

Classifier Ensembles

• Friday 11, 10-13
• Wednesday 16, 11-13
• Friday 18, 10-13

Classifier ensembles are a growing area of pattern recognition and machine learning. This course will set up the pattern recognition background and will introduce the basics of classifier ensembles. Ensemble approaches and methods will be presented including the "classics" such as bagging, boosting, random subspace, random forest end ECOC ensembles, as well as more recent additions to the ensemble collection, e.g., rotation forest and random oracle ensembles. We will take a look at the ensemble design strategies and combination rules, and will discuss intuition and theory about why classifier ensembles work. Diversity among the classifiers in the ensemble will be explained, along with its relationship with the ensemble accuracy. Cluster ensembles will be briefly presented, and we will touch upon classifier ensembles for changing environments.
The course (8 lecture hours) will be roughly structured as shown below:

1. Pattern Recognition: basic concepts, evaluation and comparison of classifiers. Slides

2. Base Classifier Models. Slides

3. Combination Methods. Slides

4. Diversity in Classifier Ensembles. Slides

5. Building Classifier Ensembles: error correcting output codes (ECOC); classifier fusion and classifier selection; dynamic classifier selection; bagging, AdaBoost, random forests and random oracle ensembles. Slides

6. Feature Selection:  Selection and extraction of features for ensembles and from ensembles. Rotation Forest. Slides

7. Cluster ensembles: construction methods, aggregation of the individual decisions, diversity. Slides

8. Classifier Ensembles for Changing Environments: concept drift and types of changes; detecting a change; individual online classifiers; ensemble approaches for concept drift. Slides