New Insights into Decision Trees Ensembles.

Summary Tree ensembles are currently one of the most powerful statistical learning methods. However, their theoretical properties, as well as their empirical performance, are still subject to many questions. In this thesis, we propose to shed new light on these methods. More specifically, after having discussed the current theoretical aspects (chapter 1) of three main set schemes (Random Forests, Boosting and Stochastic Discrimination), we will propose an analysis tending towards the existence of a common point for the soundness of these three principles (chapter 2). This principle takes into account the importance of the first two moments of the margin in obtaining an ensemble with good performance. From this, we derive a new algorithm called OSS (Oriented Sub-Sampling) whose steps are in full agreement and follow logically from the framework we introduce. The performance of OSS is empirically superior to that of popular algorithms such as Random Forests and AdaBoost. In a third section (Chapter 3), we analyze the Random Forests method by adopting a "kernel" point of view. The latter allows to improve the understanding of the forests with, in particular, the understanding and observation of the regularization mechanism of these techniques. Adopting a kernel point of view allows us to improve Random Forests via popular post-processing methods such as SVM or multiple kernel learning. These methods show significantly better performance than the basic algorithm, and also allow for pruning the ensemble by keeping only a small part of the classifiers.