Proximal boosting and its acceleration - Université de Paris - Faculté des Sciences Access content directly
Preprints, Working Papers, ... Year : 2020

Proximal boosting and its acceleration

Abstract

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm when the empirical risk to minimize is not differentiable to introduce a novel boosting approach, called proximal boosting. Besides being motivated by non-differentiable optimization, the proposed algorithm benefits from Nesterov’s acceleration in the same way as gradient boosting [Biau et al., 2018]. This leads to a variant, called accelerated proximal boosting. Advantages of leveraging proximal methods for boosting are illustrated by numerical experiments on simulated and real-world data. In particular, we exhibit a favorable comparison over gradient boosting regarding convergence rate and prediction accuracy.
Fichier principal
Vignette du fichier
paper.pdf (1.9 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01853244 , version 1 (02-08-2018)
hal-01853244 , version 2 (22-01-2020)
hal-01853244 , version 3 (27-07-2021)
hal-01853244 , version 4 (29-11-2022)

Identifiers

  • HAL Id : hal-01853244 , version 2

Cite

Erwan Fouillen, Claire Boyer, Maxime Sangnier. Proximal boosting and its acceleration. 2020. ⟨hal-01853244v2⟩
357 View
310 Download

Share

Gmail Facebook X LinkedIn More