Boosting Algorithms for Maximizing the Soft Margin
Manfred Warmuth, Karen Glocer, Gunnar R¨tsch a Exp erimental performance
0.05 0.04

(T15)

Corrective SoftBoost

SmoothBoost

Totally Corrective SoftBoost Adaptively optimizes soft margin 
log N 2
soft margin objective

0.03

LPBoost

0.02

iterations

0.01

for N examples & precision 

0

Totally Corrective SoftBoost
-0.01

-0.02 10
0

10

1

10

2

10

3

number of iterations

LPBo ost (N ) iterations for separable case to achieve   1 We force LPBoost to concentrate its distribution on single examples Holds regardless of the LP optimization algorithm Shows need for regularization

Counter +1 +1 +1 +1 -.98 -.97 -.97 -.97

Example -.95 -.93 -.95 -.93 -.95 -.93 -.95 -.93 +1 -.93 -.96 +1 -.95 -.94 -.95 -.93

-.91 -.91 -.91 -.91 -.91 -.91 +1 -.92

-.99 -.99 -.99 -.99 +.99 +.99 +.99 +.99