Given n feature vectors x 1=(x 11,…,x 1p ),..., x n =(x n1,…,x np ) of size p and a vector of class labels y=(y 1,…,y n ), where y i K = {0, ..., J-1} describes the class to which the feature vector x i belongs and J is the number of classes, the problem is to build a multi-class LogitBoost classifier.

Training Stage

The LogitBoost model is trained using the Friedman method [Friedman00].

Let y i,j = I {x i j} is the indicator that the i-th feature vector belongs to class j. The scheme below, which uses the stump weak learner, shows the major steps of the algorithm:

  1. Start with weights w ij = 1/n, F j (x) = 0, p j (x) = 1/J, i = 1,...,n, j =0,...,J-1

  2. For m=1,...,M


    • For j = 1,...,J
    • Do
      • (i) Compute working responses and weights in the j-th class:

        w ij = p i (x i )(1-p i (x i )), w ij = max(z ij ,Thr1)

        z ij = (y ij - p i (x i )) /w ij , z ij = min(max(z ij ,-Thr2),Thr2)

      • (ii) Fit the function f mj (x) by a weighted least-squares regression of z ij to x i with weights w ij using the stump-based approach.
    • End do

    End do

The result of the model training is a set of M stumps.

Prediction Stage

Given the LogitBoost classifier and r feature vectors x 1,…,x r , the problem is to calculate the labels argmax jF j (x) of the classes to which the feature vectors belong.

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)