BrownBoost Classifier
BrownBoost is a boosting classification algorithm.
It is more robust to noisy data sets than other boosting classification algorithms [Freund99].
BrownBoost is a binary classifier.
For a multi-class case, use Multi-class Classifier framework of the library.
Details
Given
of size
, where
describes the class
to which the feature vector
belongs, and a weak learner algorithm,
the problem is to build a two-class BrownBoost classifier.
n
feature vectors
p
and a vector of class labels
Training Stage
The model is trained using the Freund method [Freund01] as follows:
- Calculate
, where:
is an inverse error function,
is a target classification error of the algorithm defined as
is the error function,
is a hypothesis formulated by the
i-th weak learner,,
is the weight of the hypothesis.
- Set initial prediction values:
.
- Set “remaining timing”:
.
- Do for
until
- With each feature vector and its label of positive weight, associate
.
- Call the weak learner with the distribution defined by normalizing Lmath:W_i(x, y)to receive a hypothesis
.
- Solve the differential equationwith given boundary conditionst = 0and
to find
and
such that either
or
, where
is a given small constant needed to avoid degenerate cases.
- Update the prediction values:
.
- Update “remaining time”:
.
End do
The result of the model training is the array of
.
M
weak learners
Prediction Stage
Given the BrownBoost classifier and
,
the problem is to calculate the final classification confidence, a number from the interval
, using the rule:
r
feature vectors
Batch Processing
BrownBoost classifier follows the general workflow described in Classification Usage Model.
Training
For a description of the input and output, refer to Classification Usage Model.
At the training stage, a BrownBoost classifier has the following parameters:
Parameter | Default Value | Description |
---|---|---|
algorithmFPType | float | The floating-point type that the algorithm uses for intermediate computations. Can be float or double . |
method | defaultDense | The computation method used by the BrownBoost classifier. The only training method supported so far is the Y. Freund’s method. |
nClasses | 2 | The number of classes. |
weakLearnerTraining | DEPRECATED : Pointer to an object of the weak learner training classUSE INSTEAD : Pointer to an object of the classification stump training class | DEPRECATED : Pointer to the training algorithm of the weak learner. By default, a stump weak learner is used.USE INSTEAD : Pointer to the classifier training algorithm. Be default, a classification stump with gini split criterion is used. |
weakLearnerPrediction | DEPRECATED : Pointer to an object of the weak learner prediction classUSE INSTEAD : Pointer to an object of the classification stump prediction class | DEPRECATED : Pointer to the prediction algorithm of the weak learner. By default, a stump weak learner is used.USE INSTEAD : Pointer to the classifier prediction algorithm. Be default, a classification stump with gini split criterion is used. |
accuracyThreshold | 0.01 | BrownBoost training accuracy
|
maxIterations | 100 | The maximal number of iterations for the BrownBoost algorithm. |
newtonRaphsonAccuracyThreshold | Accuracy threshold of the Newton-Raphson method used underneath the BrownBoost algorithm. | |
newtonRaphsonMaxIterations | 100 | The maximal number of Newton-Raphson iterations in the algorithm. |
degenerateCasesThreshold | The threshold used to avoid degenerate cases. |
Prediction
For a description of the input and output, refer to Classification Usage Model.
At the prediction stage, a BrownBoost classifier has the following parameters:
Parameter | Default Value | Description |
---|---|---|
algorithmFPType | float | The floating-point type that the algorithm uses for intermediate computations. Can be float or double . |
method | defaultDense | Performance-oriented computation method, the only method supported by the BrownBoost classifier. |
nClasses | 2 | The number of classes. |
weakLearnerPrediction | DEPRECATED : Pointer to an object of the weak learner prediction classUSE INSTEAD : Pointer to an object of the classification stump prediction class | DEPRECATED : Pointer to the prediction algorithm of the weak learner. By default, a stump weak learner is used.USE INSTEAD : Pointer to the classifier prediction algorithm. Be default, a classification stump with gini split criterion is used. |
accuracyThreshold | 0.01 | BrownBoost training accuracy
|
Examples
C++ (CPU)
Batch Processing:
Java*
Python*
Batch Processing: