Version: 2021.3
Last Updated: 06/28/2021
Public Content

Contents

Multi-class Classifier

While some classification algorithms naturally permit the use of more than two classes, some algorithms, such as Support Vector Machines (SVM), are by nature solving a two-class problem only. These two-class (or binary) classifiers can be turned into multi-class classifiers by using different strategies, such as One-Against-Rest or One-Against-One.

oneDAL implements a Multi-Class Classifier using the One-Against-One strategy.

Multi-class classifiers, such as SVM, are based on two-class classifiers, which are integral components of the models trained with the corresponding multi-class classifier algorithms.

Details

Given n
feature vectors $LaTex Math image.$ of size p
, the number of classes K
, and a vector of class labels $LaTex Math image.$ , where $LaTex Math image.$ , the problem is to build a multi-class classifier using a two-class (binary) classifier, such as a two-class SVM.

Training Stage

The model is trained with the One-Against-One method that uses the binary classification described in [Hsu02] as follows: For each pair of classes $LaTex Math image.$ , train a binary classifier, such as SVM. The total number of such binary classifiers is $LaTex Math image.$ .

Prediction Stage

Given a new feature vector $LaTex Math image.$ , the classifier determines the class to which the vector belongs.

oneDAL provides two methods for class label prediction:

Wu method. According to the algorithm 2 for computation of the class probabilities described in [Wu04]. The library returns the index of the class with the largest probability.

Vote-based method. If the binary classifier predicts the feature vector to be in i
-th class, the number of votes for the class i is increased by one, otherwise the vote is given to the j-th class. If two classes have equal numbers of votes, the class with the smallest index is selected.

Usage of Training Alternative

To build a Multi-class Classifier model using methods of the Model Builder class of Multi-class Classifier, complete the following steps:

Create a Multi-class Classifier model builder using a constructor with the required number of features and classes.

Use the

setTwoClassClassifierModel

method for each pair of classes to add the pre-trained two-class classifiers to the model. In the parameters to the method specify the classes’ indices and the pointer to the pre-trained two-class classifier for this pair of classes. You need to do this for each pair of classes, because the One-Against-One strategy is used.

Use the

getModel

method to get the trained Multi-class Classifier model.

Use the

getStatus

method to check the status of the model building process. If

DAAL_NOTHROW_EXCEPTIONS

macros is defined, the status report contains the list of errors that describe the problems API encountered (in case of API runtime failure).

Examples

oneAPI C++

Batch Processing

svm_two_class_thunder_dense_batch.cpp

C++ (CPU)

Batch Processing

svm_multi_class_model_builder.cpp

Java*

There is no support for Java on GPU.

Batch Processing

SVMMultiClassModelBuilder.java

Python*

svm_multi_class_model_builder.py

Batch Processing

Multi-class classifier follows the general workflow described in Classification Usage Model.

Training

At the training stage, a multi-class classifier has the following parameters:

Parameter	Default Value	Description
algorithmFPType	float	The floating-point type that the algorithm uses for intermediate computations. Can be float or double .
method	defaultDense	The computation method used by the multi-class classifier. The only training method supported so far is One-Against-One.
training	Pointer to an object of the SVM training class	Pointer to the training algorithm of the two-class classifier. By default, the SVM two-class classifier is used.
nClasses	Not applicable	The number of classes. A required parameter.

Prediction

At the prediction stage, a multi-class classifier has the following parameters:

Parameter	Method	Default Value	Description
algorithmFPType	defaultDense or voteBased	float	The floating-point type that the algorithm uses for intermediate computations. Can be float or double .
pmethod	Not applicable	defaultDense	Available methods for multi-class classifier prediction stage: defaultDense - the method described in [Wu04] voteBased - the method based on the votes obtained from two-class classifiers.
tmethod	defaultDense or voteBased	training::oneAgainstOne	The computation method that was used to train the multi-class classifier model.
prediction	defaultDense or voteBased	Pointer to an object of the SVM prediction class	Pointer to the prediction algorithm of the two-class classifier. By default, the SVM two-class classifier is used.
nClasses	defaultDense or voteBased	Not applicable	The number of classes. A required parameter.
maxIterations	defaultDense	100	The maximal number of iterations for the algorithm.
accuracyThreshold	defaultDense	1.0e-12	The prediction accuracy.
resultsToEvaluate	voteBased	computeClassLabels	The 64-bit integer flag that specifies which extra characteristics of the decision function to compute. Provide one of the following values to request a single characteristic or use bitwise OR to request a combination of the characteristics: computeClassLabels for prediction computeDecisionFunction for decisionFunction

Output

In addition to classifier output, multiclass classifier calculates the result described below. Pass the

Result ID

as a parameter to the methods that access the result of your algorithm. For more details, see Algorithms.

Result ID	Result
decisionFunction	A numeric table of size $LaTex Math image.$ containing the results of the decision function computed for all binary models when the computeDecisionFunction option is enabled.