Quality Metrics for Binary Classification Algorithms
For two classes
and
C2
, given a vector
of class labels
computed at the prediction stage of the classification algorithm and
a vector
of expected class labels, the problem is to evaluate the classifier
by computing the confusion matrix and connected quality metrics: precision, recall, and so on.QualityMetricsId
for binary classification is confusionMatrix
.Details
Further definitions use the following notations:
true positive  the number of correctly recognized observations for class
 
true negative  the number of correctly recognized observations that do not belong to the class
 
false positive  the number of observations that were incorrectly assigned to the class
 
false negative  the number of observations that were not recognized as belonging to the class

The library uses the following quality metrics for binary classifiers:
Quality Metric  Definition 

Accuracy  
Precision  
Recall  
Fscore  
Specificity  
Area under curve (AUC) 
For more details of these metrics, including the evaluation focus, refer to [Sokolova09].
The confusion matrix is defined as follows:
Classified as Class
 Classified as Class
 

Actual Class
 tp  fn 
Actual Class
 fp  tn 
Batch Processing
Algorithm Input
The quality metric algorithm for binary classifiers accepts the input described below.
Pass the
Input ID
as a parameter to the methods that provide input for your algorithm.
For more details, see Algorithms.Input ID  Input 

predictedLabels  Pointer to the
numeric table that contains labels computed at the prediction stage of the classification algorithm. This input can be an object of any class derived from NumericTable except PackedSymmetricMatrix , PackedTriangularMatrix , and CSRNumericTable . 
groundTruthLabels  Pointer to the
numeric table that contains expected labels. This input can be an object of any class derived from NumericTable except PackedSymmetricMatrix , PackedTriangularMatrix , and CSRNumericTable . 
Algorithm Parameters
The quality metric algorithm has the following parameters:
Parameter  Default Value  Description 

algorithmFPType  float  The floatingpoint type that the algorithm uses for intermediate computations. Can be float or double . 
method  defaultDense  Performanceoriented computation method, the only method supported by the algorithm. 
beta  1  The
parameter of the Fscore quality metric provided by the library. 
Algorithm Output
The quality metric algorithm calculates the result described below.
Pass the
Result ID
as a parameter to the methods that access the results of your algorithm.
For more details, see Algorithms.Result ID  Result 

confusionMatrix  Pointer to the
numeric table with the confusion matrix. By default, this result is an object of the HomogenNumericTable class,
but you can define the result as an object of any class derived from NumericTable except PackedTriangularMatrix , PackedSymmetricMatrix , and CSRNumericTable . 
binaryMetrics  Pointer to the
numeric table that contains quality metrics, which you can access by an appropriate Binary Metrics ID:
By default, this result is an object of the HomogenNumericTable class, but you can define the result as an object of any class
derived from NumericTable except PackedTriangularMatrix , PackedSymmetricMatrix , and CSRNumericTable . 
Examples
C++ (CPU)
Batch Processing:
Java*