Developer Guide

Contents

Logistic Regression Details

Given
n
feature vectors of
n
p
-dimensional feature vectors a vector of class labels
y
= (
y
1
,…,
y
n
), where
y
i
∈ {0, 1, ...,
K
- 1} and
K
is the number of classes, describes the class to which the feature vector
x
i
belongs, the problem is to train a logistic regression model.
The logistic regression model is the set of vectors
that gives the posterior probability
(1)
for a given feature vector x = (x
1
,…,x
p
) and class label y ∈ {0, 1, ...,
K
- 1} for each k =0,…,K-1. [Hastie2009].
If the categorical variable is binary, the model is defined as a single vector
that determines the posterior probability
,
(2)

Training Stage

Training procedure is an iterative algorithm which minimizes objective function
,
where the first term is the negative log-likelihood of conditional Y given X, and the latter terms are regularization ones that penalize the complexity of the model (large
β
values),
λ
1
and
λ
2
are non-negative regularization parameters applied to L1 and L2 norm of vectors in β.
For more details, see [Hastie2009] [Bishop2006]
For the objective function minimization the library supports the iterative algorithms defined by the interface of
daal::algorithms::iterative_solver
. See Analysis > Optimization Solvers > Iterative Solvers

Prediction Stage

Given logistic regression model and vectors
x
1
, ...,
x
r
, the problem is to calculate the responses for those vectors, and their probabilities and logarithms of probabilities if required. The computation is based on formula (1) in multinomial case and on formula (2) in binary case.

Training Alternative

If you already have a set of pre-calculated coefficients {
β
ij
}, you can use the Model Builder class to get a trained Intel DAAL Logistic Regression model with these coefficients. After the model is built, you can proceed to the prediction stage.
The set of pre-calculated coefficients should contain either
K
vectors
β
i
, where
K
is the number of responses, or a single vector
β
0
(if there are only two classes). If the number of classes is equal to two, vector
β
0
contains coefficients corresponding to the class with label
y
=1. Each vector
β
i
should be either of dimension
p
+1 (for vectors that include an intercept) or of dimension
p
otherwise, where
p
is the number of features. Note that the intercept value is stored as the zeroth component of
β
i
.
For general information on using the Model Builder class, see Training and Prediction. For details on using the Model Builder class for Logistic Regression, see Usage of training alternative.

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804