Logistic Regression Details

Given n feature vectors of n p-dimensional feature vectors a vector of class labels y = (y 1,…,yn ), where yi ∈ {0, 1, ..., K - 1} and K is the number of classes, describes the class to which the feature vector xi belongs, the problem is to train a logistic regression model.

The logistic regression model is the set of vectors that gives the posterior probability


for a given feature vector x = (x 1 ,…,x p ) and class label y ∈ {0, 1, ..., K - 1} for each k =0,…,K-1. [Hastie2009].

If the categorical variable is binary, the model is defined as a single vector that determines the posterior probability , (2)

Training Stage

Training procedure is an iterative algorithm which minimizes objective function


where the first term is the negative log-likelihood of conditional Y given X, and the latter terms are regularization ones that penalize the complexity of the model (large β values), λ 1 and λ 2 are non-negative regularization parameters applied to L1 and L2 norm of vectors in β.

For more details, see [Hastie2009] [Bishop2006]

For the objective function minimization the library supports the iterative algorithms defined by the interface of daal::algorithms::iterative_solver. See Analysis > Optimization Solvers > Iterative Solvers

Prediction Stage

Given logistic regression model and vectors x 1, ..., x r , the problem is to calculate the responses for those vectors, and their probabilities and logarithms of probabilities if required. The computation is based on formula (1) in multinomial case and on formula (2) in binary case.

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)