Developer Guide

Contents

Least Absolute Shrinkage and Selection Operator

Least Absolute Shrinkage and Selection Operator (LASSO) is a method for modeling relationship between a dependent variable (which may be a vector) and one or more explanatory variables by fitting regularized least squares model. Trained LASSO model can produce sparse coefficients due to the use of
L
1
regularization term. LASSO regression is widely used in feature selection tasks. For example, in the field of compressed sensing it is used to effectively identify relevant features associated with the dependent variable from a few observations with a large number of features. LASSO regression is also used to overcome multicollinearity of feature vectors in the training data set.

Details

Let (
x
1
, ...,
x
p
) be a vector of input variables and
y
= (
y
1
, ...,
y
k
) be the response. For each
j
= 1, ...,
k
, the LASSO model has the form similar to linear and ridge regression model [ Hoerl70 ], except that the coefficients are trained by minimizing a regularized by
L
1
penalty mean squared error (MSE) objective function.
Here
x
i
,
i
= 1, ...,
p
are referred to as independent variables,
y
j
is referred to as dependent variable or response and
.
Training Stage
Let (
x
11
, ...,
x
1
p
,
y
11
, ...,
y
1
k
), ..., (
x
n
1
, ...,
x
np
,
y
n
1
, ...,
y
nk
) be a set of training data (for regression task,
n
>>
p
, and for feature selection
p
could be greater than
n
). The matrix
X
of size
n
x
p
contains observations
x
ij
,
i
= 1, ...,
n
,
j
= 1, ...,
p
of independent variables.
For each
y
j
,
j
= 1, ...,
k
, the LASSO regression estimates
by minimizing the objective function:
Where the first term is mean squared error function and the second one is regularization term that penalizes the
L
1
norm of vector
For more details, see [ Hastie2009 ].
By default, Coordinate Descent iterative algorithm is used for minimization of the objective function. SAGA solver is also applicable for minimization. See Analysis > Optimization Solvers > Iterative Solvers.
Prediction Stage
LASSO regression based prediction is done for input vector of independent variables (
x
1
, ...,
x
p
) using the equation
where

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804