Developer Guide

Contents

Elastic Net

Elastic Net is a method for modeling relationship between a dependent variable (which may be a vector) and one or more explanatory variables by fitting regularized least squares model. Elastic Net regression model has the special penalty, a sum of
L
1
and
L
2
regularizations, hat takes advantage of both Ridge Regression and LASSO algorithms. This penalty is particularly useful in a situation with many correlated predictor variables [Friedman2010].

Details

Let (
x
1
, ...,
x
p
) be a vector of input variables and
y
= (
y
1
, ...,
y
k
) be the response. For each
j
= 1, ...,
k
, the Elastic Net model has the form similar to linear and ridge regression models [Hoerl70] with one exception: the coefficients are estimated by minimizing mean squared error (MSE) objective function that is regularized by
L
1
and
L
2
penalties.
Here
x
i
,
i
= 1, ...,
p
are referred to as independent variables,
y
j
,
is referred to as dependent variable or response.
Training Stage
Let (
x
11
, ...,
x
1
p
,
y
11
, ...,
y
1
k
), ..., (
x
n
1
, ...,
x
np
,
y
n
1
, ...,
y
nk
) be a set of training data (for regression task,
n
>>
p
, and for feature selection
p
could be greater than
n
). The matrix
X
of size
n
x
p
contains observations
x
ij
,
i
= 1, ...,
n
,
j
= 1, ...,
p
of independent variables.
For each
y
j
,
j
= 1, ...,
k
, the Elastic Net regression estimates
by minimizing the objective function:
In the equation above, the first term is a mean squared error function, the second and the third are regularization terms that penalize the
L
1
and
L
2
norms of vector
, where
For more details, see [Hastie2009] and [Friedman2010].
By default, Coordinate Descent iterative algorithm is used for minimization of the objective function. SAGA solver is also applicable for minimization. See Analysis > Optimization Solvers > Iterative Solvers.
Prediction Stage
Prediction based on Elastic Net regression is done for input vector (
x
1
, ...,
x
p
) using the equation
where

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804