Developer Guide and Reference

  • 2021.2
  • 03/26/2021
  • Public Content

Elastic Net

Elastic Net is a method for modeling relationship between a dependent variable (which may be a vector) and one or more explanatory variables by fitting regularized least squares model. Elastic Net regression model has the special penalty, a sum of L1 and L2 regularizations, that takes advantage of both Ridge Regression and LASSO algorithms. This penalty is particularly useful in a situation with many correlated predictor variables [Friedman2010].


Let LaTex Math image. be a vector of input variables and LaTex Math image. be the response. For each
j = 1, …, k
, the Elastic Net model has the form similar to linear and ridge regression models [Hoerl70] with one exception: the coefficients are estimated by minimizing mean squared error (MSE) objective function that is regularized by LaTex Math image. and LaTex Math image. penalties.
LaTex Math image.
Here LaTex Math image.,
i = 1, ldots, p
, are referred to as independent variables, LaTex Math image.,
j = 1, …, k
, is referred to as dependent variable or response.
Training Stage
Let LaTex Math image. be a set of training data (for regression task, LaTex Math image., and for feature selection
could be greater than
). The matrix
of size LaTex Math image. contains observations LaTex Math image.,
i = 1, …, n
j = 1, ldots, p
of independent variables.
For each LaTex Math image.,
j = 1, …, k
, the Elastic Net regression estimates LaTex Math image. by minimizing the objective function:
LaTex Math image.
In the equation above, the first term is a mean squared error function, the second and the third are regularization terms that penalize the LaTex Math image. and LaTex Math image. norms of vector LaTex Math image., where LaTex Math image., LaTex Math image.,
j = 1, …, k
For more details, see [Hastie2009] and [Friedman2010].
By default, Coordinate Descent iterative solver is used to minimize the objective function. SAGA solver is also applicable for minimization.
Prediction Stage
Prediction based on Elastic Net regression is done for input vector LaTex Math image. using the equation LaTex Math image. for each
j = 1, …, k

Product and Performance Information


Performance varies by use, configuration and other factors. Learn more at