Developer Guide and Reference

  • 2021.2
  • 03/26/2021
  • Public Content
Contents

Ridge Regression

The ridge regression method is similar to the least squares procedure except that it penalizes the sizes of the regression coefficients. Ridge regression is one of the most commonly used methods to overcome data multicollinearity.

Details

Let LaTex Math image. be a vector of input variables and LaTex Math image. be the response. For each LaTex Math image., the ridge regression model has the form similar to the linear regression model [Hoerl70], except that the coefficients are estimated by minimizing a different objective function [James2013]:
LaTex Math image.
Here LaTex Math image., LaTex Math image., are referred to as independent variables, and LaTex Math image. are referred to as dependent variables or responses.
Training Stage
Let LaTex Math image. be a set of training data, LaTex Math image.. The matrix
X
of size LaTex Math image. contains observations LaTex Math image., LaTex Math image., LaTex Math image., of independent variables.
For each LaTex Math image.,
j = 1, …, k
, the ridge regression estimates LaTex Math image. by minimizing the objective function:
LaTex Math image.
where LaTex Math image. are ridge parameters [Hoerl70], [James2013].
Prediction Stage
Ridge regression based prediction is done for input vector LaTex Math image. using the equation LaTex Math image. for each LaTex Math image..

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.