Contents

# Ridge Regression

The ridge regression method is similar to the least squares procedure except that it penalizes the sizes of the regression coefficients. Ridge regression is one of the most commonly used methods to overcome data multicollinearity.

## Details

Let (
x
1
,…,
x
p
) be a vector of input variables and y=(
y
1
,…,
y
k
) be the response. For each
j
=1,…,
k,
the ridge regression model has the form similar to the linear regression model [Hoerl70], except that the coefficients are estimated by minimizing a different objective function [James2013]:
y
j
=
β
0
j
+
β
1
j
x
1
+...+
β
pj
x
p
.
Here
x
i
,
i
=1,...,
p
, are referred to as independent variables, and
y
j
are referred to as dependent variables or responses.
Training Stage
Let (
x
11
,...,
x
1
p
,
y
11
,...,
y
1
k
),…,(
x
n
1
,...,
x
np
,
y
n
1
,...,
y
n
k
) be a set of training data,
n
>>
p
. The matrix
X
of size
n
x
p
contains observations
x
ij
,
i
=1,...,n,
j
=1,….,
p
, of independent variables.
For each
y
j
,
j
= 1,…,
k,
the ridge regression estimates (
β
0
j
,
β
1
j
...,
β
pj
) by minimizing the objective function: where
λ
j
0 are ridge parameters [Hoerl70, James2013].
Prediction Stage
Ridge regression based prediction is done for input vector (
x
1
,…,
x
p
) using the equation
y
j
=
β
0
j
+
β
1
j
x
1
+...+
β
pj
x
p
for each
j
=1,…,
k
.

#### Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804