Regression analysis is widely used to predict and forecast, as well as to understand which of the independent variables are related to dependent variables, along with the form of the relationship. The regression function, which is estimated based on the training data, defines the form of the relationship.
Regression methods are divided into the following groups:
where the regression function is defined in terms of a finite number of unknown parameters. For example: Linear Regression or Linear Least Squares.
where the regression function is constructed from a set of kernel functions that may be infinite-dimensional. For example: Non-Parametric Multiplicative Regression or Regression Trees.