Developer Guide and Reference

  • 2021.3
  • 06/28/2021
  • Public Content
Contents

Regression Decision Forest

Decision forest regression is a special case of the Decision Forest model.

Details

Given:
  • n
    feature vectors LaTex Math image. of size
    p
    ;
  • their non-negative sample weights LaTex Math image.;
  • the vector of responses LaTex Math image.
The problem is to build a decision forest regression model that minimizes the Mean-Square Error (MSE) between the predicted and true value.
Training Stage
Decision forest regression follows the algorithmic framework of decision forest training algorithm based on the mean-squared error (MSE) [Breiman84]. If sample weights are provided as input, the library uses a weighted version of the algorithm.
MSE is an impurity metric (
D
is a set of observations that reach the node), calculated as follows:
Without sample weights
With sample weights
LaTex Math image.
LaTex Math image.
LaTex Math image., which is equivalent to the number of elements in
S
LaTex Math image.
Prediction Stage
Given decision forest regression model and vectors LaTex Math image., the problem is to calculate the responses for those vectors. To solve the problem for each given query vector LaTex Math image., the algorithm finds the leaf node in a tree in the forest that gives the response by that tree as the mean of dependent variables. The forest predicts the response as the mean of responses from trees.
Out-of-bag Error
Decision forest regression follows the algorithmic framework for calculating the decision forest out-of-bag (OOB) error, where aggregation of the out-of-bag predictions in all trees and calculation of the OOB error of the decision forest is done as follows:
  • For each vector LaTex Math image. in the dataset
    X
    , predict its response LaTex Math image. as the mean of prediction from the trees that contain LaTex Math image. in their OOB set:
    LaTex Math image., where LaTex Math image. and LaTex Math image. is the result of prediction LaTex Math image. by LaTex Math image..
  • Calculate the OOB error of the decision forest T as the Mean-Square Error (MSE):
    LaTex Math image.
  • If OOB error value per each observation is required, then calculate the prediction error for LaTex Math image.:
    LaTex Math image.

Batch Processing

Decision forest regression follows the general workflow described in Decision Forest.
Training
For the description of the input and output, refer to Regression Usage Model.
In addition to the decision forest parameters described in Batch Processing, the training algorithm for decision forest regression has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
The computation method used by the decision forest regression.
For CPU:
  • defaultDense
    - default performance-oriented method
  • hist
    - inexact histogram computation method
For GPU:
Output
In addition to the output of regression described in Regression Usage Model, decision forest regression calculates the result of decision forest. For more details, refer to Batch Processing.
Prediction
For the description of the input and output, refer to Regression Usage Model.
In addition to the parameters of regression, decision forest regression has the following parameters at the prediction stage:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
The computation method used by the decision forest regression. The only prediction method supported so far is the default dense method.
Examples
oneAPI DPC++
Batch Processing:
oneAPI C++
Batch Processing:
C++ (CPU)
Batch Processing:
Java*
There is no support for Java on GPU.
Batch Processing:
Python*
Batch Processing:

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.