Batch Processing

Layer Input

The forward batch normalization layer accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm. For more details, see Algorithms.

Input ID

Input

data

Tensor of size n1 x n2 x ... x np that stores the input data for the forward batch normalization layer. This input can be an object of any class derived from Tensor.

weights

One-dimensional tensor of size nk that stores weights for scaling ω (k). This input can be an object of any class derived from Tensor.

biases

One-dimensional tensor of size nk that stores biases for shifting the scaled data β(k). This input can be an object of any class derived from Tensor.

populationMean

One-dimensional tensor of size nk that stores population mean μ computed in the previous stage. This input can be an object of any class derived from Tensor.

populationVariance

One-dimensional tensor of size nk that stores population variance s2 computed in the previous stage. This input can be an object of any class derived from Tensor.

Layer Parameters

For common parameters of neural network layers, see Common Parameters.

In addition to the common parameters, the forward batch normalization layer has the following parameters:

Parameter

Default Value

Description

algorithmFPType

float

The floating-point type that the algorithm uses for intermediate computations. Can be float or double.

method

defaultDense

Performance-oriented computation method, the only method supported by the layer.

alpha

0.01

Smoothing factor of the exponential moving average used to compute the population mean and variance.

epsilon

0.00001

Constant added to the mini-batch variance for numerical stability.

dimension

1

Index of dimension k for which normalization is performed.

Layer Output

The forward batch normalization layer calculates the result described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.

Result ID

Result

value

Tensor of size n1 x n2 x ... x np that stores the result of the forward batch normalization layer. This input can be an object of any class derived from Tensor.

resultForBackward

Collection of data needed for the backward batch normalization layer.

Element ID

Element

auxData

Tensor of size n1 x n2 x ... x np that stores the input data for the forward batch normalization layer. This input can be an object of any class derived from Tensor.

auxWeights

One-dimensional tensor of size nk that stores weights for scaling ω (k). This input can be an object of any class derived from Tensor.

auxMean

One-dimensional tensor of size nk that stores mini-batch mean μk. This input can be an object of any class derived from Tensor.

auxStandardDeviation

One-dimensional tensor of size nk that stores mini-batch standard deviation σ(k). This input can be an object of any class derived from Tensor.

auxPopulationMean

One-dimensional tensor of size nk that stores the resulting population mean μ. This input can be an object of any class derived from Tensor.

auxPopulationVariance

One-dimensional tensor of size nk that stores the resulting population variance s2. This input can be an object of any class derived from Tensor.

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)