Developer Guide

Contents

Batch Processing

Layer Input

The forward batch normalization layer accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm. For more details, see Algorithms.
Input ID
Input
data
Tensor of size
n
1
x
n
2
x ... x
n
p
that stores the input data for the forward batch normalization layer. This input can be an object of any class derived from
Tensor
.
weights
One-dimensional tensor of size
n
k
that stores weights for scaling
ω
(
k
)
. This input can be an object of any class derived from
Tensor
.
biases
One-dimensional tensor of size
n
k
that stores biases for shifting the scaled data
β
(
k
)
. This input can be an object of any class derived from
Tensor
.
populationMean
One-dimensional tensor of size
n
k
that stores population mean
μ
computed in the previous stage. This input can be an object of any class derived from
Tensor
.
populationVariance
One-dimensional tensor of size
n
k
that stores population variance
s
2
computed in the previous stage. This input can be an object of any class derived from
Tensor
.

Layer Parameters

For common parameters of neural network layers, see Common Parameters.
In addition to the common parameters, the forward batch normalization layer has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
Performance-oriented computation method, the only method supported by the layer.
alpha
0.01
Smoothing factor of the exponential moving average used to compute the population mean and variance.
epsilon
0.00001
Constant added to the mini-batch variance for numerical stability.
dimension
1
Index of dimension
k
for which normalization is performed.

Layer Output

The forward batch normalization layer calculates the result described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.
Result ID
Result
value
Tensor of size
n
1
x
n
2
x ... x
n
p
that stores the result of the forward batch normalization layer. This input can be an object of any class derived from
Tensor
.
resultForBackward
Collection of data needed for the backward batch normalization layer.
Element ID
Element
auxData
Tensor of size
n
1
x
n
2
x ... x
n
p
that stores the input data for the forward batch normalization layer. This input can be an object of any class derived from
Tensor
.
auxWeights
One-dimensional tensor of size
n
k
that stores weights for scaling
ω
(
k
)
. This input can be an object of any class derived from
Tensor
.
auxMean
One-dimensional tensor of size
n
k
that stores mini-batch mean
μ
k
. This input can be an object of any class derived from
Tensor
.
auxStandardDeviation
One-dimensional tensor of size
n
k
that stores mini-batch standard deviation
σ
(
k
)
. This input can be an object of any class derived from
Tensor
.
auxPopulationMean
One-dimensional tensor of size
n
k
that stores the resulting population mean
μ
. This input can be an object of any class derived from
Tensor
.
auxPopulationVariance
One-dimensional tensor of size
n
k
that stores the resulting population variance
s
2
. This input can be an object of any class derived from
Tensor
.

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804