Developer Guide

Contents

Xavier Initializer

A Xavier initializer is an initializer algorithm to initialize a
p
-dimensional tensor
W
R
n
1
x ... x
n
p
that represents weights and biases of the appropriate layer. The algorithm initializes this tensor with random numbers uniformly distributed on the interval [-
α
,
α
). The value of
α
is defined using the sizes of the
r
-dimensional input tensor
X
R
n
x
m
2
x... x
m
r
and
q
-dimensional value tensor
Y
R
n
x
k
2
x... x
k
q
for the layer:
where:
  • It is assumed without loss of generality that tensors
    X
    and
    Y
    have batch dimension of size
    n
For more details, see [Glorot2010].

Algorithm Parameters

In addition to common parameters of the initializer interface, a Xavier initializer has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
Performance-oriented computation method, the only method supported by the algorithm.

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804