Developer Guide

Contents

Two-Dimensional Stochastic Pooling Forward Layer

The forward two-dimensional (2D) stochastic pooling layer is a form of non-linear downsampling of an input tensor
X
R
n
1
x
n
2
x ... x
n
p
with non-negative elements
x
i
1
...
i
p
. 2D stochastic pooling partitions the input tensor data into 2D subtensors along dimensions
k
1
and
k
2
and applies the transformations (kernels).
At the training stage, the layer selects an element in each subtensor using sampling from a multinomial distribution. Probabilities required in the distribution are calculated by normalizing the subtensor. In the output, the selected element replaces the entire subtensor.
At the prediction stage, the layer replaces each subtensor with the weighted average of its elements using the probabilities as weights. For more details, see [Mattew2013].

Problem Statement

Given:
  • The tensor
    X
    R
    n
    1
    x
    n
    2
    x ... x
    n
    p
    of input data with non-negative elements
    x
    i
    1
    ...
    i
    p
  • Dimensions
    k
    1
    and
    k
    2
    along which kernels are applied
  • Kernel sizes
    m
    1
    and
    m
    2
    :
    where
    p
    1
    and
    p
    2
    are paddings
The problem is to compute the value tensor
Y
R
l
1
x ... x
l
p
using the downsampling technique, where:
and
s
1
and
s
2
are strides.
To define the values of
Y
, let's apply the kernel to the subtensor at position {
i
1
, ...,
i
p
} and consider elements of the subtensor as the vector
a
of size
with elements
The probabilities used in the multinomial distribution are defined as:
Training Stage
At the training stage, the layer computes the value
y
as the element at the position
t
in the subtensor. The position
t
is picked as a sample from the multinomial distribution and kept for use in the backward step:
Prediction Stage
At the prediction stage, the layer computes the value as a weighted average of elements of the vector
a
that represents the subtensor:

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804