Developer Guide

Contents

Fully-connected Backward Layer

The forward fully-connected layer computes values
for
n
input arguments
x
k
, weights
w
ki
, weights mask
s
ki
, and biases
b
i
, where
k ∈
{1, ...,
n
},
i ∈
{1, ...,
m
}, and
m
is the number of layer outputs. For more details, see Forward Fully-connected Layer.
The backward fully-connected layer computes the following values:
where
E
is the objective function used at the training stage, and g
j
is the input gradient computed on the preceding layer.

Problem Statement

Given:
  • p
    -dimensional tensor
    X
    of size
    n
    1
    x ... x
    n
    k
    ... x
    n
    p
  • p
    -dimensional tensor
    W
    of size
    n
    1
    x ... x
    n
    k
    -1
    x
    m x
    n
    k
    +1
    ... x
    n
    p
  • p
    -dimensional tensor
    S
    of size
    n
    1
    x ... x
    n
    k
    -1
    x
    m x
    n
    k
    +1
    ... x
    n
    p
  • 1-dimensional tensor
    B
    of size
    m
  • 2-dimensional tensor
    G
    of size
    n
    k
    x
    m
The problem is to compute:
  • The
    p
    -dimensional tensor
    Z
    of size
    n
    1
    x ... x
    n
    k
    ... x
    n
    p
    such that:
  • Values:
In the above formulas:

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804