Developer Guide

Contents

Loss Softmax Cross-entropy Forward Layer

The loss softmax cross-entropy layer implements an interface of the loss layer.
For an input tensor
X
R
n
1
x
n
2
x ... x
n
k
x ... x
n
p
, selected dimension
k
of size
n
k
, and ground truth tensor
T
R
n
1
x
n
2
x ... x 1 x ... x
n
p
, the layer computes a one-dimensional tensor with the cross-entropy value:
where
s
m t
m
defined below is the probability that the sample
m
corresponds to the ground truth
t
m
.

Problem Statement

Given:
  • The
    p
    -dimensional tensor
    X
    = (
    x
    j
    1
    ...
    j
    k
    ...
    j
    p
    ) ∈
    R
    n
    1
    x
    n
    2
    x ... x
    n
    k
    x ... x
    n
    p
    with input data
  • The
    p
    -dimensional tensor
    T
    = (
    t
    j
    1
    ...
    j
    k
    ...
    j
    p
    ) ∈
    R
    n
    1
    x
    n
    2
    x ... x 1 x ... x
    n
    p
    that contains the values of ground truth, where
The problem is to compute a one-dimensional tensor
Y
R
1
such that:
The library uses the numerically stable formula for computing the probability value
s
j
1
...
j
k
-1
i
j
k
+1
..
j
p
:
where

Product and Performance Information

1

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804