Local-Response Normalization Backward Layer

For a given dimension k ∈ {1, ..., p} of size n k , the forward local response normalization layer normalizes the input tensor XR n 1 x n 2 x ... x n p . For more details and notations, see Forward Local Response Normalization Layer.

For a dimension k ∈ {1, ..., p} of size n k , the backward local response normalization layer computes the value:

where:

  • g i 1...i p is the input gradient computed on the preceding layer

  • α, β, κ ∈ R

  • n is a positive integer number

Problem Statement

See [Krizh2012] for an exact definition of local response normalization.

Given p-dimensional tensors:

  • XR n 1 x n 2 x ... x n p of size n 1 x n 2 x ... x n p

  • GR n 1 x n 2 x ... x n p - the gradient computed on the preceding layer

The problem is to compute the p-dimensional tensor Z = (z i 1...i p ) ∈ R n 1 x n 2 x ... x n p .

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)