Local-Contrast Normalization Backward Layer

For given dimensions k 1 of size n k1 , k 2 of size n k2 , and f different from k 1 and k 2, the forward local contrast normalization layer normalizes the input p-dimensional tensor XR n 1 x n 2 x ... x n p . For more details, see Forward Local Contrast Normalization Layer.

The library supports four-dimensional input tensors XR n 1 x n 2 x n 3 x n 4 .

Without loss of generality let's assume that backward local contrast normalization is applied to the last two dimensions. The backward local contrast normalization layer takes:

  • Four-dimensional tensor XR n 1 x n 2 x n 3 x n 4
  • Four-dimensional tensor GR n 1 x n 2 x n 3 x n 4 with the gradient computed on the preceding layer
  • Two-dimensional tensor KR m 1 x m2 that contains kernel parameters/weights of kernels, where m 1 n 3, m 2 n 4

The layer computes the four-dimensional value tensor ZR n 1 x n 2 x n 3 x n 4 :



Problem Statement

The computation depends on whether the dimension f is set:

  • Dimension f is set; let n 2 be the sum dimension:























































    Consequently:



  • Dimension f is not set:





















































Consequently:



For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)