For given dimensions k 1 of size n k1 , k 2 of size n k2 , and f different from k 1 and k 2, the forward local contrast normalization layer normalizes the input p-dimensional tensor X ∈ R n 1 x n 2 x ... x n p . For more details, see Forward Local Contrast Normalization Layer.
The library supports four-dimensional input tensors X∈ R n 1 x n 2 x n 3 x n 4 .
Without loss of generality let's assume that backward local contrast normalization is applied to the last two dimensions. The backward local contrast normalization layer takes:
- Four-dimensional tensor X∈ R n 1 x n 2 x n 3 x n 4
- Four-dimensional tensor G ∈ R n 1 x n 2 x n 3 x n 4 with the gradient computed on the preceding layer
- Two-dimensional tensor K ∈ R m 1 x m2 that contains kernel parameters/weights of kernels, where m 1≤ n 3, m 2≤ n 4
The layer computes the four-dimensional value tensor Z∈ R n 1 x n 2 x n 3 x n 4 :
The computation depends on whether the dimension f is set:
Dimension f is set; let n 2 be the sum dimension:
Dimension f is not set: