2D Locally-connected Backward Layer

The forward two-dimensional (2D) locally-connected layer computes the value tensor Y by applying a set of nKernels 2D kernels K of size m 1 x m 2 to the input argument x. The library supports four-dimensional input tensors XR n 1 x n 2 x n 3 x n 4 . Therefore, the following formula applies:



where i + a < n 1, j + b < n 2, and r is the kernel index.

A set of kernels is specific to the selected dimensions of the input argument x.

For more details, see Forward 2D Locally-connected Layer.

The backward 2D locally-connected layer computes the derivatives of the objective function E with respect to the input argument, weights, and biases.

Problem Statement

Without loss of generality, let's assume that convolution kernels are applied to the last two dimensions.

Given:

  • Four-dimensional tensor GR n 1 x nKernels x l 3 x l 4 with the gradient from the preceding layer

  • Four-dimensional tensor XR n 1 x n 2 x n 3 x n 4 with input data of the forward layer

  • Six-dimensional tensor KR nKernels x l 3 x l 4 x m 2 x m 3 x m 4 with kernel parameters/weights

  • Three-dimensional tensor BR nKernels x l 3 x l 4 with the bias of each kernel.

For the above tensors:

  • and p i is the respective padding.

  • nGroups is defined as follows: let's assume that n 2 is the group dimension in the input tensor for the forward 2D locally-connected layer. The output gradient tensor is split along this dimension into nGroups groups, and the input gradient tensor and weights tensor are split into nGroups groups along the nKernels dimension. nKernels and n 2 must be multiples of nGroups.

The problem is to compute:

  • Four-dimensional tensor ZR n 1 x n 2 x n 3 x n 4 such that:









  • Values:





In the above formulas:

  • s 3 and s 4 are strides



For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)