Fully Connected Forward Layer

The forward fully-connected layer computes values

for n input arguments x k , weights w ki , weights mask s ki , and biases b i , where k ∈ {1, ..., n}, i ∈ {1, ..., m}, and m is the number of layer outputs.

Problem Statement

Given:

  • Dimension k

  • p-dimensional tensor X of size n 1 x ... x n k ... x n p

  • p-dimensional tensor W of size n 1 x ... x n k-1 x m x n k+1 ... x n p

  • p-dimensional tensor S of size n 1 x ... x n k-1 x m x n k+1 ... x n p

  • 1-dimensional tensor B of size m

The problem is to compute the 2-dimensional tensor Y of size n k x m such that:

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)