Usage Model: Training and Prediction
- Provide the neural network with the input data for training. You can provide either one sample or a set of samples. ThebatchSizeparameter specifies the number of simultaneously processed samples.
- Computex=i+1f(x), where:i
- xis the input data for the layerii
- xis the output value of the layeri+1i
- f(ix) is the function corresponding to the layeri.
- i= 0, …,n-1 is the index of the layer
- Compute the input gradient for the penultimate layer as the gradient of the loss layergrad= ∇nf(lossx,n-1y).
- Computegrad= ∇if(ix)*igrad, where:i+1
- gradis the gradient obtained at theii-the layer
- gradis the gradient obtained at the (i+1i+1)-the layer
- i=n- 1, ..., 0
- Apply one of the optimization methods to the results of the previous step. Computew,b=optimizationSolver(w,b,grad), whereow= (wo,w1, ...,w),n-1b= (bo,b1, ...,b). For available optimization solver algorithms, see Optimization Solvers.n-1
- Clones all the forward layers of the training model except the loss layer.
- Replaces the loss layer with the layer returned by thegetLayerForPredictionmethod of the forward loss layer. For example, the loss softmax cross-entropy forward layer is replaced with the softmax forward layer.