Adaptive Subgradient Method

The adaptive subgradient method (AdaGrad) [Duchi2011] follows the algorithmic framework of an iterative solver with the algorithm-specific transformation T, set of intrinsic parameters S t defined for the learning rate η, and algorithm-specific vector U and power d of Lebesgue space defined as follows:

  1. where g i (θ t-1) is the i-th coordinate of the gradient g(θ t-1)

  2. where

Convergence check:

For more complete information about compiler optimizations, see our Optimization Notice.
Select sticky button color: 
Orange (only for download buttons)