GD

Algorithm (Test)

\[\begin{split} \begin{array}{l} \; \\ \hline % \mathbf{\mbox{Inputs}} \color{white}{ \Large A} \\ % --- \begin{array}{llll} \mbox{OptProb} & : & \mbox{Routines associated with an optimization problem} & \left\{ \begin{array}{l} \bullet \; F(\mathbf{u}) \\ \bullet \; \nabla F(\mathbf{u}) \\ \bullet \; \mbox{etc.} \end{array} \right. \\ \alpha_0 & : & \mbox{Initial step-size} & \quad \\ \mbox{nIter} & : & \mbox{Maximum number of global iterations} & \quad \\ \mbox{hyperP} & : & \mbox{Routines associated with hyper-parameters selection} & \end{array} \\ \hline % % --- % \color{white}{ \Large A} \\ % --- % \mathbf{u}_0 = \mbox{OptProb}.initSol() \phantom{\Large A} \\ % \mathbf{\mbox{for }} k=0,1,\ldots \\ \begin{array}{lllll} & \mathbf{g} & = & \mbox{OptProb}.grad(\mathbf{u}_{k}) & \qquad \color{darkgray} \mbox{computes } \nabla F(\mathbf{u}_k)\\ & \alpha & = & \mbox{hyperP}.ss(k, \alpha_0, \mathbf{u}_{k}, \mathbf{g}) & \qquad \color{darkgray} \mbox{Given a policy, computes } \alpha \\ & \mathbf{u}_{k+1} & = & \mathbf{u}_k - \alpha \cdot \mathbf{g} & \qquad \color{darkgray} \mathbf{u}_k - \alpha \cdot F(\mathbf{u}_k) \end{array} \\ % \; \\ \hline % --- \end{array} \; \\ \end{split}\]