source_localization.gradient_descent module

source_localization.gradient_descent.gradient_descent(fun, x0, args=(), options=None, callback=None, f_old=numpy.inf)[source]

Minimize a function \(f(x)\) using the gradient descent method.

The steps of the algorithme are the following:

  • Initialization: Starting from an initial point \(x_0\).

  • For each iteration \(k\):

    1. Compute the gradient of the function \(\nabla f(x_k)\) at the current point \(x_k\).

    2. Update the estimate of \(x\) using the rule \(x_{k+1} = x_k - \alpha \nabla f(x_k)\) where \(\alpha\) is the step size.

  • Convergence Check: the algorithm stops when the change in the function value or the gradient is below a predefined threshold, or when the maximum number of iterations is reached.

Parameters:

  • fun (callable) – Method that, given \(x\), returns the evaluation of the function \(f(x)\) and its gradient \(\nabla f(x)\).

  • x0 (ndarray) – Initial point \(x_0\) from which the iterative optimization algorithm starts.

  • args (tuple) – Arguments that should be given to the callable fun() besides the array containing the current value of \(x\).

  • options (dict, optional, default: None) –

    Dictionary containing several options to customize the gradient descent method and its stopping criteria. These options include:

    • ’step size’: float or int, default: 1, step size of the gradient descent \(\alpha\), also called learning rate.

    • ’nit_max’: int, default: 50, maximum number of iterations.

    • ’ftol’: float, default: -1, tolerance such that the algorithm terminates successfully when \(|f(x_{k+1})-f(x_{k})|<\text{ftol}\)

    • ’gtol’: float, default: -1, tolerance such that the algorithm terminates successfully when \(|\nabla f(x_{k})|<\text{gtol}\)

  • callback (callable, optional) – Method called at each optimization iteration. The method should take as sole input the array containing the current value of \(x\).

  • f_old (float, optional, default: numpy.inf) – Previous value of the function to optimize \(f(x)\) if the optimization process is restarted.

Returns:

  • x_opt (ndarray) – Optimal value \(x^*\).

  • f_opt (float) – Evaluation of the function to minimize at the optimal value \(f(x^*)\).

  • grad_opt (ndarray) – Evaluation of the gradient of the function to minimize at the optimal value \(\nabla f(x^*)\).

  • n_iter (int) – Number of iterations needed to converge.