pygsti.optimize.customlm
¶
Custom implementation of the LevenbergMarquardt Algorithm
Module Contents¶
Classes¶
The result from an optimization. 

An optimizer. Optimizes an objective function. 

A LevenbergMarquardt optimizer customized for GSTlike problems. 
Functions¶

An implementation of the LevenbergMarquardt leastsquares optimization algorithm customized for use within pyGSTi. 

Attributes¶
 pygsti.optimize.customlm._MACH_PRECISION = 1e12¶
 class pygsti.optimize.customlm.OptimizerResult(objective_func, opt_x, opt_f=None, opt_jtj=None, opt_unpenalized_f=None, chi2_k_distributed_qty=None, optimizer_specific_qtys=None)¶
Bases:
object
The result from an optimization.
 Parameters
objective_func (ObjectiveFunction) – The objective function that was optimized.
opt_x (numpy.ndarray) – The optimal argument (x) value. Often a vector of parameters.
opt_f (numpy.ndarray) – the optimal objective function (f) value. Often this is the leastsquares vector of objective function values.
opt_jtj (numpy.ndarray, optional) – the optimial dot(transpose(J),J) value, where J is the Jacobian matrix. This may be useful for computing approximate error bars.
opt_unpenalized_f (numpy.ndarray, optional) – the optimal objective function (f) value with any penalty terms removed.
chi2_k_distributed_qty (float, optional) – a value that is supposed to be chi2_k distributed.
optimizer_specific_qtys (dict, optional) – a dictionary of additional optimization parameters.
 class pygsti.optimize.customlm.Optimizer¶
Bases:
pygsti.baseobjs.nicelyserializable.NicelySerializable
An optimizer. Optimizes an objective function.
 class pygsti.optimize.customlm.CustomLMOptimizer(maxiter=100, maxfev=100, tol=1e06, fditer=0, first_fditer=0, damping_mode='identity', damping_basis='diagonal_values', damping_clip=None, use_acceleration=False, uphill_step_threshold=0.0, init_munu='auto', oob_check_interval=0, oob_action='reject', oob_check_mode=0, serial_solve_proc_threshold=100)¶
Bases:
Optimizer
A LevenbergMarquardt optimizer customized for GSTlike problems.
 Parameters
maxiter (int, optional) – The maximum number of (outer) interations.
maxfev (int, optional) – The maximum function evaluations.
tol (float or dict, optional) – The tolerance, specified as a single float or as a dict with keys {‘relx’, ‘relf’, ‘jac’, ‘maxdx’}. A single float sets the ‘relf’ and ‘jac’ elemments and leaves the others at their default values.
fditer (int optional) – Internally compute the Jacobian using a finitedifference method for the first fditer iterations. This is useful when the initial point lies at a special or singular point where the analytic Jacobian is misleading.
first_fditer (int, optional) – Number of finitedifference iterations applied to the first stage of the optimization (only). Unused.
damping_mode ({'identity', 'JTJ', 'invJTJ', 'adaptive'}) – How damping is applied. ‘identity’ means that the damping parameter mu multiplies the identity matrix. ‘JTJ’ means that mu multiplies the diagonal or singular values (depending on scaling_mode) of the JTJ (Fischer information and approx. hessaian) matrix, whereas ‘invJTJ’ means mu multiplies the reciprocals of these values instead. The ‘adaptive’ mode adaptively chooses a damping strategy.
damping_basis ({'diagonal_values', 'singular_values'}) – Whether the the diagonal or singular values of the JTJ matrix are used during damping. If ‘singular_values’ is selected, then a SVD of the Jacobian (J) matrix is performed and damping is performed in the basis of (right) singular vectors. If ‘diagonal_values’ is selected, the diagonal values of relevant matrices are used as a proxy for the the singular values (saving the cost of performing a SVD).
damping_clip (tuple, optional) – A 2tuple giving upper and lower bounds for the values that mu multiplies. If damping_mode == “identity” then this argument is ignored, as mu always multiplies a 1.0 on the diagonal if the identity matrix. If None, then no clipping is applied.
use_acceleration (bool, optional) – Whether to include a geodesic acceleration term as suggested in arXiv:1201.5885. This is supposed to increase the rate of convergence with very little overhead. In practice we’ve seen mixed results.
uphill_step_threshold (float, optional) – Allows uphill steps when taking two consecutive steps in nearly the same direction. The condition for accepting an uphill step is that (uphill_step_thresholdbeta)*new_objective < old_objective, where beta is the cosine of the angle between successive steps. If uphill_step_threshold == 0 then no uphill steps are allowed, otherwise it should take a value between 1.0 and 2.0, with 1.0 being the most permissive to uphill steps.
init_munu (tuple, optional) – If not None, a (mu, nu) tuple of 2 floats giving the initial values for mu and nu.
oob_check_interval (int, optional) – Every oob_check_interval outer iterations, the objective function (obj_fn) is called with a second argument ‘oob_check’, set to True. In this case, obj_fn can raise a ValueError exception to indicate that it is Out Of Bounds. If oob_check_interval is 0 then this check is never performed; if 1 then it is always performed.
oob_action ({"reject","stop"}) – What to do when the objective function indicates (by raising a ValueError as described above). “reject” means the step is rejected but the optimization proceeds; “stop” means the optimization stops and returns as converged at the last knowninbounds point.
oob_check_mode (int, optional) – An advanced option, expert use only. If 0 then the optimization is halted as soon as an attempt is made to evaluate the function out of bounds. If 1 then the optimization is halted only when a wouldbe accepted step is out of bounds.
serial_solve_proc_threshold (int optional) – When there are fewer than this many processors, the optimizer will solve linear systems serially, using SciPy on a single processor, rather than using a parallelized Gaussian Elimination (with partial pivoting) algorithm coded in Python. Since SciPy’s implementation is more efficient, it’s not worth using the parallel version until there are many processors to spread the work among.
 _to_nice_serialization(self)¶
 classmethod _from_nice_serialization(cls, state)¶
 run(self, objective, profiler, printer)¶
Perform the optimization.
 Parameters
objective (ObjectiveFunction) – The objective function to optimize.
profiler (Profiler) – A profiler to track resource usage.
printer (VerbosityPrinter) – printer to use for sending output to stdout.
 pygsti.optimize.customlm.custom_leastsq(obj_fn, jac_fn, x0, f_norm2_tol=1e06, jac_norm_tol=1e06, rel_ftol=1e06, rel_xtol=1e06, max_iter=100, num_fd_iters=0, max_dx_scale=1.0, damping_mode='identity', damping_basis='diagonal_values', damping_clip=None, use_acceleration=False, uphill_step_threshold=0.0, init_munu='auto', oob_check_interval=0, oob_action='reject', oob_check_mode=0, resource_alloc=None, arrays_interface=None, serial_solve_proc_threshold=100, x_limits=None, verbosity=0, profiler=None)¶
An implementation of the LevenbergMarquardt leastsquares optimization algorithm customized for use within pyGSTi.
This general purpose routine mimic to a large extent the interface used by scipy.optimize.leastsq, though it implements a newer (and more robust) version of the algorithm.
 Parameters
obj_fn (function) – The objective function. Must accept and return 1D numpy ndarrays of length N and M respectively. Same form as scipy.optimize.leastsq.
jac_fn (function) – The jacobian function (not optional!). Accepts a 1D array of length N and returns an array of shape (M,N).
x0 (numpy.ndarray) – Initial evaluation point.
f_norm2_tol (float, optional) – Tolerace for F^2 where F = `norm( sum(obj_fn(x)**2) ) is the leastsquares residual. If F**2 < f_norm2_tol, then mark converged.
jac_norm_tol (float, optional) – Tolerance for jacobian norm, namely if infn(dot(J.T,f)) < jac_norm_tol then mark converged, where infn is the infinitynorm and f = obj_fn(x).
rel_ftol (float, optional) – Tolerance on the relative reduction in F^2, that is, if d(F^2)/F^2 < rel_ftol then mark converged.
rel_xtol (float, optional) – Tolerance on the relative value of x, so that if d(x)/x < rel_xtol then mark converged.
max_iter (int, optional) – The maximum number of (outer) interations.
num_fd_iters (int optional) – Internally compute the Jacobian using a finitedifference method for the first num_fd_iters iterations. This is useful when x0 lies at a special or singular point where the analytic Jacobian is misleading.
max_dx_scale (float, optional) – If not None, impose a limit on the magnitude of the step, so that dx^2 < max_dx_scale^2 * len(dx) (so elements of dx should be, roughly, less than max_dx_scale).
damping_mode ({'identity', 'JTJ', 'invJTJ', 'adaptive'}) – How damping is applied. ‘identity’ means that the damping parameter mu multiplies the identity matrix. ‘JTJ’ means that mu multiplies the diagonal or singular values (depending on scaling_mode) of the JTJ (Fischer information and approx. hessaian) matrix, whereas ‘invJTJ’ means mu multiplies the reciprocals of these values instead. The ‘adaptive’ mode adaptively chooses a damping strategy.
damping_basis ({'diagonal_values', 'singular_values'}) – Whether the the diagonal or singular values of the JTJ matrix are used during damping. If ‘singular_values’ is selected, then a SVD of the Jacobian (J) matrix is performed and damping is performed in the basis of (right) singular vectors. If ‘diagonal_values’ is selected, the diagonal values of relevant matrices are used as a proxy for the the singular values (saving the cost of performing a SVD).
damping_clip (tuple, optional) – A 2tuple giving upper and lower bounds for the values that mu multiplies. If damping_mode == “identity” then this argument is ignored, as mu always multiplies a 1.0 on the diagonal if the identity matrix. If None, then no clipping is applied.
use_acceleration (bool, optional) – Whether to include a geodesic acceleration term as suggested in arXiv:1201.5885. This is supposed to increase the rate of convergence with very little overhead. In practice we’ve seen mixed results.
uphill_step_threshold (float, optional) – Allows uphill steps when taking two consecutive steps in nearly the same direction. The condition for accepting an uphill step is that (uphill_step_thresholdbeta)*new_objective < old_objective, where beta is the cosine of the angle between successive steps. If uphill_step_threshold == 0 then no uphill steps are allowed, otherwise it should take a value between 1.0 and 2.0, with 1.0 being the most permissive to uphill steps.
init_munu (tuple, optional) – If not None, a (mu, nu) tuple of 2 floats giving the initial values for mu and nu.
oob_check_interval (int, optional) – Every oob_check_interval outer iterations, the objective function (obj_fn) is called with a second argument ‘oob_check’, set to True. In this case, obj_fn can raise a ValueError exception to indicate that it is Out Of Bounds. If oob_check_interval is 0 then this check is never performed; if 1 then it is always performed.
oob_action ({"reject","stop"}) – What to do when the objective function indicates (by raising a ValueError as described above). “reject” means the step is rejected but the optimization proceeds; “stop” means the optimization stops and returns as converged at the last knowninbounds point.
oob_check_mode (int, optional) – An advanced option, expert use only. If 0 then the optimization is halted as soon as an attempt is made to evaluate the function out of bounds. If 1 then the optimization is halted only when a wouldbe accepted step is out of bounds.
resource_alloc (ResourceAllocation, optional) – When not None, an resource allocation object used for distributing the computation across multiple processors.
arrays_interface (ArraysInterface) – An object that provides an interface for creating and manipulating data arrays.
serial_solve_proc_threshold (int optional) – When there are fewer than this many processors, the optimizer will solve linear systems serially, using SciPy on a single processor, rather than using a parallelized Gaussian Elimination (with partial pivoting) algorithm coded in Python. Since SciPy’s implementation is more efficient, it’s not worth using the parallel version until there are many processors to spread the work among.
x_limits (numpy.ndarray, optional) – A (num_params, 2)shaped array, holding on each row the (min, max) values for the corresponding parameter (element of the “x” vector). If None, then no limits are imposed.
verbosity (int, optional) – Amount of detail to print to stdout.
profiler (Profiler, optional) – A profiler object used for to track timing and memory usage.
 Returns
x (numpy.ndarray) – The optimal solution.
converged (bool) – Whether the solution converged.
msg (str) – A message indicating why the solution converged (or didn’t).
 pygsti.optimize.customlm._hack_dx(obj_fn, x, dx, jac, jtj, jtf, f, norm_f)¶