:py:mod:`pygsti.layouts.copalayout` =================================== .. py:module:: pygsti.layouts.copalayout .. autoapi-nested-parse:: A object representing the indexing into a (flat) array of circuit outcome probabilities. Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: pygsti.layouts.copalayout.CircuitOutcomeProbabilityArrayLayout .. py:class:: CircuitOutcomeProbabilityArrayLayout(circuits, unique_circuits, to_unique, elindex_outcome_tuples, unique_complete_circuits=None, param_dimensions=(), resource_alloc=None) Bases: :py:obj:`pygsti.baseobjs.nicelyserializable.NicelySerializable` The arrangement of circuit outcome probabilities into a 1D array. This class describes how the outcome probabilities for a list of circuits map to the elements of a one-dimensional array. Computation, e.g., of an objective function such as the log-likelihood, is performed using the 1D array, and the layout is then used to assign meaning to each of the array elements, i.e., to lookup which element corresponds to a given circuit and outcome. This could be a simple concatenation of all the possible outcomes for each circuit in turn. However, when not all outcomes are observed it is unnecessary to compute the corresponding probabilities, and so this layout can be "sparse" in this sense. When there are multiple processors, a layout may assign different outcome probabilities (and their derivatives) to different processors. Thus, a layout can be dependent on the available processors and holds (and "owns") a :class:`ResourceAllocation` object. This class creates a non-distributed layout that is simply duplicated across all the available processors. Parameters ---------- circuits : list of Circuits The circuits whose outcome probabilities are to be computed. This list may contain duplicates. unique_circuits : list of Circuits The same as `circuits`, except duplicates are removed. Often this value is obtained by a derived class calling the class method :meth:`_compute_unique_circuits`. to_unique : dict A mapping that translates an index into `circuits` to one into `unique_circuits`. Keys are the integers 0 to `len(circuits)` and values are indices into `unique_circuits`. elindex_outcome_tuples : collections.OrderedDict A dictionary whose keys are integer indices into `unique_circuits` and whose values are lists of `(element_index, outcome_label)` tuples that give the element index within the 1D array of the probability (or other quantity) corresponding to the given circuit and outcome label. Note that outcome labels themselves are tuples of instrument/POVM member labels. unique_complete_circuits : list, optional A list, parallel to `unique_circuits`, that contains the "complete" version of these circuits. This information is currently unused, and is included for potential future expansion and flexibility. param_dimensions : tuple, optional A tuple containing, optionally, the parameter-space dimension used when taking first and second derivatives with respect to the circuit outcome probabilities. This is meta-data bundled along with the main layout information, and is needed for allocating arrays with derivative dimensions. resource_alloc : ResourceAllocation, optional The resources available for computing circuit outcome probabilities. Attributes ---------- num_elements : int The total number of elements in this layout. In a multi-processor context, the number of elements locally owned by the current processor. global_layout : CircuitOutcomeProbabilityArrayLayout A layout containing all the circuits in their original order, that is the same on all processors and doesn't depend on a specific resource allocation. This is either the layout itself or a larger layout that this layout is a part of. .. py:property:: num_elements The total number of elements in this layout. In a multi-processor context, the number of elements locally owned by the current processor. .. py:property:: num_circuits The total number of circuits in this layout. In a multi-processor context, the number of circuits locally owned by the current processor. .. py:property:: global_layout A layout containing all the circuits in their original order, that is the same on all processors and doesn't depend on a specific resource allocation. This is either the layout itself or a larger layout that this layout is a part of. .. py:attribute:: circuits .. py:method:: create_from(circuits, model=None, dataset=None, param_dimensions=(), resource_alloc=None) :classmethod: Creates a simple layout from a list of circuits. Optionally, a model can be used to "complete" (add implied prep or POVM layers) circuits, and a dataset to restrict the layout's elements to the observed outcomes. Parameters ---------- circuits : list of Circuits The circuits to include in the layout. Note that the produced layout may not retain the ordering of these circuits internally, but that it's `.global_layout` does. model : Model, optional A model used to "complete" the circuits (add implied prep and/or POVM layers). Usually this is a/the model that will be used to compute outcomes probabilities using this layout. If `None`, then each element of `circuits` is assumed to be a complete circuit, i.e., to begin with a state preparation layer and end with a POVM layer. dataset : DataSet, optional If not None, restrict what is simplified to only those probabilities corresponding to non-zero counts (observed outcomes) in this data set. param_dimensions : tuple, optional A tuple containing, optionally, the parameter-space dimension used when taking first and second derivatives with respect to the circuit outcome probabilities. resource_alloc : ResourceAllocation, optional The resources available for computing circuit outcome probabilities. Returns ------- CircuitOutcomeProbabilityArrayLayout .. py:method:: allocate_local_array(array_type, dtype, zero_out=False, memory_tracker=None, extra_elements=0) Allocate an array that is distributed according to this layout. Creates an array for holding elements and/or derivatives with respect to model parameters, possibly distributed among multiple processors as dictated by this layout. Parameters ---------- array_type : {"e", "ep", "ep2", "epp", "p", "jtj", "jtf", "c", "cp", "cp2", "cpp"} The type of array to allocate, often corresponding to the array shape. Let `nE` be the layout's number of elements, `nP1` and `nP2` be the number of parameters we differentiate with respect to (for first and second derivatives), and `nC` be the number of circuits. Then the array types designate the following array shapes: - `"e"`: (nE,) - `"ep"`: (nE, nP1) - `"ep2"`: (nE, nP2) - `"epp"`: (nE, nP1, nP2) - `"p"`: (nP1,) - `"jtj"`: (nP1, nP2) - `"jtf"`: (nP1,) - `"c"`: (nC,) - `"cp"`: (nC, nP1) - `"cp2"`: (nC, nP2) - `"cpp"`: (nC, nP1, nP2) Note that, even though the `"p"` and `"jtf"` types are the same shape they are used for different purposes and are distributed differently when there are multiple processors. The `"p"` type is for use with other element-dimentions-containing arrays, whereas the `"jtf"` type assumes that the element dimension has already been summed over. dtype : numpy.dtype The NumPy data type for the array. zero_out : bool, optional Whether the array should be zeroed out initially. memory_tracker : ResourceAllocation, optional If not None, the amount of memory being allocated is added, using :meth:`add_tracked_memory` to this resource allocation object. extra_elements : int, optional The number of additional "extra" elements to append to the element dimension, beyond those called for by this layout. Such additional elements are used to store penalty terms that are treated by the objective function just like usual outcome-probability-type terms. Returns ------- numpy.ndarray .. py:method:: free_local_array(local_array) Frees an array allocated by :meth:`allocate_local_array`. This method should always be paired with a call to :meth:`allocate_local_array`, since the allocated array may utilize shared memory, which must be explicitly de-allocated. Parameters ---------- local_array : numpy.ndarray or LocalNumpyArray The array to free, as returned from `allocate_local_array`. Returns ------- None .. py:method:: gather_local_array_base(array_type, array_portion, extra_elements=0, all_gather=False, return_shared=False) Gathers an array onto the root processor or all the processors. Gathers the portions of an array that was distributed using this layout (i.e. according to the host_element_slice, etc. slices in this layout). This could be an array allocated by :meth:`allocate_local_array` but need not be, as this routine does not require that `array_portion` be shared. Arrays can be 1, 2, or 3-dimensional. The dimensions are understood to be along the "element", "parameter", and "2nd parameter" directions in that order. Parameters ---------- array_type : ("e", "ep", "ep2", "epp", "p", "jtj", "jtf", "c", "cp", "cp2", "cpp") The type of array to allocate, often corresponding to the array shape. See :meth:`allocate_local_array` for a more detailed description. array_portion : numpy.ndarray The portion of the final array that is local to the calling processor. This could be a shared memory array, but just needs to be of the correct size. extra_elements : int, optional The number of additional "extra" elements to append to the element dimension, beyond those called for by this layout. Should match usage in :meth:`allocate_local_array`. all_gather : bool, optional Whether the result should be returned on all the processors (when `all_gather=True`) or just the rank-0 processor (when `all_gather=False`). return_shared : bool, optional Whether the returned array is allowed to be a shared-memory array, which results in a small performance gain because the array used internally to gather the results can be returned directly. When `True` a shared memory handle is also returned, and the caller assumes responsibilty for freeing the memory via :func:`pygsti.tools.sharedmemtools.cleanup_shared_ndarray`. Returns ------- gathered_array : numpy.ndarray or None The full (global) output array on the root (rank=0) processor and `None` on all other processors, unless `all_gather == True`, in which case the array is returned on all the processors. shared_memory_handle : multiprocessing.shared_memory.SharedMemory or None Returned only when `return_shared == True`. The shared memory handle associated with `gathered_array`, which is needed to free the memory. .. py:method:: gather_local_array(array_type, array_portion, extra_elements=0, return_shared=False) Gathers an array onto the root processor. Gathers the portions of an array that was distributed using this layout (i.e. according to the host_element_slice, etc. slices in this layout). This could be an array allocated by :meth:`allocate_local_array` but need not be, as this routine does not require that `array_portion` be shared. Arrays can be 1, 2, or 3-dimensional. The dimensions are understood to be along the "element", "parameter", and "2nd parameter" directions in that order. Parameters ---------- array_portion : numpy.ndarray The portion of the final array that is local to the calling processor. This could be a shared memory array, but just needs to be of the correct size. extra_elements : int, optional The number of additional "extra" elements to append to the element dimension, beyond those called for by this layout. Should match usage in :meth:`allocate_local_array`. return_shared : bool, optional If `True` then, when shared memory is being used, the shared array used to accumulate the gathered results is returned directly along with its shared-memory handle (`None` if shared memory isn't used). This results in a small performance gain. Returns ------- result : numpy.ndarray or None The full (global) output array on the root (rank=0) processor and `None` on all other processors. shared_memory_handle : multiprocessing.shared_memory.SharedMemory or None Returned only when `return_shared == True`. The shared memory handle associated with `result`, which is needed to free the memory. .. py:method:: allgather_local_array(array_type, array_portion, extra_elements=0, return_shared=False) Gathers an array onto all the processors. Gathers the portions of an array that was distributed using this layout (i.e. according to the host_element_slice, etc. slices in this layout). This could be an array allocated by :meth:`allocate_local_array` but need not be, as this routine does not require that `array_portion` be shared. Arrays can be 1, 2, or 3-dimensional. The dimensions are understood to be along the "element", "parameter", and "2nd parameter" directions in that order. Parameters ---------- array_portion : numpy.ndarray The portion of the final array that is local to the calling processor. This could be a shared memory array, but just needs to be of the correct size. extra_elements : int, optional The number of additional "extra" elements to append to the element dimension, beyond those called for by this layout. Should match usage in :meth:`allocate_local_array`. return_shared : bool, optional If `True` then, when shared memory is being used, the shared array used to accumulate the gathered results is returned directly along with its shared-memory handle (`None` if shared memory isn't used). This results in a small performance gain. Returns ------- result : numpy.ndarray or None The full (global) output array. shared_memory_handle : multiprocessing.shared_memory.SharedMemory or None Returned only when `return_shared == True`. The shared memory handle associated with `result`, which is needed to free the memory. .. py:method:: allsum_local_quantity(typ, value, use_shared_mem='auto') Sum a local array (or scalar) distributed using this layout. Sums per-element or per-circuit values across the processors used by this layout. Each array must be the same size, but need not be allocated in any particular way. Parameters ---------- typ : {"e", "c"} Whether the array is an element or circuit array. value : float or numpy.ndarray The value to sum. Must be the same size on all processors. use_shared_mem : bool, optional If `True` then, a tempoary shared memory array is allocated and used for the sum (when shared memory is available). For large arrays, using shared memory is faster than MPI communication, but for small arrays the overhead of creating the shared memory negates these gains. This argument must be `False` when `value` is just a float. Returns ------- numpy.ndarray or float The summed value, returned on all the processors. .. py:method:: fill_jtf(j, f, jtf) Calculate the matrix-vector product `j.T @ f`. Here `j` is often a jacobian matrix, and `f` a vector of objective function term values. `j` and `f` must be local arrays, created with :meth:`allocate_local_array`. This function performs any necessary MPI/shared-memory communication when the arrays are distributed over multiple processors. Parameters ---------- j : LocalNumpyArray A local 2D array (matrix) allocated using `allocate_local_array` with the `"ep"` (jacobian) type. f : LocalNumpyArray A local array allocated using `allocate_local_array` with the `"e"` (element array) type. jtf : LocalNumpyArray The result. This must be a pre-allocated local array of type `"jtf"`. Returns ------- None .. py:method:: fill_jtj(j, jtj) Calculate the matrix-matrix product `j.T @ j`. Here `j` is often a jacobian matrix, so the result is an approximate Hessian. This function performs any necessary MPI/shared-memory communication when the arrays are distributed over multiple processors. Parameters ---------- j : LocalNumpyArray A local 2D array (matrix) allocated using `allocate_local_array` with the `"ep"` (jacobian) type. jtj : LocalNumpyArray The result. This must be a pre-allocated local array of type `"jtj"`. Returns ------- None .. py:method:: memory_estimate(array_type, dtype='d') Memory required to allocate an array of a given type (in bytes). Parameters ---------- array_type : {'e', 'ep', 'epp'} The type of array. This string specifies the shape of the array, with `'e'` indicating dimension holding the layout's elements and `'p'` indicating parameter dimensions. dtype : numpy.dtype The NumPy data type for the array. Returns ------- int The memory that would be required, in bytes. .. py:method:: indices(circuit) The element indices corresponding to a circuit in this layout. This is a slice into the element-dimension of arrays allocated using this layout, e.g. an `'e'`-type array allocated by :meth:`allocate_local_array`. The entries of such an array correspond to different outcomes of this circuit, which are separately given by :meth:`outcomes` or alongside the indices in :meth:`indices_and_outcomes`. Parameters ---------- circuit : Circuit The circuit to lookup element indices of. Returns ------- slice .. py:method:: outcomes(circuit) The outcome labels of a circuit in this layout. Parameters ---------- circuit : Circuit The circuit to lookup outcome labels of. Returns ------- tuple .. py:method:: indices_and_outcomes(circuit) The element indices and outcomes corresponding to a circuit in this layout. Returns both the element indices and outcome labels corresponding to a circuit in this layout. These quantities can be separately obtained using the :meth:`indices` and :meth:`outcomes` methods, respectively. Parameters ---------- circuit : Circuit The circuit to lookup element indices and outcomes of. Returns ------- element_indices : slice outcome_labels : tuple .. py:method:: indices_for_index(index) Lookup the element indices corresponding to a given circuit by the circuit's index. Similar to :meth:`indices` but uses a circuit's index within this layout directly, thus avoiding having to hash a :class:`Circuit` object and gaining a modicum of performance. Parameters ---------- index : int The index of a circuit within this layout, i.e., within `self.circuits`. Returns ------- slice .. py:method:: outcomes_for_index(index) Lookup the outcomes of a given circuit by the circuit's index. Similar to :meth:`outcomes` but uses a circuit's index within this layout directly, thus avoiding having to hash a :class:`Circuit` object and gaining a modicum of performance. Parameters ---------- index : int The index of a circuit within this layout, i.e., within `self.circuits`. Returns ------- tuple .. py:method:: indices_and_outcomes_for_index(index) Lookup the element indices and outcomes corresponding to a given circuit by the circuit's index. Similar to :meth:`indices_and_outcomes` but uses a circuit's index within this layout directly, thus avoiding having to hash a :class:`Circuit` object and gaining a modicum of performance. Parameters ---------- index : int The index of a circuit within this layout, i.e., within `self.circuits`. Returns ------- element_indices : slice outcome_labels : tuple .. py:method:: iter_unique_circuits() Iterate over the element-indices, circuit, and outcomes of each unique circuit in this layout. A generator used to iterate over a `(element_indices, circuit, outcomes)` tuple for each *unique* circuit held by this layout, where `element_indices` and `outcomes` are the values that would be retrieved by the :meth:`indices` and :meth:`outcomes` methods, and `circuit` is the unique circuit itself. Returns ------- generator .. py:method:: copy() Create a copy of this layout. Returns ------- MatrixCOPALayout .. py:method:: resource_alloc(sub_alloc_name=None, empty_if_missing=True) Retrieves the resource-allocation objectfor this layout. Sub-resource-allocations can also be obtained by passing a non-None `sub_alloc_name`. Parameters ---------- sub_alloc_name : str The name to retrieve empty_if_missing : bool When `True`, an empty resource allocation object is returned when `sub_alloc_name` doesn't exist for this layout. Otherwise a `KeyError` is raised when this occurs. Returns ------- ResourceAllocation