:py:mod:`pygsti.baseobjs.mongoserializable` =========================================== .. py:module:: pygsti.baseobjs.mongoserializable .. autoapi-nested-parse:: Defines the MongoSerializable class Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: pygsti.baseobjs.mongoserializable.MongoSerializable pygsti.baseobjs.mongoserializable.WriteOpsByCollection pygsti.baseobjs.mongoserializable.RecursiveRemovalSpecification Functions ~~~~~~~~~ .. autoapisummary:: pygsti.baseobjs.mongoserializable.prepare_doc_for_existing_doc_check pygsti.baseobjs.mongoserializable.recursive_compare_str .. py:class:: MongoSerializable(doc_id=None) Bases: :py:obj:`object` Base class for objects that can be serialized to a MongoDB database. At the very least, saving an object to a database creates a document in the collection named by the `collection_name` class variable (which can be overriden by derived classes). Additionally, saving the object may create other documents outside of this collection (e.g., if the object contains MongoSerializable attributes that specify their own collection name). This interface also allows an object to save large chunks of data using, e.g., MongoDB's GridFS system, when serializing itself instead of trying to write an enourmous JSON dictionary as a single document (as an object that is NicelySerializable might do). .. py:attribute:: collection_name :value: "'pygsti_objects'" .. py:method:: from_mongodb(mongodb, doc_id, **kwargs) :classmethod: Create and initialize an object from a MongoDB instance. Parameters ---------- mongodb : pymongo.database.Database The MongoDB instance to load from. doc_id : bson.objecctid.ObjectId or dict The object ID or filter used to find a single object ID wihtin the database. This document is loaded from the collection given by the `collection_name` attribute of this class. `**kwargs` : dict Additional keyword arguments poentially used by subclass implementations. Any arguments allowed by a subclass's `_create_obj_from_doc_and_mongodb` method is allowed here. Returns ------- object .. py:method:: from_mongodb_doc(mongodb, collection_name, doc, **kwargs) :classmethod: Create and initialize an object from a MongoDB instance and pre-loaded primary document. Parameters ---------- mongodb : pymongo.database.Database The MongoDB instance to load from. collection_name : str The collection name within `mongodb` that `doc` was loaded from. This is needed for the sole purpose of setting the created (returned) object's database "coordinates". doc : dict The already-retrieved main document for the object being loaded. This takes the place of giving an identifier for this object. `**kwargs` : dict Additional keyword arguments poentially used by subclass implementations. Any arguments allowed by a subclass's `_create_obj_from_doc_and_mongodb` method is allowed here. Returns ------- object .. py:method:: write_to_mongodb(mongodb, session=None, overwrite_existing=False, **kwargs) Write this object to a MongoDB database. The collection name used is `self.collection_name`, and the `_id` is either: 1) the ID used by a previous write or initial read-in, if one exists, OR 2) a new random `bson.objectid.ObjectId` Parameters ---------- mongodb : pymongo.database.Database The MongoDB instance to write data to. session : pymongo.client_session.ClientSession, optional MongoDB session object to use when interacting with the MongoDB database. This can be used to implement transactions among other things. overwrite_existing : bool, optional Whether existing documents should be overwritten. The default of `False` causes a ValueError to be raised if a document with the given `_id` already exists and is different from what is being written. `**kwargs` : dict Additional keyword arguments poentially used by subclass implementations. Any arguments allowed by a subclass's `_add_auxiliary_write_ops_and_update_doc` method is allowed here. Returns ------- bson.objectid.ObjectId The identifier (`_id` value) of the main document that was written. .. py:method:: add_mongodb_write_ops(write_ops, mongodb, overwrite_existing=False, **kwargs) Accumulate write and update operations for writing this object to a MongoDB database. Similar to :meth:`write_to_mongodb` but collects write operations instead of actually executing any write operations on the database. This function may be preferred to :meth:`write_to_mongodb` when this object is being written as a part of a larger entity and executing write operations is saved until the end. As in :meth:`write_to_mongodb`, `self.collection_name` is the collection name and `_id` is either: 1) the ID used by a previous write or initial read-in, if one exists, OR 2) a new random `bson.objectid.ObjectId` Parameters ---------- write_ops : WriteOpsByCollection An object that keeps track of `pymongo` write operations on a per-collection basis. This object accumulates write operations to be performed at some point in the future. mongodb : pymongo.database.Database The MongoDB instance to write data to. overwrite_existing : bool, optional Whether existing documents should be overwritten. The default of `False` causes a ValueError to be raised if a document with the given `_id` already exists and is different from what is being written. `**kwargs` : dict Additional keyword arguments poentially used by subclass implementations. Any arguments allowed by a subclass's `_add_auxiliary_write_ops_and_update_doc` method is allowed here. Returns ------- bson.objectid.ObjectId The identifier (`_id` value) of the main document that was written. .. py:method:: remove_me_from_mongodb(mongodb, session=None, recursive='default') .. py:method:: remove_from_mongodb(mongodb, doc_id, collection_name=None, session=None, recursive='default') :classmethod: Remove the documents corresponding to an instance of this class from a MongoDB database. Parameters ---------- mongodb : pymongo.database.Database The MongoDB instance to remove documents from. doc_id : bson.objectid.ObjectId The identifier of the root document stored in the database. collection_name : str, optional the MongoDB collection within `mongodb` where the main document resides. If `None`, then `.collection_name` is used (which is usually what you want). session : pymongo.client_session.ClientSession, optional MongoDB session object to use when interacting with the MongoDB database. This can be used to implement transactions among other things. recursive : RecursiveRemovalSpecification, optional An object that filters the type of documents that are removed. Used when working with inter-related experiment designs, data, and results objects to only remove the types of documents you know aren't being shared with other documents. Returns ------- None .. py:class:: WriteOpsByCollection(session=None, allowed_collection_names=None) Bases: :py:obj:`dict` A dictionary of `pymongo` write operations stored on a per-collection basis. A dictionary with collection-name (string) keys and where each value is a list of `pymongo.InsertOne` and `pymongo.ReplaceOne` operations to be performed on the named collection at some later (unpsecified) time. The collection names (keys) can be restricted to a predefined set if desired. Initialize self. See help(type(self)) for accurate signature. .. py:attribute:: allowed_collection_names :value: 'None' .. py:attribute:: special_ops :value: '[]' .. py:attribute:: session :value: 'None' .. py:method:: add_ops_by_collection(other_ops) Merge the information from another :class:`WriteOpsByCollection` into this one. Parameters ---------- other_ops : WriteOpsByCollection The dictionary of write operations to merge in. Returns ------- None .. py:method:: add_one_op(collection_name, uid, doc, overwrite_existing, mongodb, check_local_ops=True) Add a single write operation to this dictionary, if one is allowed and needed. Adds a `pymongo.InsertOne` or `pymongo.ReplaceOne` object to `self[collection_name]` in order to write a document `doc` with unique identifer `uid`. If `overwrite_existing` is `False` then an error is raised if a document with `uid` already exists -- either in the database or within the list of existing write operations -- *and* the existing document doesn't match the document being written (`doc`). Parameters ---------- collection_name : str The collection name (key) to add an operation to. uid : bson.object.ObjectId or dict Unique identifier for the document to write. doc : dict The document to write. overwrite_existing : bool, optional Whether existing documents should be overwritten. The default of `False` causes a ValueError to be raised if a document with the given `uid` already exists *and* is different from `doc`. mongodb : pymongo.database.Database The MongoDB instance documents will eventually be written to. This database is *not* written to, and used solely to test for existing documents so that insert vs. replace operations are chosen correctly. check_local_ops : bool, optional Whether the queued-up write operations contained within this :class:`WriteOpsByCollection` object are considered when testing for the existence of documents. Leaving this set to `True` is the safe option. Set this to `False` to get a performance increase when you're sure there's no possibility a document with `uid` could have been "written" to this `WriteOpsByCollection` since its initialization. Returns ------- bson.objectid.ObjectId The identifier (`_id` value) of the document to be written or that already exists and matches the one being written. .. py:method:: add_gridfs_put_op(collection_name, doc_id, binary_data, overwrite_existing, mongodb) Add a GridFS put operation to this dictionary of write operations. This is a special type of operation for placing large chunks of binary data into a MongoDB. Arguments are similar to :meth:`add_one_op`. .. py:method:: execute(mongodb) Execute all of the "queued" operations within this dictionary on a MongoDB instance. Note that `mongodb` should be the same as the `mongodb` given to any :meth:`add_one_op` and :meth:`add_gridfs_put_op` method calls. The session given at the initialization of this object is used for these write operations. On exit, this dictionary is empty, indicating there are no more queued operations. Parameters ---------- mongodb : pymongo.database.Database The MongoDB instance to execute write operations on. Returns ------- None .. py:function:: prepare_doc_for_existing_doc_check(doc, existing_doc, set_id=True, convert_tuples_to_lists=True, convert_numpy_dtypes=True, round_to_sigfigs=6) Prepares a to-be inserted document for comparison with an existing document. Optionally (see parameters): 1. sets _id of `doc` to that of `existing_doc` . This is useful in cases where the _id field is redundant with other uniquely identifying fields in the document, and so inserted documents don't need to match this field. 2. converts all of `doc` 's tuples to lists, as the existing_doc is typically read from a MongoDB which only stores lists and doesn't distinguish between lists and tuples. 3. converts numpy datatypes to native python types 4. rounds floating point values Parameters ---------- doc : dict the document to prepare existing_doc : dict the existing document set_id : bool, optional when `True`, add an `'_id'` field to `doc` matching `existing_doc` when one is not already present. convert_tuples_to_lists : bool, optional when `True` convert all of the tuples within `doc` to lists. convert_numpy_dtypes : bool, optional when `True` convert numpy data types to native python types, e.g. np.float64 -> float. Returns ------- dict the prepared document. .. py:function:: recursive_compare_str(a, b, a_name='first obj', b_name='second obj', prefix='', diff_accum=None) Compares two objects and generates a list of strings describing how they differ. Recursively traverses dictionaries, tuples, and lists. Parameters ---------- a, b : object The objects to compare a_name, b_name : str, optional Names for the `a` and `b` objects for referencing them in the output strings. prefix : str, optional An optional prefix to the descriptions in the returned strings. diff_accum : list, optional An existing list that is accumulating difference-descriptions. `None` means start a new list. Returns ------- list A list of strings, each describing a difference between the objects. .. py:class:: RecursiveRemovalSpecification(edesigns=False, data=False, results=False, circuits=False, protocols=False, children=False) Bases: :py:obj:`object` Specifies which types of objects to remove when performing a recursive removal of MongoDB documents. Parameters ---------- edesigns : bool, optional Whether experiment designs are allowed to be removed. data : bool, optional Whether protocol data objects are allowed to be removed. results : bool, optional Whether result objects and directories are allowed to be removed. circuits : bool, optional Whether circuit objects stored in common collections (e.g. `"pygsti_circuits"`) are allowed to be removed. protocols : bool, optional Whether protocol objects are allowed to be removed. children : bool, optional Whether child objects of :class:`TreeNode` objects are allowed to be removed. .. py:attribute:: edesigns :value: 'False' .. py:attribute:: data :value: 'False' .. py:attribute:: results :value: 'False' .. py:attribute:: circuits :value: 'False' .. py:attribute:: protocols :value: 'False' .. py:attribute:: children :value: 'False' .. py:method:: cast(obj, root_cls_being_deleted=None) :classmethod: Create a :class:`RecursiveRemovalSpecification` object from another object. If `obj` is already a :class:`RecursiveRemovalSpecification` object then it is just returned directly. Otherwise, `obj` can be a string, boolean value, or `None`: - False, None, or `"none"`: no objects are removed recursively (the safe option) - True or `"all"`: all recursive removal operation are permitted (the un-safe option) - `"default"`: only recursive removal of the type being removed is allowed. For example, when removing a `ProtocolData` object, it and any child `ProtocolData` objects are removed but experiment designs are not, nor are (potentially shared) circuits. - `"upstream"`: recursive removal of the type being removed and "upstream" types is allowed. "Upstream" refers to items closer to the front of the list: experiment designs, data objects, result objects. For example, when removing a `ProtocolData` object, it and any child `ProtocolData` objects, along with their experiment designs are removed, but circuit objects are not. - `"all_but_circuits"`: everything but circuit objects are allowed to be removed. Circuit objects are treated specially because they are very likely to be re-used (shared). Parameters ---------- obj : object The object to convert. See description above. root_cls_being_deleted : class, optional The Python class of the main object being deleted. This additional information is needed when (and only when) `obj == "default"`. Returns ------- RecursiveRemovalSpecification