Quickstart

Creating new ONDE files

Import the basic PyONDE classes and helper functions:

>> from PyONDE import (ONDEDatasetFile,
                       ONDEObject,
                       ONDEArray,
                       ONDEReferenceArray,
                       ONDEValue,
                       generate_onde_uuid)

Create a new file object:

>> of = ONDEDatasetFile.new("myfile.onde", "w", onde_version = "0.9.1pre")

The mode parameter (“w”) is passed directly to h5py.File() and is usually “w” for writing or “r” for reading. The onde_version parameter selects a version of the onde specification that is embedded in PyONDE. As an alternative you can pass a parameter class_defs_path pointing to the ONDE csv field definitions, and also pass additional class definitions with the extra_class_defs parameter.

The next step is to create an instance of an ONDE_DATASET subclass:

>> ds = ONDEObject.new(of,"ONDE_DATASET_UT_ASCAN")

Before it can be added to the file, the new dataset needs a unique identifier (UUID). Such an identifier can be created using the generate_onde_uuid() helper function:

>> import uuid
>> from datetime import datetime
>> ds.UUID = generate_onde_uuid("ISUCNDE",str(uuid.getnode()),datetime.now(),"")

The parameters are intended to be unique in combination. They indicate a vendor name, some kind of serial number, a microsecond-accurate timestamp, and a suffix. The UUID is a 128-bit hash of those parameters that is stored as a decimal string following the fixed prefix 2.25. for interoperability with the DICOM/DICONDE universe. Let us view the UUID:

>> ds.UUID
ONDEValue
                ID: 7fb4b681dbe0
              type: str
             value: 2.25.124217692408202938307279138509891347358

(Your UUID value will be different)

Now we can fill out more fields. For example,:

>> ds.LABEL="My Dataset"
>> ds
InstanceWrapper(ONDE_DATASET_UT_ASCAN)(ONDEObject)
                ID: 7fb4b681c6e0
AMPLITUDE_DIMENSION: None
              DATA: None
     DATE_AND_TIME: None
  INDEX_DIMENSIONS: None
             LABEL: ONDEValue(My Dataset)
         ONDE:TYPE: ONDEArray([['ONDE_DATASET', 'ONDE_DATASET_UT', 'ONDE_DATASET_UT_ASCAN']])
          OPERATOR: None
             SETUP: None
              UUID: ONDEValue(2.25.124217692408202938307279138509891347358)

At any point after the UUID is set, the dataset can be added to the ONDE file:

>> of.graph.add(ds)

The dataset is identified within the file by its UUID. Because the UUID is very long, you can access the dataset using a shortened version (omitting the 2.25.). If we just ask Python to show us a representation of the file graph, it will show us a handy shortened version:

>> of.graph
ONDEFileGraph (ONDE v0.9.1 class definitions)
ONDEFileGraphSnapshot
                  ID: 7fb4b681c2f0
                1242: ONDEObject(ID=7fb4b681c6e0)

The representation indicates that within the ONDEFileGraphSnapshot, index 1242 refers to an ONDEObject. Notice that in the ds.UUID output above, after the 2.25. was the same 1242. This is a shortened index you can use to access the dataset (in fact, you can shorten it further so long as it is unique).

Therefore, we can access the dataset by indexing of.graph:

>> of.graph["1242"]
InstanceWrapper(ONDE_DATASET_UT_ASCAN)(ONDEObject)
                ID: 7fb4b681c6e0
AMPLITUDE_DIMENSION: None
              DATA: None
     DATE_AND_TIME: None
  INDEX_DIMENSIONS: None
             LABEL: ONDEValue(My Dataset)
         ONDE:TYPE: ONDEArray([['ONDE_DATASET', 'ONDE_DATASET_UT', 'ONDE_DATASET_UT_ASCAN']])
          OPERATOR: None
             SETUP: None
              UUID: ONDEValue(2.25.124217692408202938307279138509891347358)

A full example of creating a UT dataset is included in the demos folder as build_onde_file.py. In general, scalar numeric values, strings, and integers can be assigned as we did with the LABEL above. More sophisticated types will need to be created explicitly.

Once objects are assigned into the graph, they are frozen and can no longer be changed. So, for example, if we were to try to change ds.LABEL, we would get an error:

>> ds.LABEL="Your Dataset"
Traceback (most recent call last):
[...]
RuntimeError: Attempting to modify an object that is already frozen

We can indeed change objects after they are inserted into the graph, but we have to do so by changing the graph: i.e. replacing them with copies. When you access an object from the graph, you actually get a reference to a proxy class that helps perform the graph modification. So all you have to do is make the change via the graph itself:

>> of.graph["1242"].LABEL="Your Dataset"
>> of.graph["1242"]
InstanceWrapper(ONDE_DATASET_UT_ASCAN)(ONDEObject)
                ID: 7fb4b68551d0
AMPLITUDE_DIMENSION: None
              DATA: None
     DATE_AND_TIME: None
  INDEX_DIMENSIONS: None
             LABEL: ONDEValue(Your Dataset)
         ONDE:TYPE: ONDEArray([['ONDE_DATASET', 'ONDE_DATASET_UT', 'ONDE_DATASET_UT_ASCAN']])
          OPERATOR: None
             SETUP: None
              UUID: ONDEValue(2.25.124217692408202938307279138509891347358)

Performing the assignment automatically and implicitly started a transaction for changing the graph. It then duplicated the ONDE_DATASET_UT_ASCAN object, replacing the LABEL as requested. It then walked the graph back to the corresponding entry point, of.graph[“2.25.124217692408202938307279138509891347358”] abbreviated as of.graph[“1242”], replacing all nodes with copies now pointing at new versions that lead to the updated LABEL. Finally, it ended the transaction creating an atomic update to of.graph.

WARNING: The variable ds refers to the original ONDE_DATASET_UT_ASCAN ONDEObject instance that was added to the graph. It was frozen when it was added to the graph. Any changes made since it was added will not appear when you look at ds.

Let us return to filling out the dataset object. The ONDEReferenceArray class represents an array of references to ONDE objects. We can create such an array to represent the dataset index dimensions:

>> ind_dims = ONDEReferenceArray.new(of,shape=(4,))
>> ind_dims
ONDEReferenceArray
                ID: 7fbc6248c2f0
             shape: (4,)

We can then populate the array with ONDE_DIMENSION instances:

>> ind_dims[0] = ONDEObject.new(of,"ONDE_DIMENSION")
>> ind_dims[0].COORDINATE = "U Position"
>> ind_dims[0].OFFSET = 0.0
>> ind_dims[0].SCALE = 1.0
>> ind_dims[0].UNITS = "meters"

and similarly for the remaining three dimensions. When instantiating an ONDE class ONDEObject.new(), the first parameter is the ONDEFile or ONDEFileGraph object and the second parameter is the name of the ONDE class.

We created ind_dims and stored it in a temporary variable so that it would not be frozen and we could still modify it before assigning it into our dataset (if we had not yet added the dataset to the graph, this would not be necessary). So we still need to assign it into the dataset:

>> of.graph["1242"].INDEX_DIMENSIONS = ind_dims

Numeric arrays can be similarly stored with the ONDEArray class. For example:

>> import numpy as np
>> of.graph["1242"].DATA = ONDEArray.new(value=np.zeros((1,1,1,1000), dtype="d"))

Using the above methods, the rest of the dataset can be filled out.

Once you have created and added all the datasets, you can write them to the disk file using the .flush() method:

>> of.flush()

Finally, you can close the disk file with the .close() method:

>> of.close()

Reading existing ONDE files

As with creating new files, it is a good idea to import the basic PyONDE classes and helper functions:

>> from PyONDE import (ONDEDatasetFile,
                       ONDEObject,
                       ONDEArray,
                       ONDEReferenceArray,
                       ONDEValue,
                       generate_onde_uuid)

Create a new file object from an on-disk file:

>> of = ONDEDatasetFile.new("myfile.onde", "r", onde_version = "0.9.1pre")

The mode parameter (“r”) is passed directly to h5py.File() and is usually “w” for writing or “r” for reading. The onde_version parameter selects a version of the onde specification that is embedded in PyONDE. As an alternative you can pass a parameter class_defs_path pointing to the ONDE csv field definitions, and also pass additional class definitions with the extra_class_defs parameter.

The onde_version that you specify for reading does NOT need to match the version stored in the file itself. Rather, the version that you specify controls which classes are defined and the known fields of those classes. Therefore, when reading ONDE files programmatically, you should specify the ONDE version corresponding to the fields you reference in your code. PyONDE can happily manage unknown fields, but such fields will need to be accessed using brackets and with class definition prefixes, e.g. ds.graph[“12345”][“MYORG_MY_ACCESSORY_CLASS:MYFIELD”] rather than through the usual Python attribute shorthand.

Once the file has been loaded, it is accessible just like when it was written. You index the various datasets by their UUID or a shortened version. For example:

>> of
ONDEFile open on "myfile.onde"
ONDEFileGraph (ONDE v0.9.1 class definitions)
  ONDEFileGraphSnapshot
                    ID: 7f9ca6c0f4d0
                  1242: ONDEObject(ID=7f9ca6c0f750)

Then index into the dataset we saved above (note that its UUID is the same as when we generated it above):

>> of.graph["1242"]
InstanceWrapper(ONDE_DATASET_UT_ASCAN)(ONDEObject)
                ID: 7f9ca6c0f750
AMPLITUDE_DIMENSION: None
              DATA: ONDEArray(shape=(1, 1, 1, 1000),dtype=float64)
     DATE_AND_TIME: None
  INDEX_DIMENSIONS: InstanceWrapper of ONDEReferenceArray(shape=(4,))
             LABEL: ONDEValue(Your Dataset)
         ONDE:TYPE: ONDEArray([['ONDE_DATASET', 'ONDE_DATASET_UT', 'ONDE_DATASET_UT_ASCAN']])
          OPERATOR: None
             SETUP: None
              UUID: ONDEValue(2.25.124217692408202938307279138509891347358)

You can get a list of all of the datasets with the .keys() method:

>> of.graph.keys()
['2.25.124217692408202938307279138509891347358']

Since the UUID’s are very long, there is also a .shortkeys() method:

>> of.graph.shortkeys()
['1242']

For regular objects, you can show the various fields with the ._keys() method:

>> of.graph["1242"]._keys()
['ONDE_DATASET:OPERATOR', 'ONDE_DATASET:INDEX_DIMENSIONS', 'ONDE_DATASET:SETUP', 'ONDE_DATASET:AMPLITUDE_DIMENSION', 'ONDE:UUID', 'ONDE:TYPE', 'ONDE:LABEL', 'ONDE_DATASET:DATE_AND_TIME', 'ONDE_DATASET:DATA']

To get the shorthand version, you can instead use the .shortkeys() method:

>> of.graph["1242"]._shortkeys()
['OPERATOR', 'SETUP', 'DATA', 'DATE_AND_TIME', 'LABEL', 'UUID', 'ONDE:TYPE', 'AMPLITUDE_DIMENSION', 'INDEX_DIMENSIONS']

You can also list attributes with dir():

>> dir(of.graph["1242"])
 ['AMPLITUDE_DIMENSION', 'DATA', 'DATE_AND_TIME', 'INDEX_DIMENSIONS', 'LABEL', 'ONDE:TYPE', 'OPERATOR', 'SETUP', 'UUID', '_freeze', '_frozen', '_get_attr', '_has_attr', '_list_attrs', '_set_attr']

If a class or accessory class is unknown, even the short version of the field name will contain a colon, so you won’t be able to access it with the usual Python attribute notation. You can always use brackets as an alternative:

>> of.graph["1242"]["ONDE_DATASET:INDEX_DIMENSIONS"]
ONDEReferenceArray
                ID: 7fc425754980
             shape: (4,)

Note that just because a field is listed in dir() or ._keys() or ._shortkeys(), doesn’t mean the field has a value. If the field has no value, attempting to read it will raise an exception. You should always be able to assign a field.