Versioning SMTK data

Simulation Modeling Toolkit (SMTK) Design
1

Purpose

We have had several discussions lately about version numbers inside SMTK files. This page exists to summarize, discuss the use cases involved, and document our decisions.

Uses of Versioning

We have identified 3 different ways versioning is being or will be used in SMTK:

  • to track the provenance of user-generated data. For instance, we wish to determine when opening an attribute resource whether the information in it applies to the current model file it lists or some predecessor (i.e., has the model been changed without updating the attribute – perhaps because the attribute was not loaded when the model was changed?).
  • to tag the schema of an attribute so that applications and/or operations know how to process it. Some applications may wish to support old attribute schema for a time. A particular use case is when multiple versions of a simulation code should be supported; features in the simulation code may change from version to version and require structural changes to the attribute system that prohibit a single schema for all versions. Schema versioning may be more or less granular:
    • project-level (coarse-grained) versioning is possible once we implement projects.
    • individual attribute, definition, item, and item-definition version numbers are available for fine-grained schema versioning.
  • to identify the file format used to store resources. We need to be clear about which version numbers are used for file format versioning so that developers do not attempt to use them for any of the above.
2

File format versioning

For our JSON-based file formats, which all have a dictionary as their top-level object, the version key has a string value that is currently used for file format versioning. The file format version is not stored in smtk::resource::Resource objects. Furthermore, loading and saving a resource will silently upgrade the file format version since SMTK aims to read all file-format versions but only write the most recent.

Similarly, the XML attribute format stores its file format version number as the version attribute of the top-level SMTK_AttributeManager tag (since renamed to SMTK_AttributeResource). As with the JSON format, this version number is not preserved across reads and writes.