Supporting Units in SMTK's Attribute Resource

Bob_Obara · February 22, 2023, 10:05pm

Current “Support” of Units

In SMTK 23.01, a designer can set the units of any ValueItemDefinition which is the base class used to represent String, Integer, and DoubleItem Definitions. The units are represented as a string and is used in the UI to indicate the units the value is assumed to be specified in.

There is now a call to take this to the next level and formally support units and unit conversion.

How Units Could Be Supported

When setting the value of the DoubleItem with the current setValue(double) method, it would be assumed that the value is in the default units specified by the designer. If the value is specified with its own units then if the units are different from the default, a unit conversion would take place. In either case, the value() method would always return the value in the default units. In the case of unit conversion there would be the need to access the value the user entered as well as the units specified.

Specifying a value with units

We could support either specifying the value and units using two separate line edit widgets (line edit and combobox) or with a single line edit with a validator and completer.

Screenshot 2023-02-22 at 4.43.16 PM

Note that the second approach would require the UI to somehow display the default units (in this case I’ve added them to the label),

The UI should also provide a mechanism to display the value in the default/preferred units. This could be as simple as having over the widget and using a tooltip or providing a checkbox to display the converted form. The former would take up less space.
Screenshot 2023-02-22 at 5.03.23 PM

Note: if there are no default units specified then SMTK would assume the value is unites and no unit information would be displayed.

Integration of a unit conversion library

SMTK would delegate the task of doing the conversion to an existing Open-Source library such as:

Proposed API Changes

Move the units specification to DoubleItemDefinition instead of ValueItemDefinition, unless folks can think of use cases where Integers and/or Strings can also benefit from having unit support.
Add one of the two possible APIs to DoubleItem to specify a value with units:
- setValue(double val, const std::string& units)
- setValue(const std::string& valueWithUnits)
Add one of the two possible API’s to DoubleItem to retrieve the original information:
- double value(std::string& units) const
- std::string value() const - Note that we could also use the existing valueAsString method
Add the ability to save the original entries in the XML/JSON file formats

Bob_Obara · February 22, 2023, 10:05pm

@Aaron @justin.2.wilson @rohith @dcthomp @johnt FYI

dcthomp · February 22, 2023, 10:42pm

A more compact form might be:

Screenshot from 2023-02-22 17-53-45

After a user enters a number with units, the right-hand label would (optionally, based on item-configuration) display the converted value:

Screenshot from 2023-02-22 17-51-54

If configured not to display a preview of the converted number, then a tooltip would always be available:

Screenshot from 2023-02-22 17-57-04

The right-hand label could also be cycled by a user click to display/hide the converted value.

johnt · February 23, 2023, 5:05pm

Regarding the 2 options

I think 2 separate fields would be more intuitive for users. Among the reasons is that it provides better clarity when the value and/or units are invalid.
I would also prefer that users get a dropdown list (or some equivalent fixed list) from which to choose the unit type, at least to start. (And maybe later we can add the “enter your own” as a future option based on user demand.) This would require new syntax in our xml and smtk file formats. We should come up with a scheme so that template authors don’t have to copy/paste alot of lists throughout the xml file.
If we go with separate value & unit fields, should we update the value when the user changes units? I presume so, otherwise range checking would get “interesting”. But I might be overlooking some other consequences.
I am not sure we need to show the user the equivalent value in solver-input units, but the suggestion of a context menu or tooltip works for me. Anything that doesn’t eat real estate.

Also

It might be handy at startup if the user could optionally set a preferred unit system and smtk updates the attdef defaults.
I agree that we should move units from value items to just doubles. The only integer example I can think of might be a “time” item that could be set to an integer number of minutes, hours, or days, etc. We can handle as a special case (2 items) if it ever comes up.

dcthomp · February 28, 2023, 7:04am

Another alternative that came up during conversations is to use the placeholder text to show units. This would eliminate the need for a second QLabel to the right of the QLineEdit.

Screenshot from 2023-02-28 01-42-16

The downside is that as soon as you start typing, the placeholder text disappears; if you forget the default units you will not be reminded of them. Either a context menu would have to show this or you would have to erase your entered value to see the placeholder text again. Here’s an example of what that might look like:

unit-proto-placeholder+context

johnt · March 2, 2023, 4:40pm

Here is a preview of how a double item might look with a units dropdown.

To save time, the units list was provided by a Python library called “Pint”.
This library finds all aliases with the same dimensionality, hence the “statmho” item which, I now know, is a unit of electrical conductance with base dimensions that resolve to distance/time.
The lists includes some other items that are probably not applicable to most use cases as well (“speed_of_light” maybe). We should provide developers a way to exclude items from the list for a given dimension.
Some units libraries also have APIs to add new units/quantities. We could consider adding that feature too.

units-preview-mph-tab

johnt · March 9, 2023, 8:39pm

One more follow-up, mostly fyi. I wrote a script to traverse the udunits2-common.xml file and extract all non-SI aliases, organized by “quantity” according to xml comment nodes in the file. I never came up with a good way to format the results, but settled on the following.

| Angular Velocity | cps | rotation_per_second | rotations_per_second | rpm | rps |

| Volume per time | sverdrup |

| Acceleration | gravity | standard_free_fall |

| Heat | clo |

For the record, I skipped a few quantities, labeled: Lineic Mass (related to fibers), Some “units” that make subsequent definitions easier, Electricity and Magnetism, Illumination, and Miscellaneous.

johnt · March 14, 2023, 8:38pm

Further thoughts and opinions:

Although the Pint library is really good, I think we have to drop it from consideration for the SMTK units library. Using Pint would require embedding Python and pybind11 into the smtkCore C++ library, which is a Rubicon we shouldn’t be crossing.
I think there is general agreement that the application should be able to control the list of compatible unit names displayed in the dropdown box for a given DoubleItem. To do this, I suggest that applications optionally provide a .csv file that is read in at runtime. API and format are TBD, but for a strawman:
- The .csv file could be applied to individual projects, but I would start with a global file.
- I would use one row per quantity (e.g., length, mass, speed, force, …)
- Maybe the first column could be the quantity label
- And maybe the second column could be the canonical SI representation (e.g., meter-second^-1) even if that is not a supported alias. This would allow the smtk parsing code to check that each of the remaining columns match its dimensionality.
- It might make sense to have 2 .csv files for (i) alias names and (ii) alias symbols TBD.
- For convenience, SMTK should probably provide a function to dump out all of the unit names and symbols it recognizes in the same .csv format.
Software design specifics haven’t been given alot of attention, but for a strawman:
- I would start with a singleton smtk::common::DimensionalUnitsUtility class that would initialize the third-party units library and probably read/write the .csv files mentioned above.
- It might make sense to also have a smtk::common::DimensionUnits class that would encapsulate the units functionality for both smtk::attribute::DoubleItem and whatever code we implement for the “specify output units” feature. We can decide when we get there…
- On the UI side, it might also make sense to have a smtk::extension::qtDimensionalUnitsWidget class to handle the units display (editable combobox).
The overload(s) of DoubleItem::value() that include unit conversion need some way to return whether the conversion was valid or not. One possibility:

    double smtk::attribute::DoubleItem::value(
        int element, const std::string& units, bool& isValid) const

would require the calling code to check the isValid flag before using the return value.

Maybe there should be a function to return the value in whatever the DoubleItemDefinition units are specified. This, presumably would always be a valid conversion.

    double smtk::attribute::DoubleItem::valueInDefinitionUnits(
        int element) const

dcthomp · March 16, 2023, 1:18am

Instead of CSV can we do something more structured like JSON? It is super-easy to deserialize into structures with sets and maps, so having something like:

// A base holding names.
struct Entity
{
  smtk::string::Token m_name;
  smtk::string::Token m_symbol;
  std::unordered_set<smtk::string::Token> m_aliases;
  smtk::string::Token m_description;
};

// One of the seven or so base dimensions.
struct Dimension : Entity { };

// Dimensions mapped to their exponents:
using ExponentMap = std::map<Dimension*, double>;

// Any unit (composite or otherwise.)
struct Unit : Entity
{
  ExponentMap m_dimensions;
};

// The entire unit configuration file.
struct UnitSystem
{
  std::unordered_map<smtk::string::Token, Dimension> m_dimensions;
  std::unordered_map<smtk::string::Token, Unit> m_derivedUnits;
  std::map<ExponentMap, std::unordered_set<Unit*>> m_indexByDimension;
};

The UnitSystem’s m_derivedUnits map would index all the units by name, symbol, and alias. It could even be a multi-map. The JSON equivalents to all these would be very straightforward.

dcthomp · March 16, 2023, 1:34am

I wrote a simple PEGTL parser written on Saturday that can be used to prevent ambiguous/confusing expressions that udunits2’s parser accepts. It is here for anyone curious.

You can compile it by placing it in SMTK’s source tree and pointing it to your cmb-superbuild like so:

 c++ -g -O0 -Wall -Wextra -std=c++14 \
  -I. -I../src -I /path/to/superbuild/install/include/ \
  -I /usr/include/udunits2 \
  -isystem /usr/include/qt5 \
  -isystem /usr/include/qt5/QtWidgets \
  -isystem /usr/include/qt5/QtGui \
  -isystem /usr/include/qt5/QtCore \
  -L /usr/lib64/qt5 -lQt5Gui -lQt5Widgets -lQt5Core \
  -ludunits2 -o unit-parser-proto \
  ../src/unit-parser-proto.cxx

As you can see when you run it, it will attempt to parse what you type into a QLineEdit and change the background color to

pink when there’s a parse failure
red when units are parsed but do not conform to length.

It also prints a bunch of debug crud to the console.

johnt · March 16, 2023, 1:28pm

I proposed .csv format so that developers could configure their units features in a spreadsheet. Maybe we could support both a “simple” version with .csv and and “advanced” version with json?

johnt · March 16, 2023, 1:42pm

This works for single units (m, kg, s) but it’s not obvious how to extend, for example, from “m” to “m/s”. Would we need the parser to extract each unit and prefix? might get tricky.

dcthomp · March 16, 2023, 3:33pm

The parser already recognizes units and operations, it just does nothing with them since the demo uses udunits to re-parse the validated string and perform conversion from whatever to meters. Really, we are not that far from having our own unit library.

johnt · March 17, 2023, 1:17pm

I still have gaps in understanding udunits. If it’s incremental, can you update your prototype to change “meter” to “meter/sec”?

dcthomp · March 17, 2023, 2:48pm

Sure, just remove the lines that create the destinationUnit variable and replace with

auto* destinationUnit = ut_parse(unitSystem, "m/s", UT_UTF8);

dcthomp · March 22, 2023, 3:32am

As we have been evaluating the various libraries available for unit conversion, I have been underwhelmed by the feature set compared to our desired features. Since the unit libraries we are considering would be add dependencies to smtkCore, I am also unhappy with some of the luggage. Finally, the example code in this reply above implements a parser that, in effect, provides a shadow unit system on top of udunits2 to prevent its native parser from accepting confusing inputs.

So, I took a little personal time last week and started my own unit library. With 2-3 more days of work, it could do everything we need (parsing dimensions, parsing units defined in terms of dimensions, parsing measurements of quantities that include units, unit conversion, and completion/prompting of units that are conformal to some desired/destination unit or dimension).