Extending SMTK Resource Query Syntax

Extending SMTK’s Query Mechanism

Current Formats:

  • Model Syntax
  • Attribute Syntax - attribute[type =‘foo’]

So examples used in an association rule could look like:

 <Resource Name="smtk::attribute::Resource" Filter="attribute[type='testDef']" />
 <Resource Name="smtk::model::Resource" Filter="face[string{'pedigree'='zz'}]" />

What Needs to be Extended

  • The ability to for queries on the resource side of the query so we can search based on properties on the resource
  • The ability to have more than one predicate on the component side of the query
  • Expand beyond always assuming the property is a vector
  • The ability to make sure of parent child relationships that may exist among the components.
  • Do we need the ability to search resources based on its name instead of its type?
  • Side Note: In the attribute XML (and probably in the JSON format as well) - we use a Name key when in reality it should be a Type key

Possible XPATH derived Syntax

Instead of creating a new grammar John suggested that we should look at existing query grammars such as XPATH which is used to search HTML/XML structures. There is definitely some similarities between a SMTK resource structure and those for which XPATH was designed:

  • Nodes in XML could be analogous to Persistent Objects
  • Attributes (w/r XML) could be analogous to SMTK Properties

In terms of differences:

  • XPATH was designed for trees and not general graphs - though if you treat the root as the resource itself then at least both do have a single root
  • Assuming we do treat the resource as the root - then unlike XPATH we might want to put constraints on the root node (for example only consider those roots with certain properties
  • XPATH attributes are uniquely defined by a name while SMTK Properties are uniquely defined by data structure and name so the simple @ syntax will not be enough.

Should the resource be part of the query?

Currently the Resource Manager is handed a string that refers to the type of resource. Using this, the manager matches the resource type either directly or by asking the resource if it is derived from it using the manager’s find method. Instead we could pass the manager a query string itself and have it return the result directly. This would involve delegating part of the query to the resource itself.

Else we will need to extend resource::Manager::find to be a resource query method.

Possible Syntax

  • /[…] - refers to a resource that matches a set of predicates
    • name=’…’ - resource name constraint
    • type=’…’ - resource type constraint
    • @[…] - property constraint
      • name=’…’ property with name constraint
      • type=’…’ property with type constraint
      • val=’…’ property with value constraint
    • constraints can be combined using and / or
    • /res - resource of name = res - equivalent to /[name=‘res’]
    • /res[…] - equivalent to /[name=‘res’ and …]
  • /res//[…] - returns a set of components that match a set of constraints that are part of a resource named res
    • name=’…’ - component name constraint
    • type=’…’ - component type constraint
    • @[…] - property constraint
      • name=’…’ property with name constraint
      • type=’…’ property with type constraint
      • val=’…’ property with value constraint
    • constraints can be combined using and / or
  • a/b - refers to all components named b that is a child of component named a

Possible Examples

  • /res - refers to the resource named res
  • /res//alpha - the component named alpha under resource named res
  • /res//alpha/beta - the component named beta whose parent is named alpha that is part of resource res
  • /[type=‘smtk::model::Resource’]//[type=‘face’] - all faces that are part of resources of type model
  • /[type=‘smtk::model::Resource’ and @[name=‘alpha’ and type=‘string’ and val=‘a’]//[type=‘face’] - all faces that are part of resources of type model that contains a string property call alpha whose value is a.
  • /[type=‘smtk::attribute::Resource’ and @[name=‘alpha’ and type=‘string’ and val=‘a’]//[type=‘attribute’ and @[name=‘beta’] - all attributes that have a property called beta and that are part of resources of type attribute that contains a string property call alpha whose value is a.

With regard to the first bullet item at the top (“extending the resource side of the query”) and the section “Should the resource be part of the query?”, I would describe it slightly different. I would say that its general/underlying requirement is to specify a resource query and a component query in cascade. Might be a distinction without a difference, but I think our design should reflect the “two queries in daisy-chain” perspective.

With that in mind, I would prefer keeping the resource and component strings separate (although, yes, you can easily split or join strings).

My preference would be to use the resource/component type as the query’s “test” expression, in front of any predicate logic. This presumes that every query must specify the resource and component type, (unless/until we choose to also support “any”).

In the query string itself, I would also recommend changing “Resource” to “Query” and “Filter” to “Component”, mostly to be more consistent with the notion of two queries in cascade.

Adapting from the current syntax, an example might be:

<Query Resource="smtk::model::Resource[integer{'analysis'=1}]" Component="attribute[type='material'] />

This might be implied, but just in case: I think we should only adopt the @ symbol as syntactic sugar for a textual keyword, presumably “property”. As for combining property selectors, maybe we can use comma as an alias for “and”? So these 4 examples would be equivalent:

property[name='alpha' and type='string' and val='a']
@[name='alpha' and type='string' and val='a']

property[name='alpha',type='string',val='a']
@[name='alpha',type='string',val='a']

As for searching our more extensive/structured properties, we might get some insight from some of the json-query libraries (search “json path”) or even GraphQL. Not sure if either will be sufficient for our feature set.

It terms of one or two query strings - Since the proposed format uses:
/“resource related query”//“component related query”

//“component related query” is in itself a valid query meaning that you do not have to both pieces to be a valid query. Also you can give a query string with both pieces to the resource (and not to the resource manager). In that case the resource would confirm its matches the resource part of the query.

Fair enough. From outside the box, it’s seems awkward to specify the resource type in the first string and then put resource predicates in the second string. But if that format is more practicable for us to implement, that’s fine.

So if we go that way, I would also take back my recommendation to change “Filter” to “Component”, and instead stick with the more generic “Filter”.

As for the keyword to reference the resource in the filter string. I would recommend using the full word “resource” instead of “res”.

Regarding the “//” symbol, it looks to me like its function is to set/change the search axis from resource to components. I presume will could potentially generalize that to support attribute or maybe other axes too. Do we want to consider using the explicit “axis::test” syntax? It might just be more clutter, because we can infer the axis from the keyword. Nevertheless, here is a before and after example:

resource[integer{'analysis'=1}]//face[integer{'pedigreeId'=2}]
resource[integer{'analysis'=1}]/component::face[integer{'pedigreeId'=2}]

Related to that, would the proposed, fully-enumerated version of the first line translate to the following?

resource[property[type=integer and name='analysis' and value=1]]//face[property[type=integer and name='pedigreeId' and value=2]

How does this fit in with relative paths in attribute queries (assuming we are aiming to use the same filtering grammar for path queries as well)? Would even relative path queries start with // or is there a third form, assumed to act on components that begins with ./ or ../?

My high-level concerns are in two areas:

  • Switching to the XPath syntax will require supporting our existing syntax and the new one for a time. That is fine, but we should have a clear plan for the transition.
    • Are we going to rewrite the old queries as we read files in (so that saving will produce a new-style query and only the JSON and XML readers refer to the old grammar)?
    • Are we going to try to infer which grammar a given filter string uses or require file versioning to be consistent?
  • The DOM and SMTK’s resource/component model are not precisely identical; or at least the proposal above does not consider all the possibilities:
    • Links between resources and/or components, especially if our use of roles expands or we want to support applications that define their own roles.
    • Tags appear to only be used by operation specification attributes, but there is no constraint that they must be used this way. Will we ever want to query attributes by the tags present on their definitions?
    • Component relationships (find edges bounding faces with property X) are possible as are DOM element relationships, but we have not carefully considered potential differences. For instance, the XPath queries for nth() items make a lot of sense when processing lists of things, but one-to-many relationships in geometric models do not always attach a lot of meaning to order. They do attach meaning to orientation and sense of uses. They also attach meaning in non-binary ways (instead of asking for the third edge of a face, you might ask for the edge of face A that also bounds face B).
    • Non-POD-valued properties are possible. Should the query language allow resources to support them. For example, the RGG session might store duct and pin specifications in a struct. If the RGG session wants to allow query strings that filter based on this, how can the grammar support it?
Privacy Notice