# Replacing \`smtk::model::Tessellation\`

**URL:** <https://discourse.kitware.com/t/replacing-smtk-tessellation/343>\
**Category:** Design\
**Created:** [November 10, 2019, 12:16am UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343 "2019-11-10T00:16:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)\
**Post date:** [November 10, 2019, 12:16am UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/1 "2019-11-10T00:16:10Z")

</div>

## Purpose

I’m making progress on removing `smtk::model::Tessellation` in favor of a class that will provide VTK (and/or VTK-m) data on demand. I would like some feedback before I go too much further.

The replacement class is [named GeometryProvider](https://gitlab.kitware.com/dcthomp/smtk/blob/tessellation-redux/smtk/resource/GeometryProvider.h) and here are its relevant features:

1. Instead of each `Component` having a `Tessellation` object, now each _`Resource`_ may (if the `Resource` has any renderable geometry) supply a non-null `GeometryProvider`. You can ask the provider about the `Resource` or any `Component`. In particular you can ask for:  
a. the geometry’s **serial number** - like a generation number that is incremented each time the geometry changes. This must be a fast query.  
b. the object’s **bounds** – as a 6-tuple; this is assumed to be fast.  
c. the geometry’s **role** – which specifies how the object is to be treated by the renderer (as normal tessellation, as instance placements, as image/volume data, possibly with labels to be volume rendered). The role lets us decide which blocks inside the `vtkResourceMultiBlockSource`'s output to place the provided data:  
d. **data** (as a `vtkDataObject*`) – specifying the object’s geometry. This operation is assumed to be slow and may copy data (but may be fast using zero-copy).
2. The `GeometryProvider` must also provide a `visit()` method to traverse all objects that have geometry (whether they have been converted to a `vtkDataObject` yet or not). Traversing would not cause conversion, just report objects. This may be implemented by simply mirroring the resource’s component-visitor but some providers might be wish to be more intelligent.
3. Using `smtk::common::Generator<ResourceSubclass, GeometryProviderPtr>`, it is possible for `Resource` classes in the smtkCore to construct geometry providers present in other libraries (as long as the application is linked to them or SMTK has loaded a plugin containing them). Note that this means **if smtkCore is loaded with no other libraries, classes like `smtk::mesh::Resource` will report no renderable geometry**. Operations that run on resources using their native modeling kernel should work fine, but any that assume the resource’s components have a `Tessellation` will not work. I have not found any existing operations which do this, but it will mean some unit tests need a rewrite.
4. Because all resources may provide geometry, we can eliminate the subclasses of `vtkResourceMultiBlockSource`. And `vtkResourceMultiBlockSource` will just ask the resource for its geometry provider and visit all objects with geometry, only calling `GeometryProvider::data()` on objects with stale or non-existent cache entries.

## Open Questions

1. @tj.corona I’m not really familiar with `smtk::mesh::Resource` and have a question: will a `MeshSet`'s elements ever change? If so, can I tell when? It seems like this is possible if someone (a) modifies point coordinates or (b) adds/removes handles from the `MeshSet`. But I do not know how to detect this so that I can increment the relevant `MeshSets`’ (and only the relevant `MeshSets`') serial numbers. It feels like we should avoid forcing operations to query for a `GeometryProvider` in order to mark entries in its cache of `vtkDataObjects` dirty. Should the `GeometryProvider` be observing `operation::Managers`? That seems restrictive and then I’m not sure how the provider could discover the managers on its own… the application would need to configure every relevant geometry provider. Bleh! This means Python scripts, too. Double-bleh! Maybe I am changing my mind about forcing operations to do some of the work. Opinions? Or am I an ignoramus who doesn’t know how mesh can track changes?
2. Is requiring geometry providers to implement `visit()` premature optimization?
3. There are some special cases with roles I’m not sure exactly how to handle. Even if I don’t implement anything to cover these cases immediately, I’d like to make sure we’re not making it impossible to deal with them:  
a. Some components may share geometry but use it differently.  
i. Example: If we ever deal with model face-uses instead of model faces, the uses may share geometry but want to specify whether they apply to the front-face or the back-face.  
ii. Example: For medical applications, geometry is usually a volumetric image. Multiple tissues/organs (each of which will be an SMTK model entity) may exist on the same image as either (a) distinct integer values in a single scalar field or (b) a separate scalar field for each tissue type.  
b. Some components might serve multiple roles. For example, when a model entity has its own geometry but also serves as the prototype for some instanced geometry. I plan to deal with this particular example by asking anything that reports its Role as an instance for its prototype’s geometry. But are there other cases?
4. One of the reasons I’m working on this now is a medical application that needs volume rendering, slice rendering, and widgets for editing segmentation label-maps. I plan to extend `vtkResourceMultiBlockSource` with a new top-level block holding vtkImageData objects and then make the `vtkSMTKResourceRepresentation` create separate actors for each image (during `ProcessViewRequest()`).  
I want to avoid separate representations (because that would be really messy to manage) but also want end users to have control over each image’s visual style (render as slice, volume render, etc.). Is this a bad design? It feels un-ParaView-like although it could be addressed to some degree with a self-generating source proxy. On the other hand, if I make separate representations for images inside a resource, how will things like close-resource, selection (incl. highlight-on-hover), and visibility-editing in descriptive phrase views all work? That feels un-SMTK-like.

---

<div class="post-metadata">

**Author:** ![Bob\_Obara](https://discourse.kitware.com/user_avatar/discourse.kitware.com/bob_obara/32/37_2.png) [@Bob\_Obara](https://discourse.kitware.com/u/Bob_Obara)\
**Post date:** [November 11, 2019, 3:58pm UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/2 "2019-11-11T15:58:01Z")

</div>

I like the concept of having an object “represent/abstract out” the tessellation. So here are my initial questions:

1. What is the granularity that the GeometryProvider provide? Is it at the componentItem level? For example is there a method to call to get the geometry of a specific component item?
2. Is the serial number referring to the entire contents of the provider or to some smaller quanta like the resource’s components?
3. You say the provider will keep bounds information - same question as above, will it also keep track of component’s current bounds as well?
4. Not sure about the need of the visit() method yet. What I think the next step would be is to show the following useCases:
  1. Resource loaded into SMTK.
  2. ModelBuilder requests the first render
  3. ModelBuilder requests the second render (resource not modified)
  4. Resource component geometry is modified
  5. Resource component is non-geometrically modified

---

<div class="post-metadata">

**Author:** ![Haocheng\_Liu](https://discourse.kitware.com/user_avatar/discourse.kitware.com/haocheng_liu/32/36_2.png) [@Haocheng\_Liu](https://discourse.kitware.com/u/Haocheng_Liu)\
**Post date:** [November 11, 2019, 4:29pm UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/3 "2019-11-11T16:29:53Z")

</div>

> [@dcthomp](#):
>
> One of the reasons I’m working on this now is a medical application that needs volume rendering, slice rendering, and widgets for editing segmentation label-maps. I plan to extend `vtkResourceMultiBlockSource` with a new top-level block holding vtkImageData objects and then make the `vtkSMTKResourceRepresentation` create separate actors for each image (during `ProcessViewRequest()` ).

Hi David, what’s the motivation behind creating separate actors for each image?

---

<div class="post-metadata">

**Author:** ![tj.corona](https://discourse.kitware.com/user_avatar/discourse.kitware.com/tj.corona/32/35_2.png) [@tj.corona](https://discourse.kitware.com/u/tj.corona)\
**Post date:** [November 11, 2019, 4:54pm UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/4 "2019-11-11T16:54:57Z")

</div>

> [@dcthomp](#):
>
> will a `MeshSet` 's elements ever change? If so, can I tell when?

Yes, meshets change. I’d imagine you would know in the same way that you’d know when a model’s elements change: the components that represent the meshsets are added to the “modified” result of an operation.

> [@dcthomp](#):
>
> It seems like this is possible if someone (a) modifies point coordinates or (b) adds/removes handles from the `MeshSet` . But I do not know how to detect this so that I can increment the relevant `MeshSets` ’ (and only the relevant `MeshSets` ') serial numbers.

I think you should treat SMTK Mesh’s handles as an implementation detail, and instead you should be listening to the results of operations (“created”, “modified”, “expunged”).

> [@dcthomp](#):
>
> It feels like we should avoid forcing operations to query for a `GeometryProvider` in order to mark entries in its cache of `vtkDataObjects` dirty. Should the `GeometryProvider` be observing `operation::Managers` ?

I’d need to see more of the implementation before I had a feel for how this should work.

> [@dcthomp](#):
>
> That seems restrictive and then I’m not sure how the provider could discover the managers on its own… the application would need to configure every relevant geometry provider. Bleh!

I only understood the last part of this.

> [@dcthomp](#):
>
> This means Python scripts, too. Double-bleh!

And the last part of this.

I may need you to walk me through these issues before I can be helpful. Otherwise, all I have is “Triple-bleh!”

---

<div class="post-metadata">

**Author:** ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)\
**Post date:** [November 11, 2019, 5:03pm UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/5 "2019-11-11T17:03:51Z")

</div>

> [@Bob\_Obara](#):
>
> - What is the granularity that the GeometryProvider provide? Is it at the componentItem level? For example is there a method to call to get the geometry of a specific component item?

The granularity is at the component and/or the resource level (i.e., the resource may provide geometry for itself and/or its components).

> [@Bob\_Obara](#):
>
> Is the serial number referring to the entire contents of the provider or to some smaller quanta like the resource’s components?

They apply to the persistent object you pass (generally components). If a resource itself has geometry, the serial number should only apply to the resource’s geometry (i.e., the resource serial number will not apply to component geometry).

> [@Bob\_Obara](#):
>
> You say the provider will keep bounds information - same question as above, will it also keep track of component’s current bounds as well?

Yes.

> [@Bob\_Obara](#):
>
> Not sure about the need of the visit() method yet.

My thought was that if a resource only provided geometry for itself (no components), then `visit()` would be much faster than iterating over components and always getting a null geometry back.

> [@Bob\_Obara](#):
>
> What I think the next step would be is to show the following useCases:
> 
> 1. Resource loaded into SMTK.

My branch does this for the VTK session.

> [@Bob\_Obara](#):
>
> 1. ModelBuilder requests the first render

I am debugging this; the geometry is being fetched but not rendered.

> [@Bob\_Obara](#):
>
> 1. ModelBuilder requests the second render (resource not modified)

This should be a no-op; nothing will have marked the `vtkResourceMultiBlockSource` as modified.

> [@Bob\_Obara](#):
>
> 1. Resource component geometry is modified

This should cause a pipeline re-execution. The `vtkResourceMultiBlockSource` will only replace leaf-node data objects with new versions when that leaf-node’s geometry has changed. However, I am not clear on what causes the composite polydata mapper to re-upload geometry to the GPU.

> [@Bob\_Obara](#):
>
> 1. Resource component is non-geometrically modified

Then the serial number should not be changed. This is up to the operation and/or geometry provider; the serial number provides a mechanism for this but does not guarantee people use it.

---

<div class="post-metadata">

**Author:** ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)\
**Post date:** [November 11, 2019, 5:04pm UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/6 "2019-11-11T17:04:47Z")

</div>

> [@Haocheng\_Liu](#):
>
> Hi David, what’s the motivation behind creating separate actors for each image?

Texture-mapping requires it.

---

<div class="post-metadata">

**Author:** ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)\
**Post date:** [November 11, 2019, 5:16pm UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/7 "2019-11-11T17:16:22Z")

</div>

> [@tj.corona](#):
>
> I’d imagine you would know in the same way that you’d know when a model’s elements change: the components that represent the meshsets are added to the “modified” result of an operation.  
> …  
> I may need you to walk me through these issues before I can be helpful. Otherwise, all I have is “Triple-bleh!”

I’m not sure that model Resource subclasses will be examining operation results to update the GeometryProvider. Currently those operations that change geometry also have calls in them to compute a new tessellation. That will be replaced with something that just increments the serial number on the geometry provider for each component involved.

I don’t know enough about `smtk::mesh` to say how feasible this is. For example, operations that apply changes to point coordinates would now be required to iterate meshsets and determine which ones refer to points whose coordinates have been changed. Also, as I understand it, multiple meshsets can refer to the same handles; that makes subsetting is easy, but means that changing topology of one meshset will cause changes to others… is that true?

It feels like the mesh geometry provider should provide some of this rather than relying on operation results (which would make GeometryProvider only work on managed resources).

Is that enough to help you move from “triple bleh” to “opinion?”

---

<div class="post-metadata">

**Author:** ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)\
**Post date:** [January 8, 2020, 5:18am UTC](https://discourse.kitware.com/t/replacing-smtk-tessellation/343/8 "2020-01-08T05:18:04Z")

</div>

As discussed here and [in this earlier topic](https://discourse.kitware.com/t/geometry-library/224), the first parts of this have been [merged](https://gitlab.kitware.com/cmb/smtk/merge_requests/1903). The change that has been merged adds new classes (in `smtk/geometry` and `smtk/extension/vtk/source`) but does not remove `smtk/model/Tessellation` yet.

If a resource (1) inherits `smtk::geometry::Resource` and (2) provides an `smtk::geometry::Geometry` object (via methods defined in the new resource class), then that geometric data will be used instead of any model Tessellation objects. Otherwise, model tessellation objects will be used to produce VTK data. The new classes allow resources to provide renderable geometry without the overhead of conversion to/from SMTK’s neutral format. This will improve the responsiveness of SMTK and ModelBuilder in some cases. However, there is another change that must occur before large datasets are handled smoothly in most cases: because the `vtkResourceMultiBlockSource` (and its subclasses) are re-run when any component in the resource has modified geometry, VTK’s composite polydata mapper must be updated to avoid re-sending unmodified geometry to the GPU.

For more information, see

- [the new documentation](https://smtk.readthedocs.io/en/latest/userguide/geometry/concepts.html)
- [this test](https://gitlab.kitware.com/cmb/smtk/blob/master/smtk/geometry/testing/cxx/TestGeometry.cxx), which is a simple example of its use; and
- the [polygon session](https://gitlab.kitware.com/cmb/smtk/tree/master/smtk/session/polygon/vtk) which provides geometry for the VTK backend.