Tearing and cutting (Meshes vertices modifyng)

[lNatxii]

Hi,
I’m now trying to implement/modify/create all the possible interactions (cutting, grasping, tearing, pushing, suturing…) in iMSTK-Unity. But now, I’m going to focus on cutting and tearing, the ones that modify the mesh deleting/adding vertices.

I’m currently using iMSTK-Unity main-version-2 and iMSTK main branch (Don’t know the exact version, but not older than 15/20 days).

(PLEASE LET ME ADD MORE THAN ONE PICTURE AND MORE OF TWO LINKS)
--------------------------------------------------------CUTTING--------------------------------------------------------
I created cutting.cs just like:

cutting = new Imstk.PbdObjectCutting(pbdModel.GetDynamicObject() as Imstk.PbdObject, rigidModel.GetDynamicObject() as Imstk.CollidingObject);

And, when a button is pressed, i just call to cutting.apply.
https:// youtu.be/Ic9TxnOObD4
Using the gizmos I’ve seen the following facts:

• 4 new vertex are always created
• The vertex are not connected in the correct order (image)
• Splitting the old cut vertex in two vertex, and connecting them to the correct “neighbor” should be enough.

pd: theoretical, not theorical, sorry.

The image is a cut linemesh that was created from vertex number 4, to vertex number 0, with 4 segments, so, with vertex {0,1,2,3,4}

There is also a not important (for now) “bug” that causes an error in the normals calculation. The explanation is in iMSTK gitlab issues, I’m not gonna write it here because is not important.

--------------------------------------------------------TEARING------------------------------------------------------------
Using tearing.cs script that is in main-version-2, I’ve broken the mesh, but constraints are not totally broken. I can see that they are broken because I can puncture with a needle and the cloth offers no resistance, but the particles are still wirking together like a cloth. Video right there:

https:// youtu.be/v-9IkNYqXfg (Let me add links please :,) )

The modification i did to achieve this (visual holes) is just updating the mesh indices with the physics mesh indices, like this:

Imstk.SurfaceMesh physicsGeom = Imstk.Utils.CastTo<Imstk.SurfaceMesh>(GetPhysicsGeometry());
meshFilter.mesh.vertices = MathUtil.ToVector3Array(physicsGeom.getVertexPositions());
            
int[] indices = MathUtil.ToIntArray(physicsGeom.getTriangleIndices());
meshFilter.mesh.SetIndices(indices, MeshTopology.Triangles, 0);

[Harald_Scheirich]

Thanks this is helpful, i’ll check in the settings but if you do need to post more images it’s also ok to post a second message.

In general when something is not working in Unity i first make sure it’s working on the imstk C++ side. If what you are seeing going on in Unity does not match the imstk behavior, sometimes bug can be traced to slight differences in setup (there are some paths in imstk that aren’t tested well and the use through unity can expose those) or an actual bug in the Unity implementation. As far as i know we have neither tested the Surface Mesh tearing nor the LineMesh cutting.

In tearing we currently elimate the connecting cell (for the linemesh that is one segment) by

• removing the constraints attached to that cell
• setting the vertex indices for that cell to

No actual data is removed or added (i.e. the arrays are never reallocated)

So the mesh update in unity for tearing, updates the vertex positions, and then updates the indices (but it doesn’t change the size of the actual data)

Cutting is a little bit more complicated, the operation adds completely new cells and vertices to the mesh, as one triangle is split into two. I’m less familar with the cutting code, but i’d assume that you will need to at least resize the vertex and index array on the unity side and load all you get from unity, it looks like you’re doing that with the code at the bottom of your post for tearing but you will definitely need to do that for cutting.

The internal workings of cutting are a bit more complex, it needs to

• add two vertex at the point were the cut was detected
• add cells to connect the cut point with the rest
• add constraints for the newly created cells

The current cutting operation is also discrete and not continous, due to the cost. This means you can cut along the cutting geometry on command, but it’s not performant enough to keep cutting every frame. That would need some modifications in the cutting code.

So verify that Linemesh cutting and surface mesh tearing works in imstk, if not the bugs there would have to fix first. In Unity for tearing updating the mesh vertices and indices without reallocation should be enough. For cutting both the vertex and index array will need to be reallocated.

Hope this helps

[lNatxii]

Hi

I cannot test all of this right now, but for now I can tell you I don’t understand the constraints removal in tearing.

So the mesh update in unity for tearing, updates the vertex positions, and then updates the indices (but it doesn’t change the size of the actual data)

If I don’t write the next line (updating the mesh vertices array), an error is showed when I try to update the indices, saying some indices are referencing out of bounds vertices. The size of visual and physics vertices and indices are the same for both geometries.
meshFilter.mesh.vertices = MathUtil.ToVector3Array(physicsGeom.getVertexPositions());

You can clearly see in the video that constraints are still working between particles (this is not the expected behaviour), but they are not reacting to collisions with the rigid(Yes, this is the expected behaviour).

In Unity for tearing updating the mesh vertices and indices without reallocation should be enough

I’m already updating indices and vertices in Unity for tearing, like I showed above.

I will check if iMSTK behaviour is this one too.

Thank you, this is going to be helpful. I will update you.

[lNatxii]

EDIT: I have seen there is a cutting example, but only compiled when haptic device is used. I’m going to see what can I do.
Okay, I have been looking iMSTK examples, and I don’t know where to start to test tearing and grasping. I thought there was an existing example for each one, but there isn’t.
Can you help giving me a clue of how to make a simple scene that allows me to cut and tear a simple cloth?
I have not made anything in C++ for iMSTK apart from the UnityTransform controller, I don’t know how to make scenes. I know there is documentation, but just asking, maybe you have any started work about this.

By the way, is there any way of knowing from Unity the force that a rigid is making against a deformable? It would be helpful in future work.

[lNatxii]

Cutting is now working in line meshes, it was just necessary to add into cutting.cs the next line:

deformable.dynamicGeometry = true;

The visual things (normals, UVs…) are not working right now, but not important for now. Now I can cut a N vertices LineMesh and they behave as they have to.

I’ve modified PBDTissueCutExample.cpp, and tested that volumes can be cut. The problem is that it’s not using pbdObjectCutting, it’s using PbdObjectCellRemoval. I will try to make cellremoval.cs to see if this is enough.

[Harald_Scheirich]

That’s great. With regards to making a scene just building and looking at all the examples is a good start. For the tearing and cutting combinations that haven’t been tested combining examples is usually a good first approach.

You can get some force from the Controller, but i’m not sure if that is all that you would want.

With the amount of work that you are doing it would proably be good to have a conversation. Can you get in touch with me at harald.scheirich@kitware.com ?

[lNatxii]

I did it yesterday, I am looking forward to your response.

By the way, today I have been writing cellremoval.cs. I can cut tetrahedral meshes now, but I have the visual problem that I had with the LineMeshes. I was trying to update the mesh like in the LineMeshes, getting PhysicsGeometry and updating the visual geometry indices and vertices with PhysicsGeometry indices and vertices.

For tetrahedral objects (they have different meshes for physics geometry and visual geometry ), I think the vertices count for physics and visual geometry is not the same, but any way, the main problem is:

In Unity it doesn’t exist Meshtopology.tetrahedron for mesh.SetIndices(…), and I’ve been trying but I can’t achieve the visual mesh update. Any suggestion?

[Harald_Scheirich]

No unity can’t draw tetrahedral meshes, you can draw the surface mesh for a tetrehdral mesh, or you can write a Gizmo drawer drawing a the points and all tetrahedra edges

[lNatxii]

I know, what I need is updating the surface mesh for a tetrahedral mesh, with the tetrahedral mesh info, that has been modified with the cut. Will it be possible?

How will you focus this problem? The updated geometry is the tetrahedral one, how can I update the SurfaceMesh with the TetrahedralMesh info? Is there any way of mapping it?

[lNatxii]

Okay, I did it. UV are not woking well, and sometimes cutting gives an error saying vertex array gas less vertices than are referenced by the indices array, but it’s a start.

TetCutting

I just discovered that TethraedralMesh has a method named “extractSurfaceMesh” that does this mapping we were talking about.

Edit: The error es solved if Iuse mesh.Clear(). Now the only problem is fixing UV.

[lNatxii]

Edit 2: Temporal solution: Projecting UV planes each time I cut

[Harald_Scheirich]

That looks great, good progress, for a good overview of what certain classes can do in imstk the doxygen (iMSTK: iMSTK - Interactive Medical Simulation Toolkit) documentation should be helpful, most functions are available in C#.

[Harald_Scheirich]

You do want to keep an eye out for how long it takes to update the mesh in unity after modification, you might have to investigate the advanced api as documented in Unity - Scripting API: Mesh

[lNatxii]

The problem with the indices and vertex it’s solved, I wrote it in the last message.

Mesh.Clear() is the solution when continously changing the mesh triangles and vertices. My only problem right now is UVs, they are broken even when the cutting action doesn’t “delete” any vertex.

If I just use it in the air, (not touching the geometry), so, the list of vertex to delete is empty, the visible texture changes and starts doing weird things. I don’t know how to fix it without using plane UV projection all the frames. Any idea?

[lNatxii]

No solution for the moment. UVs and normals are a big problem in cutting due to the creation of new vertices. I’m still trying to find a solution, if anyone has any suggestion, tell me

[Harald_Scheirich]

There really isn’t a good solution to this. I actually don’t know what imstk currently does when it’s we’re cutting, as one would need to assign new uv coordinates at that time. In most cases you should be able to recalculate normals. Or you might have to create functionality that extends imstk so it retains more information about which points are new and which are old so you can update them from unity.

iMSTK has the ability to assign attributes to cells and vertices that could be used to mark points/faces when they’ve been modified or created

[lNatxii]

No problem Harald, I’m working in another things implementing iMSTK-Unity in my DaVinci simulator exercises. The only active problem in this forum entry is the constraints that are not broken when a particle is “destroyed” using tearing.cs. Don’t worry, I will work on it.

[lNatxii]

Still working on some things. iMSTK examples can cut a cloth using PbdObjectCutting (so, using SurfaceMeshCut too), but I don’t know why I can’t do it from Unity. I can cut LineMeshes too, so I guess that the the incompatibility is in SurfaceMeshCut when using SurfaceMeshes from iMSTK-Unity. This is not critical, so don’t worry about that, but I always try to write here all the “bugs” I find.

[egemenertugrul]

Hey @Harald_Scheirich and @lNatxii. I’ve been following the footsteps of this thread and managed to get to the point where I can display the vertices and cells as gizmos. However, I could not find a solution to consistently remove cells.

The white numbers on the vertex in the center (3,4,5) are obtained via SurfaceMesh.getCellsForVertex method, while the red numbers are obtained via SurfaceMesh.getCells method, each obtained value is passed to SurfaceMesh.getVertices as index, and averaged (centroid).

I then call, PbdObjectCellRemoval.removeCellOnApply and finally .apply, just as show in PBDTissueCutExample.cpp on indices such as 3,4,5. For these indices in particular, this approach works, however not for the other indices.

For further information, please watch the video: https://youtu.be/T18PbEkY3cw

I’d like to just specify the index of the vertex and remove its cells. Could you show me any directions? Thanks in advance.

[Harald_Scheirich]

Can you remove another set of indices besides 3,4,5 when you do it the first time? Does 3,4,5 work after you’ve done another set ? Are there any other faces removed when you try another set of indices ?

There were some issues with this code when looking at linemeshes that i fixed, this has been merged into main. You could try the latest main version.

Wrt to giving points rather than cells, you’re welcome to add a function inside of imstk, it will have to do the same lookup, i’m not sure it’s worth changing the binaries for that.

In general we calculate neighborhood between cells and vertices on the fly inside of imstk in the template class CellMesh here