How to grasp rigidObjects in Unity.

Hi again,

I want to implement some exercises in my Unity project where I have to grasp some rigid objects to put them in some nets. We have, for not deformable objects, a useful system to grab native Unity objects (Colliders+RigidBodies).

In this point, I have 3 options:

  • Implement RigidGrasping.cs in Unity. Grasping.cs doesn’t work with iMSTK-Unity rigids → "Grasping rigid with static geometry (mesh) not supported"

  • Use my TransformDevice in a native object, and link my rigid to it, so I can grasp objects and collide with deformables too. The problem with this option is that, when releasing the object fo example in the Net, the net will react to the rigid collision, but the rigid won’t stop because will be following the TransformDevice, that will overstep(?) the net.

  • Modify the C++ rigid code to implement a function that sets the rigid pose only once (when called), so I can set to the iMSTK rigid the position and rotation of the Unity Object when is realased. Obviously, this needs to:
    - Add native rigidbody and collider to the iMSTK-Unity rigid
    - Not updating the rigid position and rotation with the iMSTK info during the grab time.

I think 1 and 3 are the best options, and option 3 will be more useful for using some of the code that is already implemented and using our Unity native rigids grasping system. What do you think?

If you’re curious, I have tried to implement RigidGrasping.cs but runtime errors are shown and I can’t find them in the code, basically is Grasping.cs but using PbdRigidBodyGrasping (from iMSTK). I thought it would be easy:

namespace ImstkUnity
{

    public class RigidGrasping : ImstkInteractionBehaviour
    {
        public enum GraspType
        {
            Vertex,
            Cell,
        }
        public Rigid rigidModel;
        public GeometryFilter graspingGeometry;
        public Rigid graspedObject;
        [Tooltip ("If not null the collision will be disabled when a grasp happened")]
        public CollisionInteraction collisionToDisable;

        public GraspType graspType = GraspType.Cell;

        public bool isTwoWay = false;
        
        Imstk.PbdRigidObjectGrasping interaction;
        private string _collisionDetectionType;
        
        public override Imstk.SceneObject GetImstkInteraction()
        {
            if (rigidModel == null || graspedObject == null)
            {
                Debug.LogError("Both models need to be assigned for the interaction to work");
            }

            _collisionDetectionType = CollisionInteraction.GetCDType(rigidModel, graspedObject);

            if (_collisionDetectionType == "")
            {
                Debug.LogError("Could not determine collision type for grasping in " + gameObject.name);
            }

            if (! rigidModel.isActiveAndEnabled || !graspedObject.isActiveAndEnabled)
            {
                return null;
            }
            
            interaction = new Imstk.PbdRigidObjectGrasping(rigidModel.GetDynamicObject() as Imstk.PbdObject, graspedObject.GetDynamicObject() as Imstk.RigidObject2);//Two-Way

            Imstk.Geometry geom = rigidModel.GetDynamicObject().getCollidingGeometry();
 
            Imstk.AnalyticalGeometry analytical = Imstk.Utils.CastTo<Imstk.AnalyticalGeometry>(geom);

            if (analytical == null)
            {
                Debug.LogError("Can't convert to analytical geometry" + geom.getTypeName());
            }

            return interaction;
        }

        public void StartGrasp()
        {
            if (interaction == null) return;

            Imstk.Geometry rigidGeometry = rigidModel.GetDynamicObject().getCollidingGeometry();
            Imstk.AnalyticalGeometry analyticalGraspingGeom;
            if (graspingGeometry == null)
            {
                analyticalGraspingGeom = Imstk.Utils.CastTo<Imstk.AnalyticalGeometry>(rigidGeometry);
            } 
            else
            {
                analyticalGraspingGeom = Imstk.Utils.CastTo<Imstk.AnalyticalGeometry>(graspingGeometry.GetOutputGeometry());
                analyticalGraspingGeom.setTransform(rigidGeometry.getTransform());
                analyticalGraspingGeom.updatePostTransformData();
            }

            if (analyticalGraspingGeom == null)
            {
                Debug.LogError("Grasping Geometry can't be null in " + gameObject.name);
                return;
            }

            switch (graspType)
            {
                case (GraspType.Cell):
                    interaction.beginCellGrasp(analyticalGraspingGeom);
                    break;
                case (GraspType.Vertex):
                    interaction.beginVertexGrasp(analyticalGraspingGeom);
                    break;
            }
        }

        private void Update()
        {
            if (interaction != null && collisionToDisable != null)
            {
                var imstkCollision = Imstk.Utils.CastTo<Imstk.CollisionInteraction>(collisionToDisable.GetImstkInteraction());
                if (interaction.hasConstraints())
                {
                    imstkCollision.setEnabled(false);
                }
                else
                {
                    imstkCollision.setEnabled(true);
                }
            }
        }

        public void EndGrasp()
        {
            if (interaction != null)
            {
                interaction.endGrasp();
            }
        }

        /// <summary>
        /// Will return true whenever constraints where generated, this means
        /// that something has _actually_ been grasped.
        /// NOTE it takes at least one simulation manager FixedUpdate() for this
        /// to return a correct value, check in the next frame after calling 
        /// StartGrasp()
        /// </summary>
        public bool HasConstraints()
        {
            return interaction != null && interaction.hasConstraints();
        }
    }

}

You are getting the error message because the thing that you are grasping with isn’t a rigid, i don’t know your setup to create the constraint both objects need to be PBD objects at least. To enable grasping your VR input should be connected to a Rigid Controller, that should be connected to a Rigid and that in turn should be used for grasping. Yes that is your second option. I don’t understand the second part of that paragraph though.

The PbdRigidObjectGrasping is the incorrect class as it would connect to RigidObject2 objects and not PBD. imstkUnity on the tip of the development main-2 and the new branches that you have been working with only uses PBD based objects for both deformable and rigids. Summary, grasping a rigid should work with the current code

image

The rigid here is supposed to be the tool that is doing the grasping (is the PBD object that the grasped object will be attached to) the Grasping Geometry is an option geometry filter that can be used for the intersection check, grasped object is the thing that you are trying to grasp (and as you can see it’s “Dynamical Model”, if you want you may disable collisions with the grasped object. And the “Grasp Type” indicates wether the grasp with be on a vertex or a cell (i.e. line, triangle tetrahedron)

Using the “Grasping Manager” should make things easier as well
image

As it will help you manage grasping multiple things in the same scene

Try and let me know please

Hi,

The rigid here is supposed to be the tool that is doing the grasping (is the PBD object that the grasped object will be attached to) the Grasping Geometry is an option geometry filter that can be used for the intersection check, grasped object is the thing that you are trying to grasp (and as you can see it’s “Dynamical Model”, if you want you may disable collisions with the grasped object. And the “Grasp Type” indicates wether the grasp with be on a vertex or a cell (i.e. line, triangle tetrahedron)

I know how grasping works, I have been using it this months, is one of the first mechanics that I have used in iMSTK-Unity. I have been using Grasping Manager too, so I also know how it works. That’s why I don’t really understand what is happening.

The PbdRigidObjectGrasping is the incorrect class as it would connect to RigidObject2 objects and not PBD. imstkUnity on the tip of the development main-2 and the new branches that you have been working with only uses PBD based objects for both deformable and rigids

Yes, you’re right, I just tried a crazy thing just because it had “Rigid” on the name.

To enable grasping your VR input should be connected to a Rigid Controller, that should be connected to a Rigid and that in turn should be used for grasping. Yes that is your second option. I don’t understand the second part of that paragraph though.

I’m using it with a Rigid Controller, basically I’m using a DaVinci robot arms with some rigids(GeometryFilters - Capsule) and their respectives TransformDevices, so this is not the problem.

I’m using the blue capsule as the rigid model and the red one as the grasping geometry.

I don’t know, I will work on it but I think it isn’t working.

I don’t understand the second part of that paragraph though.

About this, I’m just saying that, if I add a TransformDevice to a native rigid, and I use it as a rigid controller in the iMSTK-Unity rigid (the ones that can interact with deformables), It will work while grabbing the object, but it won’t behave as I want. This is because when colliding with a deformable, the native rigid will not stop, while the iMSTK-Unity will try to stop. This can lead to:

  • Native and iMSTK rigid are not in the same place, so, trying to grab again the iMSTK-rigid won’t do anything, because the native is the grabbable one.
  • iMSTK rigid will not stay where it should be (for example, won’t remain inside a deformable net, because it will follow the native rigid, that didn’t collide in the net).

That’s why I think that options 1 or 3 are the best ones. I will work in what you said, because it is the option 1.

Okay, is working in some test scenes, I’m going to check what is happening in my scene setup.

Okay, I found it. I added (some weeks ago) to Grasping.cs the option of making a One-Way grasp, and this seems to be another of the problems. Only two-way grasping is working for me to grasp rigids.