Unity XR InputDevice - Hand Tracking

Refer to Wave XR Plugin Packages about VIVE Wave™ XR plugin packages.

Wave.OpenXR.InputDeviceHand: Provides an interface to access UnityEngine.XR.Bone.

InputDeviceHand Public Memeber Functions

To use the Wave.OpenXR.InputDeviceHand class, you have to import the XRSDK package.

The InputDeviceHand class provides APIs to check the hand tracking status and retrieve the pose of Bone easily.

ActivateNaturalHand

Activates/deactivates the Hand Tracking.

void ActivateNaturalHand(bool active)

IsHandDevice

Checks if an InputDevice is a Wave Hand.

bool IsHandDevice(InputDevice input, bool isLeft)

IsAvailable

Checks if there is an available hand.

bool IsAvailable(bool isLeft) // isLeft is true for left hand.

IsTracked

Checks if a hand has valid poses.

bool IsTracked(bool isLeft) // isLeft is true for left hand.

GetPalm

Retrieves the bone of Palm.

Bone GetPalm(bool isLeft)

GetWrist

Retrieves the bone of Wrist.

Bone GetWrist(bool isLeft)

GetFingerBones

Retrieves the bone list of a finger.

Thumb finger has only 4 bones: joint0, joint1, joint2 and tip.

Index, Middle, Ring and Pinky have 5 bones: joint0, joint1, joint2, joint3 and tip.

List<Bone> GetFingerBones(bool isLeft, HandFinger finger)

/// Sample code
// Retrieves the Right Index joint1 bone.
var indexBones = InputDeviceHand.GetFingerBones(false, HandFinger.Index);
if (indexBones.Count > 1) // joint0 and joint1
{
        Bone indexJoint1 = indexBones[1];
}

// Retrieves the Right Thumb tip.
var thumbBones = InputDeviceHand.GetFingerBones(false, HandFinger.Thumb);
if (thumbBones.Count > 3) // joint0, joint1, join2 and tip
{
        Bone thumbTip = thumbBones[3];
}

GetHandScale

Retrieves the scale of wrist.

bool GetHandScale(bool isLeft, out Vector3 scale)

GetHandConfidence

Retrieves the wrist confidence which is a 0~1 float value where 1 means the most accurate.

bool GetHandConfidence(bool isLeft, out float confidence)

GetWristLinearVelocity

Retrieves the left/right wrist velocity.

bool GetWristLinearVelocity(bool isLeft, out Vector3 velocity)

GetWristAngularVelocity

Retrieves the left/right wrist angular velocity.

bool GetWristAngularVelocity(bool isLeft, out Vector3 velocity)

GetHandMotion

Retrieves current left/right hand motion.

bool GetHandMotion(bool isLeft, out HandMotion motion)

GetHandHoldRole

Retrieves the role of left/right hand while holding.

bool GetHandHoldRole(bool isLeft, out HandHoldRole role)

GetHandHoldType

Retrieves the type of left/right handheld object.

bool GetHandHoldType(bool isLeft, out HandHoldType type)

GetPinchOrigin

Retrieves the origin of left/right hand pinch.

bool GetPinchOrigin(bool isLeft, out Vector3 origin)

GetPinchDirection

Retrieves the direction of left/right hand pinch.

bool GetPinchDirection(bool isLeft, out Vector3 direction)

GetPinchStrength

Retrieves the strength of left/right hand pinch.

bool GetPinchStrength(bool isLeft, out float strength)

GetPinchThreshold

Retrieves the system default threshold always used to judge if a “selection” happens when the pinch strength value is greater than the threshold.

bool GetPinchThreshold(out float threshold)

GetPinchOffThreshold

Retrieves the system default threshold always used to judge if a “release” happens when a hand is pinching and the pinch strength value is less than the threshold.

Note that the Pinch Off Threshold should NOT be greater than the Pinch Threshold.

bool GetPinchOffThreshold(out float threshold)

IsHandPinching

Checks if a hand is pinching currently.

bool IsHandPinching(bool isLeft)

GetGraspStrength

Retrieves the strength of left/right hand grasp.

bool GetGraspStrength(bool isLeft, out float strength)

IsHandGrasping

Checks if a hand is grasping currently.

bool IsHandGrasping(bool isLeft)

Hand Manager

Before using the Hand Tracking API, add the HandManager component from the menu item Wave > GameObject > Add Hand Manager.

To use Hand Tracking API, check if the Hand Manager existed by using the following code.

using Wave.Essence.Hand;

if (HandManager.Instance != null)

Hand Manager Class Reference

Namespace Wave.Native from Wave Native package is used by Hand Manager.

Hand Manager Public Types

HandType

enum HandType {
    Right = 0,
    Left = 1
}

HandJoint

enum HandJoint {
    Palm = WVR_HandJoint.WVR_HandJoint_Palm,
    Wrist = WVR_HandJoint.WVR_HandJoint_Wrist,
    Thumb_Joint0 = WVR_HandJoint.WVR_HandJoint_Thumb_Joint0,
    Thumb_Joint1 = WVR_HandJoint.WVR_HandJoint_Thumb_Joint1,
    Thumb_Joint2 = WVR_HandJoint.WVR_HandJoint_Thumb_Joint2,
    Thumb_Tip = WVR_HandJoint.WVR_HandJoint_Thumb_Tip,
    Index_Joint0 = WVR_HandJoint.WVR_HandJoint_Index_Joint0,
    Index_Joint1 = WVR_HandJoint.WVR_HandJoint_Index_Joint1,
    Index_Joint2 = WVR_HandJoint.WVR_HandJoint_Index_Joint2,
    Index_Joint3 = WVR_HandJoint.WVR_HandJoint_Index_Joint3,
    Index_Tip = WVR_HandJoint.WVR_HandJoint_Index_Tip,
    Middle_Joint0 = WVR_HandJoint.WVR_HandJoint_Middle_Joint0,
    Middle_Joint1 = WVR_HandJoint.WVR_HandJoint_Middle_Joint1,
    Middle_Joint2 = WVR_HandJoint.WVR_HandJoint_Middle_Joint2,
    Middle_Joint3 = WVR_HandJoint.WVR_HandJoint_Middle_Joint3,
    Middle_Tip = WVR_HandJoint.WVR_HandJoint_Middle_Tip,
    Ring_Joint0 = WVR_HandJoint.WVR_HandJoint_Ring_Joint0,
    Ring_Joint1 = WVR_HandJoint.WVR_HandJoint_Ring_Joint1,
    Ring_Joint2 = WVR_HandJoint.WVR_HandJoint_Ring_Joint2,
    Ring_Joint3 = WVR_HandJoint.WVR_HandJoint_Ring_Joint3,
    Ring_Tip = WVR_HandJoint.WVR_HandJoint_Ring_Tip,
    Pinky_Joint0 = WVR_HandJoint.WVR_HandJoint_Pinky_Joint0,
    Pinky_Joint1 = WVR_HandJoint.WVR_HandJoint_Pinky_Joint1,
    Pinky_Joint2 = WVR_HandJoint.WVR_HandJoint_Pinky_Joint2,
    Pinky_Joint3 = WVR_HandJoint.WVR_HandJoint_Pinky_Joint3,
    Pinky_Tip = WVR_HandJoint.WVR_HandJoint_Pinky_Tip,
}

HandMotion

enum HandMotion {
    None = WVR_HandPoseType.WVR_HandPoseType_Invalid,
    Pinch = WVR_HandPoseType.WVR_HandPoseType_Pinch,
}

HandHoldRole

enum HandHoldRole {
        None = WVR_HandHoldRoleType.WVR_HandHoldRoleType_None,
        Main = WVR_HandHoldRoleType.WVR_HandHoldRoleType_MainHold,
        Side = WVR_HandHoldRoleType.WVR_HandHoldRoleType_SideHold,
}

HandHoldType

enum HandHoldType {
        None = WVR_HandHoldObjectType.WVR_HandHoldObjectType_None,
        Gun = WVR_HandHoldObjectType.WVR_HandHoldObjectType_Gun,
        OCSpray = WVR_HandHoldObjectType.WVR_HandHoldObjectType_OCSpray,
        LongGun = WVR_HandHoldObjectType.WVR_HandHoldObjectType_LongGun,
        Baton = WVR_HandHoldObjectType.WVR_HandHoldObjectType_Baton,
        FlashLight = WVR_HandHoldObjectType.WVR_HandHoldObjectType_FlashLight,
}

TrackerStatus

enum TrackerStatus {
    // Initial, can call Start API in this state.
    NotStart,
    StartFailure,

    // Processing, should NOT call API in this state.
    Starting,
    Stopping,

    // Running, can call Stop API in this state.
    Available,

    // Do nothing.
    NoSupport
}

TrackerType

enum TrackerType {
    Natural = WVR_HandTrackerType.WVR_HandTrackerType_Natural,
    Electronic = WVR_HandTrackerType.WVR_HandTrackerType_Electronic,
}

Hand Manager Private Data Fields

uint m_NaturalHandJointCount

WVR_HandTrackingData_t m_NaturalHandTrackerData

WVR_HandJointData_t m_NaturalHandJointDataLeft, m_NaturalHandJointDataRight

WVR_Pose_t[] s_NaturalHandJointsPoseLeft, s_NaturalHandJointsPoseRight

Hand Manager Public Memeber Functions

StartHandTracker

Starts Hand Tracking with specific tracker. We support natural hand tracking only.

void StartHandTracker(TrackerType tracker)

StopHandTracker

Stops Hand Tracking with specific tracker. We support natural hand tracking only.

void StopHandTracker(TrackerType tracker)

GetHandTrackerStatus

Retrieves current hand tracking status.

TrackerStatus GetHandTrackerStatus()

RestartHandTracker

Restarts the Hand Tracking.

void RestartHandTracker()

IsHandPoseValid

Checks if the left/right hand pose is valid.

bool IsHandPoseValid(bool isLeft)

GetHandConfidence

Retrieves the confidence value (0~1) where 1 means the most accurate of Hand Tracking.

float GetHandConfidence(bool isLeft)

GetJointPosition

Retrieves the joint position in Vector3.

bool GetJointPosition(HandJoint joint, ref Vector3 position, bool isLeft)

GetJointRotation

Retrieves the joint rotation in Quaternion.

bool GetJointRotation(HandJoint joint, ref Quaternion rotation, bool isLeft)

GetHandMotion

Checks if the left/right hand motion is Pinch.

HandMotion GetHandMotion(bool isLeft)

GetHandHoldRole

Retrieves the hand role when holding.

HandHoldRole GetHandHoldRole(bool isLeft)

GetHandHoldType

Retrieves the object type when holding.

HandHoldType GetHandHoldType(bool isLeft)

GetPinchOrigin

Retrieves the left/right hand pinch origin in Vector3.

bool GetPinchOrigin(ref Vector3 origin, bool isLeft)

GetPinchDirection

Retrieves the left/right hand pinch direction in Vector3.

bool GetPinchDirection(ref Vector3 direction, bool isLeft)

GetPinchStrength

Retrieves the left/right hand pinch motion strength (0~1) where 1 means the thumb tip and index tip is touching.

float GetPinchStrength(bool isLeft)

GetPinchThreshold

Retrieves the system default threshold always used to judge if a “selection” happens when the pinch strength value is greater than the threshold.

float GetPinchThreshold(TrackerType tracker)

GetPinchOffThreshold

Retrieves the system default threshold always used to judge if a “release” happens when a hand is pinching and the pinch strength value is less than the threshold.

Note that the Pinch Off Threshold should NOT be greater than the Pinch Threshold.

float GetPinchOffThreshold(TrackerType tracker)

IsHandPinching

Checks if a hand is pinching currently.

bool IsHandPinching(TrackerType tracker, bool isLeft)

GetGraspStrength

Retrieves the left/right hand’s grasp strength.

float GetGraspStrength(TrackerType tracker, bool isLeft)

IsHandGrasping

Retrieves the left/right hand’s grasp status.

bool IsHandGrasping(TrackerType tracker, bool isLeft)

FuseWristPositionWithTracker

To enhance the stability of wrist position tracking by fusing the position with a tracker.

void FuseWristPositionWithTracker(bool fuse)

IsWristPositionFused

Checks if the stability of wrist position is enhanced by fusing the position with a tracker.

bool IsWristPositionFused()

Wave Native Types

WVR_HandTrackingData_t

struct WVR_HandTrackingData_t {
    long timestamp;
    WVR_HandJointData_t right;    /**< The hand tracker data of right hand, refer to @ref WVR_HandJointData_t. */
    WVR_HandJointData_t left;     /**< The hand tracker data of left hand, refer to @ref WVR_HandJointData_t. */
}

WVR_HandJointData_t

struct WVR_HandJointData_t {
    bool isValidPose;    /**< The label of valid(true)/invalid(false) pose. */
    float confidence;    /**< The hand confidence value. */
    uint jointCount;     /**< Specify the size of the @ref WVR_Pose_t array. */
    IntPtr joints;       /**< The array of the @ref WVR_Pose_t. */
    WVR_Vector3f_t scale; /**< defualt is 1. */
    WVR_Vector3f_t wristLinearVelocity;
    WVR_Vector3f_t wristAngularVelocity;
}

WVR_Pose_t

WVR_Vector3f_t position;

WVR_Quatf_t rotation;

WVR_Vector3f_t

struct WVR_Vector3f_t {
    float v0;
    float v1;
    float v2;
}

WVR_Quatf_t

struct WVR_Quatf_t {
    float w;
    float x;
    float y;
    float z;
}