Hand Gesture

Introduce

VIVE Wave™ XR plugin provides the Hand feature in the XRSDK and Essence packages. (refer to Wave XR Plugin Packages).

The Hand functionality within the VIVE Wave™ XR plugin offers two key features: Hand Tracking and Hand Gesture.

Once you have imported the Essence package and enabled Hand Gesture, you can access both System (Default) and Custom Gestures.

The System (Default) gestures are automatically provided by the application at runtime after activating Hand Gesture.

The Custom gestures require configuration within the Unity Editor. While some predefined options exist, custom gestures are calculated in real-time based on Hand Tracking data. Therefore, you only need to activate Hand Tracking if you intend to utilize custom gestures. (Refer to Hand Tracking.)

Refer to Wave Hand Gesture Interface about the Hand Gesture API.

Modify AndroidManifest.xml

To enable the Hand feature, you have to add below content to your AndroidManifest.xml.

<uses-feature android:name="wave.feature.handtracking" android:required="true" />

VIVE Wave™ XR plugin provides the Project Settings > XR Plug-in Management > WaveXRSettings > Hand option to modify the AndroidManifest.xml and enable Hand Tracking automatically. Note that the option does NOT enable the Hand Gesture.

Note

Enabling Hand feature consumes additional power.

Golden Sample

Note

You have to enable VIVE Wave™ XR plugin from Project Settings > XR Plugin-in Management > Android tab > Wave XR to run the VIVE Wave™ SDK application in VIVE Focus3 or VIVE XR Elite.

Follow steps below to create your first hand gesture sample. You will have below contents in the sample.

  • A Camera with HMD tracking. (Refer to XR Rig for more detail.)
  • A text bar shows the system (default) gesture.
  • A text bar show the custom gesture.
  • Hand models. (Refer to Hand Model for more detail.)
  1. Create a new scene named HandGestureSample and remove the Main Camera.
  2. Configure Environment:

  • HMD Pose: Drag the Wave Rig prefab from Packages > VIVE Wave XR Plugin - Essence > Runtime > Prefabs to HandGestureSample.

    ../_images/16.png

  • Default Gesture: Add the Hand Manager to Wave Rig. Select the Initial Start option and Gesture types in the Gesture Options.

    ../_images/28.png

  • Custom Gesture: Add the Custom Gesture Provider to Wave Rig. While some predefined gestures exist, you can refer to Customize Single-Hand Gesture and Customize Dual-Hand Gesture about customizing other gestures.

    ../_images/33.png

  1. Add Hand Models: To make the hand models follow the camera, place them under Wave Rig > Camera Offset.

    Drag the WaveHandLeft and WaveHandRight prefabs from Asset > Wave > Essence > Hand > Model > {version} > Resources > Prefabs to beneath the Camera Offset.

../_images/19.png ../_images/29.png
  1. Add Gesture Canvas:

  • Add a text from the menu item GameObject > UI > Text. Remove the Event System from HandGestureSample. Rename the Text GameObject to DefaultGesture and adjust the Rect Transform as shown.

    ../_images/31.png

  • In the Canvas object, change Render Mode to World and adjust the Rect Transform as shown.

    ../_images/18.png

  • Add another text named CustomGesture and adjust the Rect Transform as shown.

    ../_images/32.png
  1. Retrieve Gesture: Implement a script to retrieve the default or custom gestures.

  • Go to Canvas > DefaultGesture, select Add Component > New script with the name CurrentGesture, and click Create and Add. You can find the CurrentGesture at Assets.

  • Copy and paste the provided code into CurrentGesture script.

    using UnityEngine;
using UnityEngine.UI;
using Wave.Essence.Hand;
using Wave.Essence.Hand.StaticGesture;

[RequireComponent(typeof(Text))]
public class CurrentGesture : MonoBehaviour
{
    public enum GestureType { Default = 0, Custom = 1 }

    public GestureType m_Gesture = GestureType.Default;
    private Text m_Text = null;

    private void Awake() { m_Text = GetComponent<Text>(); }
    private void Update()
    {
        if (m_Text != null)
            m_Text.text = GetGesture();
    }

    private string GetGesture()
    {
        string gesture = m_Gesture.ToString();

        if (m_Gesture == GestureType.Default && HandManager.Instance != null)
        {
            var left = HandManager.Instance.GetHandGesture(true);
            var right = HandManager.Instance.GetHandGesture(false);
            gesture += " Left: " + left + ", Right: " + right;
        }
        if (m_Gesture == GestureType.Custom && CustomGestureProvider.Current != null)
        {
            var left = CustomGestureProvider.Current.GetCustomGesture(true);
            var right = CustomGestureProvider.Current.GetCustomGesture(false);
            gesture += " Left: " + left + ", Right: " + right;
        }

        return gesture;
    }
}

  • Go to Canvas > CustomGesture. Click Add Component with CurrentGesture and select the Gesture to Custom.

    ../_images/35.png ../_images/36.png

  1. Run Application: In the application runtime, you can see the gesture information of both hands.

    ../_images/34.png

System Gesture

Before using the Hand Gesture API, add the HandManager component from the menu item Wave > GameObject > Add Hand Manager.

../_images/101.png

To retrieve the System Gesture types, you have to select the Gesture Options > Initial Start to enable the Hand Gesture feature by default and specify required types.

../_images/14.png

You can simply retrieve the current DEFAULT hand gesture by using following code.

using Wave.Essence.Hand;

// Retrieves the right hand default gesture.
HandManager.GestureType defaultType = (
    HandManager.Instance != null ?
        HandManager.Instance.GetHandGesture(false) : HandManager.GestureType.Invalid
);

Custom Gesture

You can customize gesture types by adding the CustomGestureProvider component along with HandManager. You can refer to the sample Assets > Wave > Essence > Interaction > Mode > {version} > Demo > NaturalHand for the usage of Hand Gesture.

../_images/061.png

You can simply retrieve the current DEFAULT or CUSTOM hand gesture by using following code.

using Wave.Essence.Hand.StaticGesture;

// Retrieves the right hand DEFAULT or custom gesture.
string customType = WXRGestureHand.GetSingleHandGesture(false); // false: Right, true: Left
// Retrieves the dual hand custom gesture. No DEFAULT dual hand gesture.
string dualhandType = WXRGestureHand.GetDualHandGesture();
../_images/15.png

Note

When using the WXRGestureHand functions, the default gesture types have higher priority than custom types. You will retrieve a custom gesture type only when no default gesture type is matched.

You can register an event listener which listens GestureType to On Left Gesture or On Right Gesture of Custom Gesture Provider.

You can customize a gesture type by right-clicking on the Project folder > Create > Wave > Single/Dual Hand Gesture.

../_images/071.png

Customize Single-Hand Gesture

Single-hand gesture attributes contains:

../_images/081.png

Customize Dual-Hand Gesture

Dual-hand gesture attributes contains:

../_images/091.png