Select Hand Engine

Starting from v0.9.2, Vive Hand Tracking SDK added experimental support for multiple backend engines for hand detection. By default, 3 implementations are provided in the SDK:

  1. ViveHandTrackingEngine is the original SDK implementation, using binaries in the SDK. This is supported on all devices listed in Supported Hardware section, except Vive Focus 3.
  2. WaveVRHandEngine is the engine that calls WaveVR Hand API.
  3. EditorEngine is a simulation engine that can be used in Unity Editor without real HMD. This allows developer to simulate hand skeleton and gestures using Unity inspector.

See later sections for differences between engines.

In Vive Hand Tracking -> Settings menu, you can set a list of candidate engines, all engines are checked from top to bottom. The first engine, that is supported on current platform and starts successfully, is used for detection.

../_images/engine.png

WaveVR Hand Engine

WaveVRHandEngine calls WaveVR Hand API (new in WaveVR 3.2.0) to get raw hand results. This provides easier integration for cross-platform VR development, and has little effort for existing users of Vive Hand Tracking API. Developers can also benifit from utility scripts provided in Vive Hand Tracking SDK while using native hand API from WaveVR SDK.

Hand tracking feature in WaveVR are implemented by device vendor, and not all WaveVR devices support this feature. HTC Vive Focus/Focus Plus/Focus 3 supports this feature. Devices from other vendors may also support WaveVR Hand API.

Note

WaveVR Hand API has breaking changes in WaveVR 4.0. So WaveVR SDK and device ROM must be same version (both 3.2.0 or both 4.x) to use WaveVR Hand API. See WaveVR Compatibility for details.

WaveVR Hand Engine vs Vive Hand Tracking Engine

Tip

Order of ViveHandTrackingEngine and WaveVRHandEngine only matters on Vive Focus/Focus Plus. On other WaveVR devices with WaveVRHandEngine support (e.g. HTC Vive Focus 3), ViveHandTrackingEngine is not supported.

Comparison WaveVR Hand Engine Vive Hand Tracking Engine
Architecture Always arm64 Same as VR app
Detection Accuracy Base on ROM version Base on SDK version
Pre-defined gesture classification All gestures but Victory All gestures

Benefit of listing ViveHandTrackingEngine before WaveVRHandEngine:

  • Get consistent hand tracking performance for all supported ROM versions.
  • You can update SDK to get performance improvement.
  • Get new features early.

Benefit of listing WaveVRHandEngine before ViveHandTrackingEngine:

  • Enjoy smaller latency if your apk does not support arm64 architecture.
  • Hand Tracking performance gets improved by ROM update while apk is not changed.

Using Editor Engine

Tip

It’s recommended to always put EditorEngine as last engine, since simulation cannot represent actual performance with real devices.

When editor engine is enabled, it creates a GameObject called Editor Engine Controller in the scene. This game object contains a EditorHandSimulator script, which can be used to modify status of both hands.

../_images/simulator.png
  • Follow Head: Set to true if hand transform are local to head. Set to false if hand transform is global.
  • Hand Base Position: Change this position moves both hands together.
  • Left/Right Hand Position: Change position of either hand.
  • Left/Right Hand Rotation: Change roataion of either hand.
  • Left/Right Hand Confidence: Change confidence of either hand.
  • Left/Right Hand State: Change gesture and finger close/open state of either hand.
    • Preset: Change gesture to one of preset: invsible, pre-defined gestures or pinch.
    • Thumb State: Change thumb state to close, open or pinch.
    • Index State: Change index state to close, open, relax or pinch.
    • Middle/Ring/Pinky State: Change finger state to close, open or relax.

Note

If GestureProvider is not in Skeleton mode, finger states and hand rotation are hidden, since these are not used in Point2D/Point3D mode.

Add Custom Hand Engine

You can also implement your own engine if want to use other SDKs with the same API from Vive Hand Tracking SDK. Define an engine by deriving from HandTrackingEngine class.

public class EngineState {
  public GestureMode Mode = GestureMode.Skeleton;
  public GestureStatus Status = GestureStatus.NotStarted;
  public GestureFailure Error = GestureFailure.None;
  public GestureResult LeftHand = null;
  public GestureResult RightHand = null;
}

public abstract class HandTrackingEngine: ScriptableObject {
  // holds hand state and result, shares same reference to GestureProvider
  protected internal EngineState State = null;

  // if supported for current platform, this is called first
  public abstract bool IsSupported();

  // setup engine once, e.g. grant camera permission, called before StartDetection
  // if setup failed, set status to error
  public abstract IEnumerator Setup();

  // start detection with given option, change state for status and error as return value
  public abstract IEnumerator StartDetection(GestureOption option);

  // update detection result, change state as return value
  public abstract void UpdateResult();

  // stop detection
  public abstract void StopDetection();

  // description used in the editor settings windows
  public virtual string Description() { return ""; }
}

Make sure to add your engine in Vive Hand Tracking - Settings menu. You may need to move your engine to top if you want it to have higher priority.