Basic Usage

Please complete steps in the Setup section before running your application.

If you want to use plugin in your map, just drag HandTrackingProvider actor into the map editor.

The below sections introduces the basic funtions used in HandTrackingProvider actor. If you don’t need to customize the setup, you can try out the samples in Using Samples Levels section directly.

Select Engine

This should be called first to select the engine to use. Requires target as ViveHandTracking Component.

You can set options in HandTrackingProvider -> ViveHandTrackingComponent. You can set max fps limit in the option. Mode is deprecated and it’s recommended to use skeleton mode.

Start/Stop detection

StartGestureDetection and StopGestureDetection are provided as blueprint function. Both requires target as ViveHandTracking Component.

In StartGestureDetection, you can choose the mode you prefer as input. The final mode used is modified in the option.

Get Detection Result

When detection is running, detected left and right hand results are available from blueprint function GetLeftHand and GetRightHand.

Each hand contains following members, see sample level and FViveHandTrackingResult class for details:

• points: position of the hand with different meanings for each mode selected.
• rotations: rotation of the hand with different meanings for each mode selected.
• gestureType: pre-defined gesture
• bIsValid: if current hand is detected/visible
• confidence: hand detection confidence
• pinchLevel: pinch (thumb & index) level of the hand, within [0, 1]
• bIsPinching: if currently pinching or not
• pinchStart: start position of the pinch ray
• pinchDirection: forward direction of the pinch ray
• pinchRotation: rotation of the pinch ray
• position: position of palm center
• rotation: rotation of palm center

Get Detection Status

To query current detection mode and status, use blueprint function UViveHandTrackingComponent::GetMode() and UViveHandTrackingComponent::GetStatus().

If current status is error, blueprint function UViveHandTrackingComponent::GetError() is available for querying error details.

And it is also convenient to get current frame index by using blueprint fucntion UViveHandTrackingComponent::GetFrameIndex().

Draw Detected Hands As Skeletons

Vive Hand Tracking plugin provides a sample blueprint to render hands as skeletons using joints and bones (or only a single joint in 2D/3D mode).

Add BasicHand into your map editor.

You can uncheck Show Left Hand or Show Right Hand if you only want to display one hand as skeleton.

Tip

You can set collision type to Query Only for picking up object and Collision Enabled for pushing object. Make sure that object is inherited from PickUpInterface and implemented PickUp and Drop Event. See Blueprints in PickUp folder for details.

Draw Detected Hands As 3D Model

Note

Display hand as 3D model is only supported in skeleton mode.

Vive Hand Tracking plugin provides sample blueprints to render hands as 3D models. Two models are provided in separate bludprints: cartoon model and robot model. These are only supported in skeleton mode.

Add CartoonHand or RobotHand into your map editor.

You can check Hidden in Game for LeftHand or RightHand if you don’t need both hands to display as 3d model.

Hint

CartoonHand and RobotHand have no physics assets currently, so capsule collider is attached to them for simple collision detection. You can also set collision type like BasicHand in details panel.