API Reference¶

Functions
Structs
Enums

Functions ¶

GestureFailure StartGestureDetection(GestureOption *option)¶

Start detection with given option, non-blocking.

option is both in/out parameter. Mode of option may be modified if requirements is not met on the device. The resulting mode is the actual mode used in detection. If option is null, default option will be used (auto backend + best mode).

void StopGestureDetection()¶: Stop the detection. Blocking call until the pipeline is actually stopped.

int GetGestureResult(const GestureResult **points, int *frameIndex)¶

Get detection result in world coordinate. Returns at most one left and one right hand. Return value is number of detections in points, at most 2.

You should call this function periodically to get latest results.

Points:	(return value) A pointer to an array of GestureResult. The pointer is valid until next call to `GetGestureResult()` or `StopGestureDetection()`. The pointer need NOT to be freed.
FrameIndex:	(return value) A pointer to frame index (can be null). This can be used to check if results are updated. Set to -1 if detection is not started or stopped due to error.

Example usage:

const GestureResult* points = NULL;
int frameIndex;
int size = GetGestureResult(&points, &frameIndex);

void UseExternalTransform(bool value)¶

This function should be called before StartGestureDetection() to indicate if caller is providing camera transform or not. Default is false. Call it after StartGestureDetection() has no use.

If set to true, hmd positions are not queried from raw HMD API from OpenVR or WaveVR. The caller is responsible for managing hmd positions using either of the two methods below:

Call SetCameraTransform function every frame to provide HMD transform (recommended)
Apply camera transform to result points after GetGestureResult() function call

This is useful if camera transform is different from raw HMD data, e.g. teleporting. This is also used in Unity & Unreal plugin.

void SetCameraTransform(GestureVector3 position, GestureQuaternion rotation)¶

Only takes effect if UseExternalTransform(true) is called before StartGestureDetection.

Set hmd transform for use. This function should be called regularly if HMD pose is changing. The transform is used until new one is set. Default transform is idendity.

void SetARCoreSession(ArSession *session, ArFrame *frame)¶

This function should be called before StartGestureDetection() to set arcore session and frame instance. Call it after StartGestureDetection() has no use.

If session or frame is set to null, arcore will be disabled. Frame is used to determine if frame is updated or not, and also to acquire camera image continuously. Session and Frame will be reset to null by StopGestureDetection().

Structs ¶

class GestureVector3¶

Struct representing 3D points.

float x¶: x component

float y¶: y component

float z¶: z component

class GestureQuaternion¶

Struct containing rotation information as quaternion.

float x¶: x component

float y¶: y component

float z¶: z component

float w¶: w component

class GestureOption¶

Struct used as input/output parameter for StartGestureDetection().

GestureBackend backend = GestureBackend::GestureBackendAuto ¶: Backend of detection.

Note

GestureBackend is deprecated and will be removed in future release.

GestureMode mode = GestureMode::GestureModeSkeleton ¶: Mode of detection.

Note

GestureMode is deprecated, skeleton mode will be the only supported mode in future release. If you want to use other modes, use GestureResult::position.

int maxFPS = -1¶: Limit max fps of raw detection, only used when value in range [15, 90]. Default is no limit.

Note

This has negative impact on latency. Only use it when VR application fps slows down due to hand tracking.

class GestureResult¶

Struct containing detection result for one hand.

bool isLeft¶: Returns if this hand is left/right.

GestureVector3 position¶: Returns position of palm center, use this if only need hand position instead of 21 joints.

GestureVector3 joints[21]¶

Returns position of the hand joints. Meaning of this field is different based on actual GestureMode.

GestureMode2DPoint

Only first point is used as the position of hand. z is always 0.25.
GestureMode3DPoint

Only first point is used as the position of hand. z is calculated depth.
GestureModeSkeleton

An array of 21 keypoints of the hand. See Hand Positions & Rotations (Modes) for order of the keypoints.

+x is right, +y is up, +z is front. Unit is meter.

GestureQuaternion rotations[21]¶

Returns rotation of the hand joints. Meaning of this field is different based on actual mode.

GestureMode2DPoint & GestureModeSkeleton

Only first element is used as the rotation of hand.
GestureModeSkeleton

Rotation for 21 keypoints of the hand.

Identity rotation (assume hand is five gesture): palm face front and fingers point upward.

GestureType gesture¶: Returns pre-defined gesture type.

float confidence¶: Returns confidence of the hand, within [0, 1].

GesturePinchInfo pinch¶: Returns pinch information.

class GesturePinchInfo¶

Struct containing information of pinch (thumb & index).

float pinchLevel¶: Returns pinch (thumb & index) level of the hand, within [0, 1], higher means more possible to pinch. If you only need a boolean value for pinch or not, use isPinching() instead.

bool isPinching() const¶: Returns if currently pinching or not. If you need a range value within [0, 1], you can use pinchLevel instead.

GestureVector3 pinchStart¶: Returns start position of the pinch ray.

GestureVector3 pinchDirection¶: Returns direction of the pinch ray.

Enums ¶

enum GestureBackend¶

Enum for selecting computation backend.

enumerator GestureBackendAuto = 0¶: Default backend, use GPU on PC and CPU on Android. Recommended.

enumerator GestureBackendCPU = 1¶: Use CPU, not supported on PC

enumerator GestureBackendGPU = 2¶: Use GPU, supported on PC/Android

enum GestureMode¶

Enum for detection mode. Larger mode return more info, but runs more slowly. If a mode is not supported on a device, will fallback to previous supported mode.

API Reference¶

Functions¶

Structs¶

Enums¶

Functions ¶

Structs ¶

Enums ¶