Unity Scene Perception

Introduction

Scene Perception is a feature which facilitates the development of Mixed Reality applications by bringing in spatial information from the users’ surroundings into the virtual environment.

For a comprehensive understanding of the conceptual aspects related to our MR API, refer to the Tutorial for MR Contents page.

Note

The Scene Perception feature is current in Beta, and the currently supported aspects of this feature are Scene Planes and Spatial Anchors.

Contents Prerequisite How to use Using Scene Perception Scene Planes Scene Object Scene Mesh Spatial Anchor Other Useful Functions Resources Points to take note of and Known Issues

Prerequisite

1. In Unity Editor, go to Project Settings -> Wave XR -> Essence to import the Scene Perception feature pack.

Scene Perception Feature Pack

2. You have to add below declaration to your AndroidManifest.xml to enable the Scene Perception feature.

<uses-feature android:name="wave.feature.sceneperception" android:required="true" />

You can also select Project Settings > XR Plug-in Management > WaveXRSettings > Scene Perception > Enable Scene Perception, which will add the above declaration to the AndroidManifest.xml automatically.

3. For Scene Mesh: In order to read scene mesh data after enabling the Scene Perception feature, a wave permission declaration is needed for getting permission from the user.

<uses-permission android:name="wave.permission.GET_SCENE_MESH">

You can also select Project Settings > XR Plug-in Management > WaveXRSettings > Scene Perception > Enable Scene Mesh, which will add the above declaration to the AndroidManifest.xml automatically.

How to use

1. Add a ScenePerceptionManager Component to a GameObject in your scene.
2. Assign a GameObject reference which represents the Tracking Origin as it will be needed for pose correction.

Scene Perception Manager component attached to a GameObject, and with the root of the VR Camera Rig assigned as the Tracking Origin

3. Use the APIs and helper functions provided by ScenePerceptionManager to start using Scene Perception.

Note

To call the APIs and helper functions provided by ScenePerceptionManager, add the Wave.Native and Wave.Essence.ScenePerception namespaces to your script.

In the following sections, we will be referring to sample code which are slightly modified versions of excerpts from the ScenePerceptionDemoManager.cs script which can be found in the imported feature pack resources.

Using Scene Perception

To use scene perception, follow these steps:

1. Check whether Scene Perception is supported on the device or not.
2. Call StartScene() to initialize the resources required to start using Scene Perception.
3. Call other APIs and helper functions to use the Scene Perception feature.
4. Call StopScene() once you have finished using Scene Perception.

//Sample code that leverages Unity MonoBehavior lifecycle for starting and stopping Scene Perception

public ScenePerceptionManager scenePerceptionManager = null;

private void OnEnable()
{
  //Check whether feature is supported on device or not
  if ((Interop.WVR_GetSupportedFeatures() & (ulong)WVR_SupportedFeature.WVR_SupportedFeature_ScenePerception) != 0)
  {
    WVR_Result result = scenePerceptionManager.StartScene();
    if (result == WVR_Result.WVR_Success)
    {
      result = scenePerceptionManager.StartScenePerception(WVR_ScenePerceptionTarget.WVR_ScenePerceptionTarget_2dPlane); //Start perceiving 2D planes

      if (result == WVR_Result.WVR_Success)
      {
        //Examples of things to do here:
        //Scene Planes
        //- Call scenePerceptionManager.GetScenePerceptionState() to see the current WVR_ScenePerceptionState of a specific WVR_ScenePerceptionTarget
        //- Call scenePerceptionManager.StopScenePerception() to stop perceiving a specific WVR_ScenePerceptionTarget
        //
        //Spatial Anchors
        //- Call scenePerceptionManager.GetSpatialAnchors() to retrieve all handles of existing anchors
        //- Using the retrieved handles, call scenePerceptionManager.GetSpatialAnchorState() to get information of the Spatial Anchors
      }
    }
  }
  else
  {
    Log.e(LOG_TAG, "Scene Perception is not available on the current device.");
  }
}

private void OnDisable()
{
  if (isSceneCMPStarted)
  {
    scenePerceptionManager.StopScene();
  }
}

Scene Planes

Scene Planes are flat planes that exist in the users’ surroundings. Assuming that you have started using Scene Perception successfully, follow these tips to use Scene Planes:

1. When perception state for 2D Planes is WVR_ScenePerceptionState.WVR_ScenePerceptionState_Completed, you can call scenePerceptionManager.GetScenePlanes() to retrieve the data of the perceived Scene Planes.
2. You can use the helper functions ScenePerceptionManager.IsUUIDEqual(), ScenePerceptionManager.ScenePlaneExtent2DEqual() and ScenePerceptionManager.ScenePlanePoseEqual() for comparing the uuid, extent and pose of Scene Planes respectively.
3. You can use the helper function scenePerceptionManager.ApplyTrackingOriginCorrectionToPlanePose() to convert the pose of a Scene Plane to world space position and rotation that are usable in the Unity coordinate system.

//Sample Code for getting the 2D plane perception state

public void ScenePerceptionGetState()
{
  WVR_ScenePerceptionState latestPerceptionState = WVR_ScenePerceptionState.WVR_ScenePerceptionState_Empty;
  WVR_Result result = scenePerceptionManager.GetScenePerceptionState(WVR_ScenePerceptionTarget.WVR_ScenePerceptionTarget_2dPlane, ref latestPerceptionState);
  if (result == WVR_Result.WVR_Success)
  {
    //Check perception state here
    if(latestPerceptionState == WVR_ScenePerceptionState.WVR_ScenePerceptionState_Completed)
    {
      //When perception for 2d planes is completed, you can retrieve the data of the perceived Scene Planes
      WVR_ScenePlane[] latestScenePlanes;
      result = scenePerceptionManager.GetScenePlanes(ScenePerceptionManager.GetCurrentPoseOriginModel(), out latestScenePlanes);

      if (result == WVR_Result.WVR_Success)
      {
        //Handle the retrieved data of the Scene Planes here
        //For example:
        //- Cache Scene Plane data for future reference
        //- Compare extent and pose of cached Scene Planes with the freshly retrieved ones to see if there are any updates
      }
    }
  }
}

As for the data of Scene Planes, you can refer to the struct and enum definitions as follows:

//Basic overview of Scene Plane data

public struct WVR_ScenePlane
{
  public WVR_Uuid uuid;                     //An ID unique to the Scene Plane
  public WVR_Uuid parentUuid;               //The ID of the parent of the current Scene Plane
  public UInt64 meshBufferId;               //The ID of the mesh buffer of the current Scene Plane, can be used with the API GetSceneMeshBuffer() in ScenePerceptionManager
  public WVR_Pose_t pose;                   //The pose of the Scene Plane
  public WVR_Extent2Df extent;              //The extends of the Scene Plane (width and height)
  public WVR_ScenePlaneType planeType;      //For filtering Scene Planes by type.
  public WVR_ScenePlaneLabel planeLabel;    //For filtering Scene Planes by label.
}

public enum WVR_ScenePlaneType
{
  WVR_ScenePlaneType_Unknown = 0,
  WVR_ScenePlaneType_HorizontalUpwardFacing = 1,
  WVR_ScenePlaneType_HorizontalDownwardFacing = 2,
  WVR_ScenePlaneType_Vertical = 3,
  WVR_ScenePlaneType_Max = 0x7FFFFFFF         //Can be used as the input value when you do not need to filter the results from GetScenePlanes().
}

public enum WVR_ScenePlaneLabel
{
  WVR_ScenePlaneLabel_Unknown = 0,
  WVR_ScenePlaneLabel_Floor = 1,
  WVR_ScenePlaneLabel_Ceiling = 2,
  WVR_ScenePlaneLabel_Wall = 3,
  WVR_ScenePlaneLabel_Desk = 4,
  WVR_ScenePlaneLabel_Couch = 5,
  WVR_ScenePlaneLabel_Door = 6,
  WVR_ScenePlaneLabel_Window = 7,
  WVR_ScenePlaneLabel_Stage = 8,
  WVR_ScenePlaneLabel_Max = 0x7FFFFFFF        //Can be used as the input value when you do not need to filter the results from GetScenePlanes().
}

Understanding Scene Plane poses and extents

The pose retrieved from WVR_ScenePlane is based on the OpenGL coordinate system and should be converted before it can be used in the Unity coordinate system. You can do so by calling the function scenePerceptionManager.ApplyTrackingOriginCorrectionToPlanePose(), which outputs the converted Scene Plane position and rotation as Vector3 and Quaternion objects respectively.

Assuming that pose is converted and can be used in the Unity coordinate system, a Scene Plane with an identity pose (i.e. position at (0,0,0) and an identity rotation) and has both width and height equal to 1 (in meters) can be represented by a Quad in Unity that faces the +Z direction at identity pose with its scale equal to Vector3.one:

A Quad facing the +Z direction in Unity

The above rule applies to all planes regardless of its WVR_ScenePlaneLabel or WVR_ScenePlaneType. Hence, when applying the pose of a Scene Plane to an object, you have to make sure that the orientation of the target object at identity pose is correct.

For example, you have a plane asset that faces upwards at identity pose which you plan to use as a placeholder object for a Scene Plane labeled as a WVR_ScenePlaneLabel_Desk:

A plane asset that faces horizontally upwards at identity pose

If you apply the converted pose of the Scene Plane to the plane asset directly, you will find that the plane asset is rotated more than expected by -90 degrees around the X-Axis. This is because the orientation of the plane asset at identity pose is equal to that of a quad primitive with a -90 degrees rotation around the X-Axis:

The plane asset is actually a Quad rotated by -90 degrees around the X-Axis

To make the rotation of the plane asset meet your expectations, you have to either apply a rotational offset to the converted pose of the Scene Plane, or you can simply add a parent GameObject for the plane asset, add a rotational offset to the plane asset itself, and apply the converted pose of the Scene Plane to the parent object instead.

A +90 degrees rotational offset is added to the plane asset

You can now see that when the parent GameObject is at identity pose, the orientation of the plane asset matches that of the Quad primitive in Unity, which means that the converted pose of the Scene Plane can now be applied to its parent object directly.

Scene Object

The Scene Object feature is able to get the objects detected by the ScenePerception system. The primary classes involved are ScenePerceptionManager, SceneObject, ScenePerceptionObjectTools and MeshTools. These components work together to retrieve, manage, and visualize 3D objects within a scene, including generating necessary mesh data, UV mapping, and object transformation handling.

Key Components

• ScenePerceptionManager Manages scene perception tasks, including retrieving SceneObject instances via the GetSceneObjects method. Interacts with native APIs to fetch scene data, including the vertices and indices necessary for rendering 3D objects.
• SceneObject Represents a 3D object detected in the scene, containing raw vertex and index data. This data is used to generate meshes that can be rendered within a Unity scene. SceneObject includes metadata like UUIDs, poses (position and rotation), and extents, which are used to place and size objects in the scene.
• ScenePerceptionObjectTools Provides utility methods for generating and manipulating 3D meshes based on the data retrieved from SceneObject. Includes methods for converting tracking space positions to world space, generating mesh data from vertices and indices, and handling UV mapping for textures.
• MeshTools A utility class contains the method to generates UV coordinates for meshes, ensuring that textures are correctly applied to 3D objects.

Detailed Breakdown

SceneObject and Mesh Generation

SceneObject instances retrieved by ScenePerceptionManager typically contain only the raw vertex and index data required to render the object. These objects do not include UV mappings, which are essential for texturing. The MeshTools class is utilized to generate the UVs necessary for proper texture application.

For example, the GenerateQuadUVs method in MeshTools generates UV coordinates for simple quad meshes. When more complex meshes are needed, ScenePerceptionObjectTools.GenerateSceneObjectMesh creates a mesh from the vertex and index data stored in SceneObject.

Workflow with ScenePerceptionManager

• Retrieving Scene Objects: ScenePerceptionManager.GetSceneObjects fetches the current SceneObject instances from the scene perception system. These objects are then processed to generate the corresponding 3D meshes for rendering.
• Mesh Generation and UV Mapping: The raw data from SceneObject is used to generate meshes. UV coordinates are generated using the MeshTools utility, ensuring that textures can be correctly applied to the mesh.
• Object Transformation: The pose (position and rotation) of each SceneObject is converted from tracking space to world space using ScenePerceptionObjectTools.TrackingSpaceToWorldSpace. This ensures that objects are correctly placed within the Unity scene, adhering to the user’s perspective and environment.

Functions/Methods/Properties

Below is a list of the public functions, methods, and properties related to SceneObject:

ScenePerceptionManager

// Retrieves the list of SceneObject instances detected in the current scene. The method uses the pose origin model to determine the coordinate space.
WVR_Result GetSceneObjects(TrackingOriginModeFlags originMode, out SceneObject[] objects)

SceneObject

public class SceneObject
{
    // The unique identifier for the SceneObject.
    public readonly WVR_Uuid uuid;
    // The UUID of the parent object, if applicable.
    public readonly WVR_Uuid parentUuid;
    // An identifier for the mesh buffer associated with this object.
    public readonly ulong meshBufferId;
    // Contains the position (as a Vector3) and rotation (as a Quaternion) that represent the object's transformation in space.
    public readonly Pose pose;  // the center pose of the object
    // Describes the dimensions (width, height, depth) of the object in 3D space.
    public readonly Vector3 extent;  // width, height, depth
    // A string label that provides a human-readable name or description of the object.
    public readonly string semanticName;
}

ScenePerceptionObjectTools

// Generates a mesh for a given SceneObject, applying the material and optionally attaching a collider.
static GameObject GenerateSceneObjectMesh(ScenePerceptionManager manager, SceneObject sceneObject, Material meshMaterial, bool attachMeshCollider = false, Transform trackingOrigin = null)

static GameObject GenerateSceneObjectExtentMesh(SceneObject sceneObject, Material meshMaterial, bool attachMeshCollider = false, Transform trackingOrigin = null): Creates a mesh that represents the bounding box (extent) of the SceneObject.

static bool TrackingSpaceToWorldSpace(Transform trackingOrigin, Vector3 trackingPos, Quaternion trackingRot, out Vector3 worldPos, out Quaternion worldRot): Converts a position and rotation from tracking space to world space using a specified origin.

Example Usage

// Assume scenePerceptionManager is initialized and sceneObjects is retrieved
SceneObject[] sceneObjects;
if (scenePerceptionManager.GetSceneObjects(ScenePerceptionManager.GetTrackingOriginModeFlags(), out sceneObjects) == WVR_Result.WVR_Success)
{
    foreach (var sceneObject in sceneObjects)
    {
        // Generate the mesh for the SceneObject
        GameObject sceneObjectMesh = ScenePerceptionObjectTools.GenerateSceneObjectMesh(
            scenePerceptionManager,
            sceneObject,
            someMaterial,
            false, // Attach mesh collider (optional)
            someTrackingOrigin // Transform for converting tracking space to world space
        );

        if (sceneObjectMesh != null)
        {
            // Set the object's position and rotation based on its pose
            sceneObjectMesh.transform.position = sceneObject.pose.position;
            sceneObjectMesh.transform.rotation = sceneObject.pose.rotation;

            // Add the object to the scene, parent it, or apply additional transformations as needed.
        }
    }
}

In this example, the pose (position and rotation) of each SceneObject is directly applied to the generated mesh, ensuring accurate placement in the scene based on the tracking data.

Sample

Import the ScenePerception feature pack and open the ScenePerceptionDemo scene to see a sample implementation of SceneObject generation and mesh rendering.

SceneObject generation and mesh rendering. In this case, the detected object is in an axis shape, not a cube.

Scene Mesh

Scene Mesh is the mesh obtained by scanning the users’ surroundings. Assuming that you have started using Scene Perception successfully, follow these tips to use Scene Mesh:

1. When perception state for Scene Mesh is WVR_ScenePerceptionState.WVR_ScenePerceptionState_Completed, you can call scenePerceptionManager.GetSceneMeshes() to retrieve the data of the perceived Scene Mesh.
2. To know whether or not two scene meshes are different or not, you can compare the meshBufferId of the scene meshes. Different scene meshes have different meshBufferId.
3. Before trying to get the mesh data of scene meshes, you have to make sure that the permission wave.permission.GET_SCENE_MESH is granted by the user of your application.
4. To get the mesh data of a scene mesh, call scenePerceptionManager.GetSceneMeshBuffer() with meshBufferId as the input parameter.

//Sample Code for requesting permission for Scene Mesh, see SceneMeshPermissionHelper.cs under Demo->Scripts

using Wave.Essence; //Required for using PermissionManager

private static PermissionManager pmInstance = null;
public static bool permissionGranted { get; private set; } = false; //bool for storing permission grant state
private const string scenePerceptionPermissionString = "wave.permission.GET_SCENE_MESH"; //permission name

public static void RequestSceneMeshPermission()
{
  string[] permArray = {
      scenePerceptionPermissionString
    };

  pmInstance = PermissionManager.instance;

  pmInstance?.requestPermissions(permArray, requestDoneCallback);
}

private static void requestDoneCallback(List<PermissionManager.RequestResult> results) //Callback for checking whether permission is granted by the user or not
{
  foreach (PermissionManager.RequestResult permissionRequestResult in results)
  {
    if (permissionRequestResult.PermissionName.Equals(scenePerceptionPermissionString))
    {
      permissionGranted = permissionRequestResult.Granted; //true if permission is granted
    }
  }
}

//Sample Code for getting the scene mesh perception state

public void ScenePerceptionGetState()
{
  WVR_ScenePerceptionState latestPerceptionState = WVR_ScenePerceptionState.WVR_ScenePerceptionState_Empty;
  WVR_Result result = scenePerceptionManager.GetScenePerceptionState(WVR_ScenePerceptionTarget.WVR_ScenePerceptionTarget_SceneMesh, ref latestPerceptionState);
  if (result == WVR_Result.WVR_Success)
  {
    //Check perception state here
    if(latestPerceptionState == WVR_ScenePerceptionState.WVR_ScenePerceptionState_Completed)
    {
      //When perception for scene mesh is completed, you can retrieve the data of the perceived Scene Mesh
      WVR_SceneMeshType currentSceneMeshType = WVR_SceneMeshType.WVR_SceneMeshType_VisualMesh;
      WVR_Result result = scenePerceptionManager.GetSceneMeshes(currentSceneMeshType, out WVR_SceneMesh[] currentSceneMeshes);

      if (result == WVR_Result.WVR_Success)
      {
        //Handle the retrieved data of the Scene Mesh here
        //For example:
        //- Cache Scene Mesh data for future reference
        //- Compare meshBufferId of cached Scene Meshes with the freshly retrieved ones to see if there are any updates
      }
    }
  }
}

//Sample Code for getting scene mesh data of a Scene Mesh

WVR_Vector3f_t[] sceneVertexBuffer; //Array to hold the vertices of the scene mesh
uint[] sceneIndexBuffer; //Array to hold the triangle indices of the scene mesh

WVR_Result result = GetSceneMeshBuffer(sceneMesh.meshBufferId, out sceneVertexBuffer, out sceneIndexBuffer);

if (result == WVR_Result.WVR_Success)
{
  //Handle the retrieved vertex and index data of the scene mesh here
  //Note that vertex coordinates and index order have to be converted from OpenGL convention to Unity convention before use
  //Refer to the "GenerateMesh()" function in the ScenePerceptionManager.MeshGenerationHelper class in ScenePerceptionManager.cs for more details
}

As for the data of Scene Meshes, you can refer to the struct and enum definitions as follows:

//Basic overview of Scene Plane data

public struct WVR_SceneMesh
{
  public UInt64 meshBufferId; //ID of the mesh buffer that holds the mesh data (vertices and indices) of the Scene Mesh
}

Spatial Anchor

Spatial Anchors

Spatial Anchors are anchor points that can be created and stored on the device. The Spatial Anchors can be reused across different sessions of the same app as long as the tracking map which the Spatial Anchors are tied to is active. Assuming that you have started using Scene Perception successfully, follow these tips to use Spatial Anchors:

1. Only enumerate Spatial Anchors when needed (i.e. call GetSpatialAnchors()). We recommend doing this when received an system update event. See Update list when received Event. However you can also do it when a new anchor is created, an existing anchor is destroyed and on app resume (where a tracking map change might have occurred).
2. Use the SpatialAnchorTrackingState retrieved from scenePerceptionManager.GetSpatialAnchorState() to determine whether a Spatial Anchor is still valid or not (SpatialAnchorTrackingState.Tracking means that the Spatial Anchor is still valid and active).
3. You can use the helper function ScenePerceptionManager.AnchorStatePoseEqual() for comparing the pose of Spatial Anchors.
4. You can use the helper function scenePerceptionManager.ApplyTrackingOriginCorrectionToAnchorPose() to convert the pose of a Spatial Anchor to the world space position and rotation that are usable in the Unity coordinate system.

//Retrieve Spatial Anchors

private void UpdateSpatialAnchors()
{
  WVR_Result result;

  if (anchorHandles == null || needAnchorEnumeration) //Enumerate Spatial Anchors only when needed
  {
    result = scenePerceptionManager.GetSpatialAnchors(out anchorHandles);
    needAnchorEnumeration = false;
  }

  if (anchorHandles != null)
  {
    foreach (ulong anchorHandle in anchorHandles)
    {
      var originModel = ScenePerceptionManager.GetCurrentPoseOriginModel();
      //Get the state of a Spatial Anchor using its handle
      result = scenePerceptionManager.GetSpatialAnchorState(
          anchorHandle, originModel, out trackingState, out Pose pose, out string name);
      if (result == WVR_Result.WVR_Success)
      {
        switch(trackingState) //Handle Spatial Anchors with respect to their tracking state
        {
          case SpatialAnchorTrackingState.Tracking: //Spatial Anchor is active and valid
            {
              // Process Spatial Anchor's Pose and name
              break;
            }
          case SpatialAnchorTrackingState.Paused:
          case SpatialAnchorTrackingState.Stopped:
          default:
            break;
        }
      }
    }
  }
}

//Creating a Spatial Anchor at a raycast hit point

private void HandleAnchorUpdateCreate(RaycastHit raycastHit)
{
  WVR_Result result;

  if (raycastHit.collider != null)
  {
    Vector3 anchorWorldPositionUnity = raycastHit.point;

    string anchorNameString = "SpatialAnchorNameTemplate"; //Give the Spatial Anchor a name
    char[] anchorNameArray = anchorNameString.ToCharArray();

    ulong newAnchorHandle = 0;
    result = scenePerceptionManager.CreateSpatialAnchor(anchorNameArray, anchorWorldPositionUnity, rightController.transform.rotation, ScenePerceptionManager.GetCurrentPoseOriginModel(), out newAnchorHandle, true);
    if (result == WVR_Result.WVR_Success)
    {
      needAnchorEnumeration = true;

      UpdateSpatialAnchors(); //Handle Spatial Anchor data
    }
  }
}

//Destroying a Spatial Anchor at a raycast hit point

private void HandleAnchorUpdateDestroy(RaycastHit raycastHit)
{
  WVR_Result result;

  if (raycastHit.collider != null)
  {
    ulong targetAnchorHandle = raycastHit.collider.transform.GetComponent<AnchorPrefab>().anchorHandle;

    result = scenePerceptionManager.DestroySpatialAnchor(targetAnchorHandle);
    if (result == WVR_Result.WVR_Success)
    {
      needAnchorEnumeration = true;

      UpdateSpatialAnchors(); //Handle Spatial Anchor data
    }
  }
}

As for the data of Spatial Anchors, you can refer to the struct and enum definitions as follows:

//Basic overview of Spatial Anchors data

public enum SpatialAnchorTrackingState
{
  Tracking, //Spatial Anchor is active and valid
  Paused,
  Stopped
}

Cached Spatial Anchors

Cached Spatial Anchors are a type of anchor that retains its information even after the application is closed. However, they are dependent on the VR tracking map. If the map is lost or reset, the cached anchor will be cleared. This makes them ideal for short-term persistence across multiple sessions without the need for exporting or importing data.

// Create a Spatial Anchor
ulong anchorHandle;
Vector3 anchorPosition = new Vector3(0, 1, 0);
Quaternion anchorRotation = Quaternion.identity;
WVR_PoseOriginModel originModel = WVR_PoseOriginModel.WVR_PoseOriginModel_OriginOnGround;
scenePerceptionManager.CreateSpatialAnchor("MySpatialAnchor", anchorPosition, anchorRotation, originModel, out anchorHandle);

// Cache the created Spatial Anchor
scenePerceptionManager.CacheSpatialAnchor("MyCachedAnchor", anchorHandle);

// Later, you can retrieve the names of all cached anchors
string[] cachedAnchorNames;
scenePerceptionManager.GetCachedSpatialAnchorNames(out cachedAnchorNames);

// And you can also create a Spatial Anchor from a cached name
ulong newAnchorHandle;
scenePerceptionManager.CreateSpatialAnchorFromCacheName("MyCachedAnchor", "NewSpatialAnchor", out newAnchorHandle);

Uncaching a Spatial Anchor allows you to remove a previously cached anchor from the runtime. This is useful when you no longer need the anchor or want to free up some resources.

// Assuming you have previously cached an anchor with the name "MyCachedAnchor"

// Uncache the specified Cached Spatial Anchor
scenePerceptionManager.UncacheSpatialAnchor("MyCachedAnchor");

// To verify, you can try to retrieve the names of all cached anchors
string[] cachedAnchorNames;
scenePerceptionManager.GetCachedSpatialAnchorNames(out cachedAnchorNames);
// "MyCachedAnchor" should no longer be in the list

// Remove all cached anchors
scenePerceptionManager.ClearCachedSpatialAnchors();

Persisted Spatial Anchors

Persisted Spatial Anchors offer long-term persistence. They are saved with feature points information around the object and environment, ensuring their existence even if the SLAM tracking map is reset. This makes them ideal for scenarios where anchors need to be shared across different devices or sessions.

// Create a Spatial Anchor
ulong anchorHandle;
Vector3 anchorPosition = new Vector3(0, 1, 0);
Quaternion anchorRotation = Quaternion.identity;
WVR_PoseOriginModel originModel = WVR_PoseOriginModel.WVR_PoseOriginModel_OriginOnGround;
scenePerceptionManager.CreateSpatialAnchor("MySpatialAnchor", anchorPosition, anchorRotation, originModel, out anchorHandle);

// Persist the created Spatial Anchor
scenePerceptionManager.PersistSpatialAnchor("MyPersistedAnchor", anchorHandle);

// Later, you can retrieve the names of all persisted anchors
string[] persistedAnchorNames;
scenePerceptionManager.GetPersistedSpatialAnchorNames(out persistedAnchorNames);

// Export the persisted anchor for sharing or backup
byte[] exportedData;
scenePerceptionManager.ExportPersistedSpatialAnchor("MyPersistedAnchor", out exportedData);

// Later, you can import the persisted anchor data.
scenePerceptionManager.ImportPersistedSpatialAnchor(exportedData);

// And you can also create a Spatial Anchor from a persisted name
ulong newAnchorHandle;
scenePerceptionManager.CreateSpatialAnchorFromPersistenceName("MyPersistedAnchor", "NewSpatialAnchorFromPersisted", out newAnchorHandle);

Unpersisting a Spatial Anchor removes a previously persisted anchor from the runtime. This is useful when you want to permanently delete an anchor’s data.

// Assuming you have previously persisted an anchor with the name "MyPersistedAnchor"

// Unpersist the specified Persisted Spatial Anchor
scenePerceptionManager.UnpersistSpatialAnchor("MyPersistedAnchor");

// To verify, you can try to retrieve the names of all persisted anchors
string[] persistedAnchorNames;
scenePerceptionManager.GetPersistedSpatialAnchorNames(out persistedAnchorNames);
// "MyPersistedAnchor" should no longer be in the list

// Remove all Persisted Spatial Anchors
scenePerceptionManager.ClearPersistedSpatialAnchors();

Update list when received Event

When anchors are imported, created or ready to get. System will send you an event. When receive it, you could check if there is any new anchor.

using Wave.Essence.Events;

void OnEnable()
{
  SystemEvent.Listen(WVR_EventType.WVR_EventType_SpatialAnchor_Changed, OnSpatialAnchorEvent, true);
  SystemEvent.Listen(WVR_EventType.WVR_EventType_CachedSpatialAnchor_Changed, OnSpatialAnchorEvent, true);
  SystemEvent.Listen(WVR_EventType.WVR_EventType_PersistedSpatialAnchor_Changed, OnSpatialAnchorEvent, true);
}

void OnDisable()
{
  SystemEvent.Remove(WVR_EventType.WVR_EventType_SpatialAnchor_Changed, OnSpatialAnchorEvent);
  SystemEvent.Remove(WVR_EventType.WVR_EventType_CachedSpatialAnchor_Changed, OnSpatialAnchorEvent);
  SystemEvent.Remove(WVR_EventType.WVR_EventType_PersistedSpatialAnchor_Changed, OnSpatialAnchorEvent);
}

void OnSpatialAnchorEvent(WVR_Event_t wvrEvent)
{
  if (wvrEvent.common.type == WVR_EventType.WVR_EventType_PersistedSpatialAnchor_Changed)
  {
    // Update persisted list
    scenePerceptionManager.GetPersistedSpatialAnchorNames(out string[] names);
  }
  else if (wvrEvent.common.type == WVR_EventType.WVR_EventType_CachedSpatialAnchor_Changed)
  {
    // Update cached list
    scenePerceptionManager.GetCachedSpatialAnchorNames(out string[] names);
  }
  else if (wvrEvent.common.type == WVR_EventType.WVR_EventType_SpatialAnchor_Changed)
  {
    // Update list
    scenePerceptionManager.GetSpatialAnchors(out UInt64[] anchorHandles);
    // Update tracking state...
  }
}

Other Useful Functions

Other than the ones mentioned in this tutorial, there are still other APIs and functions that might be useful to you that are not explained in this documentation, e.g. Overloads of functions that might suit different use cases. You can find the details on the APIs and helper functions in the form of XML API documentation in the ScenePerceptionManager script.

For World Alignment

In Tutorial for MR Contents , we mentioned the World Alignment problem . In here, we provide helper function to calculate it by using unity.

In ScenePerceptionManager script, the AlignWorld function can help you calculate client’s pose in aligned virtual world.

public static AlignWorld(Vector3 p1A2WPosition, Quaternion p1a2WRotation, Vector3 p1A2WScale, Vector3 p2A2Postion, Quaternion p2A2RRotation, out Vector3 p2R2WPosition, out Quaternion p2R2WRotation, out Vector3 p2R2WScale);

public static AlignWorld(Matrix4x4 p1A2W, Matrix4x4 p2A2R, out p2R2W);

Please check the sample in demo folder, ScenePerception/<version>/Demo/Scenes/WorldAlignment.unity.

Resources

The imported resources of the Scene Perception Feature Pack can be found in Assets/Wave/Essence/ScenePerception.

Scene Perception Feature Pack Assets

Points to take note of and Known Issues

1. The passthrough image might not lineup perfectly with the virtual objects such as scene planes. This will be improved in the future through updates.
2. Scene planes have to be created using another app called “Shape Editor” before you can see them in your own.
3. Scene planes and spatial anchors are tied to the active tracking map when they are created. Redoing the room setup process which changes the active tracking map will result in the loss of scene planes and spatial anchors that exist in the previous tracking map(s).