Spectator Camera

Introduction

The VIVE Wave XR plugin supports the Spectator Camera feature, allowing players to record and display their VR environment with specific recording positions, angles, and other camera attributes to an external monitor, which is very helpful for recording or streaming.

VIVE Wave SDK API allows the developer not only to control the spectator camera in a VR environment freely and highly customizable but also easily set by our out-of-box solution.

Note

The Spectator feature is current in Beta. The publicly released interfaces may be subject to changes in the future.

Contents Prerequisite How to use Resources Points to take note of and Known Issues

Prerequisite

In Unity Editor, go to Project Settings -> Wave XR -> Essence to import the Spectator package.

Spectator pack

Click and toggle Allow Spectator Camera toggles (shown in the following picture) as TRUE.

Enable Allow Spectator Camera toggler

Note

If the developer who using SpectatorCameraManager for deploying this feature requires the feature of capturing 360 photos also in their content app, please enable the toggler Allow Capture 360 Image. The toggler Allow Capture 360 Image will appear after the Allow Spectator Camera toggler is enabled.

How to use

There are two methods to provide developers for using the spectator camera feature. The first method is calling the API from Wave.XRSDK WaveXRSpectatorCameraHandle which has high custom control of the spectator camera. This method benefits developers with a specific purpose or high customizer needed.

The second method is using the out-of-box script named SpectatorCameraManager, which has some necessary logic for the spectator camera feature during the Unity Engine life-cycle and some pre-design methods (such as 360 photo capture) and data that developers can call and use so that developers can focus more on business logic related to the content development and application instead spend time on the functionally logic about spectator camera.

To use Spectator Camera in the developer project, please refer 2 methods shown on the following respectively.

Note

Generally speaking, the above two methods have pros and cons for usage, so which one is suitable for development depends on the goal of the project and the reason why the project uses this feature.

Note

Please open the system’s quick menu (click the VIVE button) and trigger the event of “Start Recording”. The spectator camera recording will start after the event is triggered. To stop the recording, please do the same operation flow as starting the recording.

Click the VIVE button to open the system quick menu

Trigger the “Start Recording” event from the system’s quick menu.

Calling from Wave.XRSDK

Overview

Calling the API from Wave.XRSDK (class WaveXRSpectatorCameraHandle) has high custom control of the spectator camera. This method benefits developers with a specific purpose or high customizer needed. On the other hand, this means the developer need to manually handle some necessary initiation and handler during the Unity Engine life-cycle.

Class Description

This class is a component that represents a spectator camera. When the user triggers the recording event from the system quick menu, this class is responsible for getting the starting or ending notification of the recording event from the Android native layer and writing the spectator camera view in the Unity scene to its RenderTexture, and then returning the current recording frame to Android native layer.

API Description

WaveXRSpectatorCameraHandle

Inherited Class: MonoBehaviour

Public Field

Name	Description
debugStartCamera	bool. Is starting the spectator camera for debugging on Unity Editor?
debugRenderFrame	bool. Whether or not to show the current view of the spectator camera.
debugFPS	Int. How many times an image is captured or rendered every second of the spectator camera.
OnSpectatorStart	The instantiate of the delegate OnSpectatorStartDelegate. This will be called when the spectator camera starts recording.
OnSpectatorStop	The instantiate of the delegate OnSpectatorStopDelegate. This will be called when the spectator camera stops recording.

Public Methods

Name	Description	Parameters	Return
GetCamera	Get the “Camera” component, which is a spectator camera.		UnityEngine.Camera
SetFixedPose	Set the spectator camera pose by specific input position and rotation.	position: The struct UnityEngine.Vector3. The position you want to set to the spectator camera. rotation: The struct UnityEngine.Quaternion. The rotation you want to set to the spectator camera.
ClearFixedPose	Reset the spectator camera pose and set it to the main camera pose.
SetFixedFOV	Set the spectator camera fov by a specific input value.	fov: float. The FOV value you want to set to the spectator camera.
ClearFixedFOV	Reset the spectator camera fov value.
SetCullingMask	Set the spectator camera culling mask by a specific input value.	cullingMask: int. The culling mask you want to set to the spectator camera.
ClearCullingMask	Reset the spectator camera culling mask.
OnCameraPostRender	Update the spectator camera texture and then submit to the render thread.

Public Static Methods

Name	Description	Parameters	Return
GetInstance	Get an instance that represents the class of WaveXRSpectatorCameraHandle.		WaveXRSpectatorCameraHandle
OnSceneLoaded	Create and initialize WaveXRSpectatorCameraHandle instance.	scene: The struct UnityEngine.SceneManagement.Scene. mode: The enum UnityEngine.SceneManagement.LoadSceneMode.

Delegates

Name	Description
OnSpectatorStartDelegate	A delegate declaration that can encapsulate a method that takes no argument and returns void. This declaration should be used on the “OnSpectatorStart” field.
OnSpectatorStopDelegate	A delegate declaration that can encapsulate a method that takes no argument and returns void. This declaration should be used on the “OnSpectatorStop” field.

Calling from SpectatorCameraManager

Overview

Using the manager class (script) SpectatorCameraManager.cs and calling the API from it is a method to make the developer quickly deploy the Spectator Camera Feature to the developer project, thanks to the out-of-box design. Some necessary logic during the Unity Engine life-cycle, such as init and destroy logic, and some features, such as camera pose will follow specific GameObject in the scene, are already pre-defined for the developer so that the developer can focus more on business logic related to the content development and application instead of spending time on the functionally logic about spectator camera.

Class Description

This class is a component that manages a spectator camera. Based on the WaveXRSpectatorCameraHandle, this class adds advanced features for controlling the camera and considers the logic during the Unity Engine life-cycle. Generally speaking, the class is designed for developers to access the spectator camera easily and control it with pre-defined methods or parameters. Hence, the developer just needs to drag the manager class (script) to the GameObject in the scene and refer it to call/modify the methods/parameters in order to interact with the spectator camera as you want.

Using Step

1. Setup your virtual world in your project scene. (Usually, install XR Interaction Toolkit via package manager and then add “XR Rig” (or named “XR Original” in the new version) GameObject to the scene)

The “XR Rig” GameObject represents the user’s eyes, ears, hands, and the center of world space in an XR scene.

2. Create a GameObject (named Spectator Camera Manager) in the scene and drag a script SpectatorCameraManager.cs to it.

Drag SpectatorCameraManager.cs script to scene GameObject

3. Assign a GameObject prefab to the “Spectator Camera Prefab” field in SpectatorCameraManager.cs in Unity inspector. This represents a camera look. We provide a spectator camera prefab for the demo, located in the Demo/Prefabs folder under the Spectator Feature Pack Asset. The feature pack path is Assets/Wave/Essence/Spectator.

Assign a GameObject prefab to the field in SpectatorCameraManager class

4. As a default, the spectator camera source is “HMD,” and its movement and rotation will follow the first enabled Camera component that is tagged “MainCamera.” (In the XR scene, it usually is the Camera component under the XR Rig GameObject)
5. We provide an interface for the developer to modify the spectator camera setting (such as Camera Layer Mask, Vertical FOV, Smooth Camera Movements, etc.) in the Unity inspector. Click the triangle to extend the foldout and modify the camera setting of HMD sources.

Modify the spectator camera setting of HMD sources

6. (Optional) We also provide the debug flow and method for developers who want to debug the spectator camera in the Unity Editor.

   1. To preview the spectator camera view in the playing mode of the Unity Editor, a developer can drag a material named “SpectatorCaptured.mat” (located in the Resources/Spectator folder under the Spectator Feature Pack Asset) to the GameObject in the scene, which contains the Material component.

      

      Drag a SpectatorCaptured.mat to the GameObject in the scene for camera debugging

   2. Enable the toggler of “Start spectator camera for debugging” and “Render spectator camera view for debugging.”

   3. Set the “Debugging Rendering fps” value to the desired value.

      

      Enable the debug flag and set the rendering fps for camera debugging.

Advanced usage (Tracker as the camera source)

The spectator camera can also follow specific GameObject (movement and rotation) in the scene. You can also set up unique camera settings on every specific GameObject that the camera will follow; once the camera source switches to this GameObject, the camera settings saved on this GameObject will apply to the spectator camera immediately.

1. Add a GameObject (named Tracker) to the scene and assign a script SpectatorCameraTracker.cs to it.

Create a tracker in the scene

2. On Spectator Camera Manager, change the “Spectator Camera Source” to “Tracker” and assign the Tracker GameObject to the “Tracker” field.

Change the spectator camera source to the tracker and set the tracker GameObject

3. On the tracker GameObject, developer can set up the camera settings that will apply to the spectator camera when the spectator camera source switches to the tracker.

Set up the camera settings on the tracker GameObject

4. When the “Tracker” field on the Spectator Camera Manager is assigned, a developer can modify the tracker setting directly on the Spectator Camera Manager by extending the Tracker setting foldout.

Set up the camera settings via the Spectator Camera Manager

API Description

SpectatorCameraManager

Inherited Class: MonoBehaviour

Implements interfaces: ISpectatorCameraSetting

Public Static Properties

Name	Description
Instance	[ Read-only ] An instance that represents the class of SpectatorCameraManager.

Public Properties

Name	Description
CameraSourceRef	The enum SpectatorCameraHelper.CameraSourceRef. The source of the spectator camera.
LayerMask	See ISpectatorCameraSetting.LayerMask
IsSmoothCameraMovement	See ISpectatorCameraSetting.IsSmoothCameraMovement
SmoothCameraMovementSpeed	See ISpectatorCameraSetting.SmoothCameraMovementSpeed
IsFrustumShowed	See ISpectatorCameraSetting.IsFrustumShowed
VerticalFov	See ISpectatorCameraSetting.VerticalFov
PanoramaResolution	See ISpectatorCameraSetting.PanoramaResolution
PanoramaOutputFormat	See ISpectatorCameraSetting.PanoramaOutputFormat
PanoramaOutputType	See ISpectatorCameraSetting.PanoramaOutputType
FrustumLineCount	See ISpectatorCameraSetting.FrustumLineCount
FrustumCenterLineCount	See ISpectatorCameraSetting.FrustumCenterLineCount
FrustumLineWidth	See ISpectatorCameraSetting.FrustumLineWidth
FrustumCenterLineWidth	See ISpectatorCameraSetting.FrustumCenterLineWidth
FrustumLineColor	See ISpectatorCameraSetting.FrustumLineColor
FrustumCenterLineColor	See ISpectatorCameraSetting.FrustumCenterLineColor
SpectatorCamera	A GameObject prefab represents a spectator camera look.
SpectatorHandler	An instance of class WaveXRSpectatorCameraHandle. It is gotten and assigned an instance from Wave.XRSDK automatically when the “Start” Unity lifecycle event.
SpectatorCameraTrackerList	[ Read-only ] Represents a read-only collection of class SpectatorCameraTracker. This includes all spectator camera trackers that exist and are active in the current scene.
FollowSpectatorCameraTracker	The tracker candidate that the camera will follow in “Tracker mode.”
DebugStartCamera	[ Available on Unity Editor Only ] bool. Is starting the spectator camera for debugging on Unity Editor?
DebugRenderFrame	[ Available on Unity Editor Only ] bool. Whether or not to show the current view of the spectator camera.
DebugFPS	[ Available on Unity Editor Only ] Int. How many times an image is captured or rendered every second of the spectator camera.

Public Methods

Name	Description	Parameters	Return
ResetSetting	See ISpectatorCameraSetting.ResetSetting
ExportSetting2JsonFile	See ISpectatorCameraSetting.ExportSetting2JsonFile
LoadSettingFromJsonFile	See ISpectatorCameraSetting.LoadSettingFromJsonFile
LoadSettingFromJsonFile	See ISpectatorCameraSetting.LoadSettingFromJsonFile
ApplyData	See ISpectatorCameraSetting.ApplyData
AddSpectatorCameraTracker	Add the spectator camera tracker to the tracker list.	tracker: The class of SpectatorCameraTracker. The tracker that requires to be added to the tracker list.
RemoveSpectatorCameraTracker	Remove the spectator camera tracker from the tracker list.	tracker: The class of SpectatorCameraTracker. The tracker that requires to be removed from the tracker list.
CaptureSpectatorCamera360Photo	Capture the 360 photos at the current spectator camera position and rotation.
IsCameraSourceAsHmd	Check if the current camera source is HMD.		bool. Return true if the current camera source is HMD.
IsCameraSourceAsTracker	Check if the current camera source is Tracker.		bool. Return true if the current camera source is Tracker.
IsFollowTrackerEqualTo	Check if the input tracker is equal to the tracker candidate in the manager.	tracker: The class of SpectatorCameraTracker. The tracker requires to be checked.	bool. Return true if the input tracker is equal to the tracker candidate in the manager.
IsSpectatorCameraHandlerExist	Check if an instance of class WaveXRSpectatorCameraHandle exists in the manager.		bool. Return true if an instance of class WaveXRSpectatorCameraHandle exists in the manager.
IsMainCameraExist	Check if a Camera component that is tagged “MainCamera” exists in the scene.		bool. Return true if a Camera component that is tagged “MainCamera” exists in the scene.
IsFollowTrackerExist	Check if the tracker candidate in the manager exists.		bool. Return true if the tracker candidate in the manager exists.
SetupFrustumLine	Setup the frustum line
SetupFrustumCenterLine	Setup the frustum center line

SpectatorCameraTracker

Inherited Class: MonoBehaviour

Implements interfaces: ISpectatorCameraSetting

Public Properties

Name	Description
CameraSourceRef	[ Read-only ] The enum SpectatorCameraHelper.CameraSourceRef.Tracker.
Position	The world space position of the tracker.
Rotation	The world space rotation of the tracker.
LayerMask	See ISpectatorCameraSetting.LayerMask
IsSmoothCameraMovement	See ISpectatorCameraSetting.IsSmoothCameraMovement
SmoothCameraMovementSpeed	See ISpectatorCameraSetting.SmoothCameraMovementSpeed
IsFrustumShowed	See ISpectatorCameraSetting.IsFrustumShowed
VerticalFov	See ISpectatorCameraSetting.VerticalFov
PanoramaResolution	See ISpectatorCameraSetting.PanoramaResolution
PanoramaOutputFormat	See ISpectatorCameraSetting.PanoramaOutputFormat
PanoramaOutputType	See ISpectatorCameraSetting.PanoramaOutputType
FrustumLineCount	See ISpectatorCameraSetting.FrustumLineCount
FrustumCenterLineCount	See ISpectatorCameraSetting.FrustumCenterLineCount
FrustumLineWidth	See ISpectatorCameraSetting.FrustumLineWidth
FrustumCenterLineWidth	See ISpectatorCameraSetting.FrustumCenterLineWidth
FrustumLineColor	See ISpectatorCameraSetting.FrustumLineColor
FrustumCenterLineColor	See ISpectatorCameraSetting.FrustumCenterLineColor

Public Methods

Name	Description	Parameters	Return
ResetSetting	See ISpectatorCameraSetting.ResetSetting
ExportSetting2JsonFile	See ISpectatorCameraSetting.ExportSetting2JsonFile
LoadSettingFromJsonFile	See ISpectatorCameraSetting.LoadSettingFromJsonFile
LoadSettingFromJsonFile	See ISpectatorCameraSetting.LoadSettingFromJsonFile
ApplyData	See ISpectatorCameraSetting.ApplyData

ISpectatorCameraSetting

Interface to define which setting should be included in the spectator camera. If a component implements this interface, this element will cooperate and be referred to by the spectator camera manager.

Properties

Name	Description
LayerMask	The struct UnityEngine.LayerMask defines which layer the camera can see or not.
IsSmoothCameraMovement	bool. Whether or not to enable the feature of smoothing the spectator camera movement.
SmoothCameraMovementSpeed	int. The speed factor to control the smoothing impact.
IsFrustumShowed	bool. True if visualize the spectator camera vertical FOV.
VerticalFov	float. The spectator camera vertical FOV.
PanoramaResolution	The enum SpectatorCameraHelper.SpectatorCameraPanoramaResolution. The panorama image resolution.
PanoramaOutputFormat	The enum TextureProcessHelper.PictureOutputFormat. The panorama image output format.
PanoramaOutputType	The enum TextureProcessHelper.PanoramaType. The panorama types.
FrustumLineCount	The enum SpectatorCameraHelper.FrustumLineCount. How many frustum lines will be shown?
FrustumCenterLineCount	The enum SpectatorCameraHelper.FrustumCenterLineCount. How many frustum center lines will be shown?
FrustumLineWidth	float. Frustum line width.
FrustumCenterLineWidth	float. Frustum center line width.
FrustumLineColor	The struct UnityEngine.Color. Frustum line color.
FrustumCenterLineColor	The struct UnityEngine.Color. Frustum center line color.

Public Methods

Name	Description	Parameters	Return
ResetSetting	Reset the spectator camera setting to the default value.
ExportSetting2JsonFile	Export the current spectator camera setting as a JSON file and then save it to the resource folder or persistent folder.	attributeFileLocation: The enum SpectatorCameraHelper.AttributeFileLocation.
LoadSettingFromJsonFile	Load the setting (JSON) file via input full file path.	jsonFilePath: string. The setting file’s full path (including file name and JSON extension).
LoadSettingFromJsonFile	Load the setting (JSON) file via input scene name, GameObject name, and the file location (resource folder or persistent folder).	sceneName: string. The scene name. gameObjectName: string. The GameObject name. attributeFileLocation: The enum SpectatorCameraHelper.AttributeFileLocation.
ApplyData	Apply the spectator camera setting to the current component.	data: The class SpectatorCameraHelper.SpectatorCameraAttribute.

Resources

The imported resources of the Spectator Feature Pack can be found in Assets/Wave/Essence/Spectator.

Spectator Feature Pack Assets

Points to take note of and Known Issues

1. Do not assume that the instance of WaveXRSpectatorCameraHandle will be created and available in Awake method. The cautious way to get the instance is by writing the required instance logic by creating the start function as “private IEnumerator Start()” and delaying one to two seconds before calling the required instance logic.
2. We define a material that can preview the spectator camera view in real-time. For usage, the developer can locate the “Resources/Spectator” folder in the Spectator demo folder and drag the material named “SpectatorCaptured.mat” (don’t modify the material file name) to the GameObject, which contains the Material component.