VIVE 3DSP Audio SDK Unity Plugin

Contents

System Requirements

  • Unity version 2017.1 or higher

  • Windows® 7 SP1, Windows® 8 or later, Windows® 10

  • Android 8.1 or later

  • Intel® Core™ i5-4590 or AMD FXTM 8350 equivalent or better

  • NVIDIA® GeForce® GTX 1060 or AMD™ Radeon™ RX 480, equivalent or better graphics card


VIVE 3DSP Audio SDK Unity Plugin Guide

Import Latest Package into New Project

  1. Download the latest version of VIVE 3DSP Audio Plugin in website.

  2. Create a new 3D project in Unity.

  3. Click Assets > Import > Custom Package.

  4. Select the latest version of Vive3DSP .unitypackage file and click Import.


Upgrade to Latest Package

  1. Close Unity project, if open.

  2. In the Unity project, go to the Assets folder and delete the Vive3DSP folder.

  3. Open the porject and import the latest version of VIVE 3DSP Audio Plugin (follow steps 3 and 4 from Import Latest Package into New Project section).


Set Spatializer Pluging

Select VIVE 3DSP Audio in Spatializer Plugin.

_images/set_spatializer_plugin_mark.png

  1. Click Edit > Project Settings > Audio.

  2. Select VIVE 3DSP Audio in Spatializer Plugin.

Note

VIVE 3DSP support 3rd order ambisonics for spatializer.


Set Ambisonic Decoder Plugin

Select VIVE 3DSP Audio in Ambisonic Decoder Plugin.

_images/set_ambisonic_decoder_plugin_1_mark.png

  1. Click Edit > Project Settings > Audio.

  2. Select VIVE 3DSP Audio in Ambisonic Decoder Plugin.

Note

VIVE 3DSP support 1st order ambisonic for decoder.

_images/set_ambisonic_decoder_plugin_2_mark.png

  1. Put the AmbiX format audio file into your Unity Assets folder.

Note

The ambisonic file should be in ACN/SN3D format.

  1. Find the file in the Project window and click it.

  2. Select Ambisonic in the Inspector window and click Apply.

  3. Add the ambisonic file as an Audio Clip to the Unity Audio Source.

Note

Make sure Spatialize in Audio Source is not selected.


Audio Mixer Settings

Add an audio mixer

  1. Click Window > Audio > Audio Mixer.

  2. Click + to add a new audio mixer.

  3. In Audio Mixer tab, click Add…, and then select VIVE 3DSP Renderer.

_images/audio_mixer_1_mark.png _images/audio_mixer_2.png

Audio Source Settings

Set an object as the audio source

  1. Do any of the following:

    1. Attach the VIVE 3DSP Audio Source component to a Unity audio source object. Click Add Component > VIVE > 3DSP_AudioSource.

    2. Attach the VIVE 3DSP Audio Source component to an object that contains a Unity Audio Source component.

  2. Add an Audio Clip to the Unity Audio Source.

  3. Set the audio mixer that include VIVE 3DSP Renderer as the Unity Audio Source output.

_images/audio_source_settings_1.png

Note

Don’t select Bypass Effects and Bypass Listener Effects in the Unity Audio Source.

There are a lot of Unity default features in the Audio Source object. The VIVE 3DSP Audio Source component offers more features for the spatializer effect:

_images/audio_source_settings_2.png

  • Gain (dB): This option sets the audio clip volume level (in dB).

  • 3D Sound Effect: This option turns on or off the 3D Sound effect, which includes the directional sound feeling and sound decay properties.

  • Room Effect: The option turns on or off the room audio application.

  • Occlusion Effect: This option turns on or off the sound occlusion application.

  • Occlusion Effect Engine: There are two types of occlusion engines that can be selected.

    1. Frequency Dependent: The different material corresponds to the different sound frequency responses during sound transmission. The sound decay caused by objects is different based on frequencies. The real-world material response is modelled in this application.

    2. Frequency Free: The sound decay influence of an obstacle is simulated by the volume decay only and the computing cost is minimal.

  • Nearfield Effect: This option turns on or off the nearfield effect-which performs the sound response-when the sound source is near to listener. The nearfield effect is only active when the distance between source and listener is less than one meter.

    1. Nearfield Effect Intensity: This option represents the strength of the nearfield effect. The higher the value, the stronger the nearfield effect will be.

    Note

    DRC is enabled by default to help prevent audio signal clipping.

  • Source Directivity: This option turns on or off the source directivity effect which simulates the sound directional and response when pattern changed.

    1. Shape: This option presents the shape of source directivity pattern.

    2. Focus: This option presents the sharpness of source directivity pattern.

  • Overwrite Volume Rolloff: If this feature is selected, the Unity built-in volume roll off effect will be disabled, and the VIVE 3DSP Sound Decay Effect will be used instead. Several types of Sound Decay Effect can be selected:

    1. Point Source Decay: This is the sound decay effect that is based on inverse-square law. It behaves as the point source where the sound energy decreased ratio is based on the propagation area (the square of the propagation distance).

    2. Line Source Decay: This is the sound decay effect that is based on inverse law. It behaves as the line source where the sound energy decreased ratio is based on the propagation distance.

    3. Linear Decay: It is the customized sound decay effect. The volume decay is linearly proportional to the distance between the source and listener. These parameters can be adjusted.

      • Minimum Distance (M): When the distance is lower than the minimum distance, the volume decay rate is 1.0.

      • Maximum Distance (M): The volume decay rate will be the Minimum Decay Volume (dB) if the distance is longer than the Maximum Distance.

    4. No Decay: No sound decay effect will be applied in this audio source.

Note

  • When Overwrite Volume Roll-off is selected, the default Unity distance curve will be set to a constant, 1.0.

  • Minimum Decay Volume (dB) can be set in this application.


Audio Room Settings

In VIVE 3DSP Audio Room, the reverberations can be represented via specific room sound effects. All audio sources inside a VIVE 3DSP Audio Room object are affected by the room properties such as size, materials, etc.

Specifically, several often-used room presets are provided in VIVE 3DSP Audio Room. Moreover, if User Define is selected, a list of materials and sound properties can be customized for the walls. This component can be selected via Add Component > VIVE > 3DSP_AudioRoom.

_images/audio_room_settings_1.png _images/audio_room_settings_2.png

  • Room Effect: This option can be turned on to apply the room reverberation effect.

  • Room Surface Material: This option shows the material of the walls. It will change the ingredients of the reflected sound. There are pre-defined material presets for each wall. A customized material can be chosen via User Define, and the parameter of the sound reflection ability of the wall can be set via Reflection Rate.

_images/audio_room_settings_3.png

  • Reflection Level: This option sets the strength of the early reflections, it ranges from -30 dB to 10 dB.

  • Reverb Level: This option sets the strength of the late reverberations, it ranges from -30 dB to 10 dB.

  • Background Audio: The background audio presents the sound effect of the ambient sounds in the environment. There are several default types in the background audio option. The preferred audio file can be chosen in User Define.

_images/audio_room_settings_4.png

Note

The background audio is counted as one audio source in Unity. The limitation of the quantity of audio sources in Unity is adjustable in AudioManager settings. The maximum is 255 (setting in Max Real Voices).

_images/audio_room_settings_5_mark.png

  • Background Volume: This option sets the strength of the Background Audio, it ranges from -40 dB to 96 dB.

  • Room Size: This option means the dimension of the room in meters. The factor, Scale in “Transform” of the attached object, is considered here.


Audio Occlusion Settings

To have the occlusion effect, it is necessary to set the environment in both the VIVE 3DSP Audio Source and VIVE 3DSP Audio Listener. In audio source objects, the VIVE 3DSP Audio Occlusion component needs to be added to establish the sound occlusion effects.

The VIVE 3DSP Occlusion component should be attached along with the object to enable the occlusion effect. This component can be selected via Add Component > VIVE > 3DSP_AudioOcclusion.

_images/audio_occlusion_settings_1.png

  • Occlusion Effect: This option turns on or off the occlusion effect on the attached object.

  • Occlusion Material: The material of an occluder can be changed here. There are some material presets in VIVE 3DSP. A customized material can be created via User Define by setting the High Freq. Attenuation (dB) and Low Freq. Attenuation Ratio.

_images/audio_occlusion_settings_2.png

  • Occlusion Intensity: This option presents the strength of the occlusion effect. The higher the value, the more occlusion effect is applied.

  • Occlusion Angle: This option presents the cover range of the occlusion effect. The higher the value, the more occlusion cover range is applied.

  • Occlusion Geometry: The different shapes of the occlusion object can be set here. For calculating the occlusion effect, the geometrical information of the occluder is necessary. The values for Occlusion Center, Size, and Radius need to be entered.

  • Occlusion Compute Mode: This is designed specifically for box-type geometric occlusion. It offers an option to select different computing modes to balance the coverage resolution and computation load.

    _images/audio_occlusion_settings_3.png

    1. Very Light: In this mode, it requires the least amount of calculation for geometric occlusion. There is only one ray emitted from the source to the listener, then evaluates whether the ray collided with the occlusion object or not.

    2. Simple: There are 6 different rays emitted from the source to the listener in this mode. The occlusion ratio is evaluated by the number of rays that collide with the occlusion object.

    3. Normal: There are 11 different rays emitted from the source to the listener in this mode. The occlusion ratio is evaluated by the number of rays that collide with the occlusion object.

    4. Detailed: This is the most accurate mode for occlusion coverage calculation. It also takes subtle movements of the occlusion object into consideration when determining the extent of occlusion.

  • Occlusion Center: This option means the center coordinates relative to the attached object.

  • Occlusion size: This option means the dimension of the occluded region in meters. The factor, Scale in “Transform” of the attached object, is considered here.

Note

The Occlusion component has its own collider calculator. The Unity collider component does not need to be added.


Audio Listener Settings

The VIVE 3DSP Audio Listener component should be attached along with the Unity Audio Listener to enable some 3D sound effects. This component can be selected via Add Component > VIVE > 3DSP_AudioListener.

_images/audio_listener_settings_1.png

  • Global Gain: This option sets the overall volume changes of the audio output from VIVE 3DSP Audio Listener. It ranges from -24 dB to 24 dB.

  • Spatial Model: The option controls the spatializer model.

    1. Real: The option performs the sound response that is measured at the different direction from real world.

    2. Lossless: The option performs the sound response that represents the characters of spatializer without any distortion.

  • Headset Config: The option controls the targeted headset model. For the best audio experience, sound optimization has been done for most headset models.

    1. Generic: Select this option if using non-VIVE VR devices and headphones.

    2. Auto Detection: Audio response is optimized for VIVE VR devices. When enabled, the system will automatically detect the type of VIVE device and provide the optimized audio experience for that device. The feature supports the following VIVE devices: VIVE Pro, VIVE Pro 2, VIVE Focus 3, and VIVE XR Elite.

  • Record Audio Listener: This option is selected to record what the listener is hearing in the scene and exports it as an audio file. Set path is applied to set the file location. If the Record Audio Listener option is turned off during Play Mode or the Play Mode is stopped, an audio file will be exported in the WAV audio format.

  • Add Watermark: This option lets you add a watermark in the audio system of the application for intellectual property rights and for identification purposes. You can find the watermark decoder in the VIVE 3DSP SDK package. For more details, refer to Decoding watermarks.

    a. Watermark: The watermark must be eight characters or less, and can only contain numbers (0–9) and UPPERCASE letters (A–Z). (Any lowercase letter will be automatically converted to uppercase letter.)


Decoding watermarks

The VIVE 3DSP Watermark Decode Tool (appWmDecToolAPP.exe) lets you check if there is a watermark in the audio file of the application. You’ll need to prepare the audio file of the application first to be able to decode the watermark.

_images/decode_tool_1.png

Decode the watermark

  1. In Your watermark, enter the watermark that you used for the audio of the application.

  2. In Your audio path, select the audio file of application.

  3. Click Start decode. The status of the decoding process will then be shown on the window. The decoding process may take several minutes.

_images/decode_tool_2.png _images/decode_tool_3.png

If the watermark entered is found in the audio file, it will be displayed on the window.


Hi-Res Audio Settings

VIVE 3DSP supports Hi-Res audio file for the audio source. This feature can be set by:

  1. Set System Sample Rate to 96000 (maximum sample rate of Unity) by clicking Edit > Project Settings > Audio.

_images/hi_res_audio_settings_1_mark.png

  1. Modify the audio clip format by changing Compression Format to PCM in the Inspector window.

_images/hi_res_audio_settings_2_mark.png

  1. Modify the operating system’s audio settings. Change the sample rate to 96000 Hz in the system sound advanced settings.

_images/hi_res_audio_settings_3_mark.png _images/hi_res_audio_settings_4_mark.png

Troubleshooting

Memory consumption keeps increasing in Unity Editor

There is an issue in Unity editor in the Unity version 2018 series. If the game object with a Unity Audio Source in a scene has been chosen, and the scene has been shown in the Unity Editor, the memory usage of Unity will sometimes keep increasing. This happens because if the sphere object is shown in the Unity scene, it keeps generating dummy meshes. This issue can be fixed by restarting Unity to delete the dummy meshes. Unity will crash or audio clippings will likely occur if the memory usage is too high.

Unity Built-in Doppler Effect might cause audio distortion

Unity provides Doppler Effect and this can be found in the audio source object. However, the audio stream might be distorted by this built-in effect if the audio source object is moving very fast. For example, a quarter of the speed of sound, the audio stream could occasionally be rugged, therefore audio clipping occurs.

_images/troubleshooting_1_mark.png

Appendix

Material Preset List

Number

Material

1

Concrete

2

PorousConcrete

3

Marble

4

Brick

5

Glass

6

Wood

7

Gypsum

8

Curtain

9

Plywood

10

Steel

11

StoneWall

12

Plaster Absorber_Concrete

13

Plaster Absorber_Brick

14

Plaster Absorber_Wood

15

Plaster Absorber_Gypsum

16

Timberboard_Concrete

17

Timberboard_Brick

18

Timberboard_Wood

19

Timberboard_Gypsum

20

Ceramic Tile_Concrete

21

Ceramic Tile_Brick

22

Ceramic Tile_Wood

23

Ceramic Tile_Gypsum

24

Corkboard_Concrete

25

Corkboard_Brick

26

Corkboard_Wood

27

Corkboard_Gypsum

28

Soft Curtain_Concrete

29

Soft Curtain_Brick

30

Soft Curtain_Wood

31

Soft Curtain_Gypsum

32

Curtain_Concrete

33

Curtain_Brick

34

Curtain_Wood

35

Curtain_Gypsum

36

Cotton_Concrete

37

Cotton_Brick

38

Cotton_Wood

39

Cotton_Gypsum

40

Plaster_Concrete

41

Plaster_Brick

42

Plaster_Wood

43

Plaster_Gypsum

44

Plywood_Concrete

45

Plywood_Brick

46

Plywood_Wood

47

Plywood_Gypsum

48

Glazed Tile_Concrete

49

Glazed Tile_Brick

50

Glazed Tile_Wood

51

Glazed Tile_Gypsum

VIVE 3DSP Audio SDK Forum Shortcut

If you have suggestions or questions, you can go to the VIVE Audio SDK forum. It can be accessed by clicking Window > VIVE 3DSP Audio SDK > Visit VIVE 3DSP Forum.

_images/appendix_1_mark.png