VIVE 3DSP Audio SDK Unity Plugin

Contents


System Requirements

  • Unity version 2017.1 or higher

  • Windows® 7 SP1, Windows® 8 or later, Windows® 10

  • Android 8.1 or later

  • Intel® Core™ i5-4590 or AMD FXTM 8350 equivalent or better

  • NVIDIA® GeForce® GTX 1060 or AMD™ Radeon™ RX 480, equivalent or better graphics card


VIVE 3DSP Audio SDK Unity Plugin Guide

Import Latest Package into New Project

  1. Download the latest version of VIVE 3DSP Audio Plugin in website.

  2. Create a new 3D project in Unity.

  3. Click Assets > Import > Custom Package.

  4. Select the latest version of Vive3DSP .unitypackage file and click Import.


Upgrade to Latest Package

  1. Close Unity project, if open.

  2. In the Unity project, go to the Assets folder and delete the Vive3DSP folder.

  3. Open the porject and import the latest version of VIVE 3DSP Audio Plugin (follow steps 3 and 4 from Import Latest Package into New Project section).


Set Spatializer Pluging

Select VIVE 3DSP Spatializer in Spatializer Plugin.

_images/set_spatializer_plugin_1_2020.png
  1. Click Edit > Project Settings > Audio.

  2. Select VIVE 3DSP Spatializer in Spatializer Plugin.

Note

VIVE 3DSP support 3rd order ambisonics for spatializer.


Set Ambisonic Decoder Plugin

Select VIVE 3DSP Spatializer in Ambisonic Decoder Plugin.

_images/set_ambisonic_decoder_plugin_1_2020.png
  1. Click Edit > Project Settings > Audio.

  2. Select VIVE 3DSP Spatializer in Ambisonic Decoder Plugin.

_images/set_ambisonic_decoder_plugin_2_2020.png

Note

VIVE 3DSP support 1st order ambisonic for decoder.

  1. Put the AmbiX format audio file into your Unity Assets folder.

Note

The ambisonic file should be in ACN/SN3D format.

  1. Find the file in the Project window and click it.

  2. Select Ambisonic in the Inspector window and click Apply.

  3. Add the ambisonic file as an Audio Clip to the Unity Audio Source.

Note

Make sure Spatialize in Audio Source is not selected.


Audio Source Settings

Set an object as the audio source

  1. Do one of the following:

    1. Attach the VIVE 3DSP Audio Source component to a Unity audio source object. Click Add Component > VIVE > 3DSP_AudioSource.

    2. Attach the VIVE 3DSP Audio Source component to an object that contains a Unity Audio Source component.

  2. Add an Audio Clip to the Unity Audio Source.

There are a lot of Unity default features in the Audio Source object. The VIVE 3DSP Audio Source component offers more features for the spatializer effect:

_images/audio_source_settings_2_2020.png
  • Gain (dB): This option sets the audio clip volume level (in dB).

  • Bypass Small Signal: In this feature, VIVE 3DSP will not process the input signal if its level is lower than the defined value (in dB).

Note

The volume of all audible sounds should be higher than the Gain value setting in Bypass Small Signal (i.e. all audible sounds should be processed).

  • Mute Far Distant Source: This feature sets the audible range of the audio source. The audio source would be silent when the distance from the audio source to the listener is larger than the setting value in this item. The Radius slider is adjustable for setting the maximum audible distance.

  • DRC: This option prevents the audio signal from sound clipping.

  • 3D Sound Effect: This option turns on or off the 3D Sound effect, which includes the directional sound feeling and sound decay properties.

_images/audio_source_settings_3_2020.png
  • Room Effect: The option turns on or off the room audio application.

  • Room Reverb Mode: There are two options in this mode: Mono and Binaural. Mono means the same reverberant outputs are generated for both ears. Binaural means the spatial effect of the reflected sound is considered in the sound reverberation. The sound feelings of both ears are different. When the Binaural mode is selected, two binaural engines can be selected:

    1. Speed Up: The numerical simplification method is considered and the computing cost is minimal.

    2. Noraml: It is the theoretically accurate method for calculating the binaural output. Theoretically accuracy requires high computing power.

  • Occlusion Effect: This option turns on or off the sound occlusion application.

  • Occlusion Effect Engine: There are two types of occlusion engines that can be selected.

    1. Frequency Dependent: The different material corresponds to the different sound frequency responses during sound transmission. The sound decay caused by objects is different based on frequencies. The real-world material response is modelled in this application.

    2. Frequency Free: The sound decay influence of an obstacle is simulated by the volume decay only and the computing cost is minimal.

_images/audio_source_settings_4_2020.png
  • Overwrite Volume Rolloff: If this feature is selected, the Unity built-in volume roll off effect will be disabled, and the VIVE 3DSP Sound Decay Effect will be used instead. Several types of Sound Decay Effect can be selected:

    1. Real World Decay: This is the sound decay effect that is based on the real-world measurement database.

    2. Point Source Decay: This is the sound decay effect that is based on inverse-square law. It behaves as the point source where the sound energy decreased ratio is based on the propagation area (the square of the propagation distance).

    3. Line Source Decay: This is the sound decay effect that is based on inverse law. It behaves as the line source where the sound energy decreased ratio is based on the propagation distance.

    4. Linear Decay: It is the customized sound decay effect. The volume decay is linearly proportional to the distance between the source and listener. These parameters can be adjusted.

      • Minimum Distance (M): When the distance is lower than the minimum distance, the volume decay rate is 1.0.

      • Maximum Distance (M): The volume decay rate will be the Minimum Decay Volume (dB) if the distance is longer than the Maximum Distance.

    5. No Decay: No sound decay effect will be applied in this audio source.

Note

  • When Overwrite Volume Roll-off is selected, the default Unity distance curve will be set to a constant, 1.0.

  • Minimum Decay Volume (dB) can be set in this application.

_images/audio_source_settings_5_2020.png
  • Graphic Equalizer (Android standard verison not support) : The graphic equalizer is a convenient tool for tuning audio. In VIVE 3DSP Audio Source, there are 20 bands in the graphic equalizer. The level bar of each band ranges from -12dB to 12dB. The Set to default button sets all level bars to 0dB.

_images/audio_source_settings_6_2020.png
  • Parametric Equalizer (Android standard verison not support) : The option sets the cutoff frequencies and tunes the gains and bandwidths to apply expected sound effects. Low/High Shelving Filters are effective at frequencies lower/higher than the user-defined cutoff frequency. Peaking filters boost/cut the frequency curve based on the user-defined frequencies. These parameters are defined as:

    1. Frequency: This option defines the center frequency for the Peaking Filter and the cutoff frequency for the Low/High Shelving Filter. It ranges from 20 Hz to 20 kHz.

    2. Gain: This option defines the amount in dB which the frequency curve is raised or lowered. It ranges from -12 dB to 12 dB.

    3. Q: Q means the quality factor of the equalizer. This option controls the sharpness of the frequency curve of the Peaking Filter. A larger Q value corresponds to a sharper frequency curve. It ranges from 0.2 to 5.

_images/audio_source_settings_7_2020.png
  • Export Audio File (Android standard verison not support) : This function exports a specific audio source attached with the VIVE 3DSP effects as an audio file. Here, the effects include the Gain, Graphic Equalizer, and Parametric Equalizer.

Note

For this function, the selected audio clip in the Unity Audio Source needs to be in the WAV audio format.


Audio Room Settings

In VIVE 3DSP Audio Room, the reverberations can be represented via specific room sound effects. All audio sources inside a VIVE 3DSP Audio Room object are affected by the room properties such as size, materials, etc.

Specifically, several often-used room presets are provided in VIVE 3DSP Audio Room. Moreover, if User Define is selected, a list of materials and sound properties can be customized for the walls. This component can be selected via Add Component > VIVE > 3DSP_AudioRoom.

_images/audio_room_settings_1_2020.png _images/audio_room_settings_2_2020.png
  • Room Effect: This option can be turned on to apply the room reverberation effect.

  • Room Surface Material: This option shows the material of the walls. It will change the ingredients of the reflected sound. There are pre-defined material presets for each wall. A customized material can be chosen via User Define, and the parameter of the sound reflection ability of the wall can be set via Reflection Rate.

_images/audio_room_settings_3_2020.png
  • Reflection Level: This option sets the strength of the early reflections, it ranges from -30 dB to 10 dB.

  • Reverb Level: This option sets the strength of the late reverberations, it ranges from -30 dB to 10 dB.

  • Background Audio (Android standard verison not support) : The background audio presents the sound effect of the ambient sounds in the environment. There are several default types in the background audio option. The preferred audio file can be chosen in User Define.

_images/audio_room_settings_4_2020.png

Note

The background audio is counted as one audio source in Unity. The limitation of the quantity of audio sources in Unity is adjustable in AudioManager settings. The maximum is 255 (setting in Max Real Voices).

_images/audio_room_settings_5_2020.png
  • Background Volume (Android standard verison not support) : This option sets the strength of the Background Audio, it ranges from -40 dB to 96 dB.

  • Room Size: This option means the dimension of the room in meters. The factor, Scale in “Transform” of the attached object, is considered here.


Audio Occlusion Settings

To have the occlusion effect, it is necessary to set the environment in both the VIVE 3DSP Audio Source and VIVE 3DSP Audio Listener. In audio source objects, the VIVE 3DSP Audio Occlusion component needs to be added to establish the sound occlusion effects. There are two kinds of occlusion components in VIVE 3DSP: Geometric Occlusion and Raycast Occlusion.

Geometric Occlusion

The VIVE 3DSP Geometric Occlusion component should be attached along with the object to enable the occlusion effect. This component can be selected via Add Component > VIVE > 3DSP_AudioOcclusion > Geometric.

_images/audio_geometric_occlusion_settings_1_2020.png
  • Occlusion Effect: This option turns on or off the occlusion effect on the attached object.

  • Occlusion Material: The material of an occluder can be changed here. There are some material presets in VIVE 3DSP. A customized material can be created via User Define by setting the High Freq. Attenuation (dB) and Low Freq. Attenuation Ratio.

_images/audio_geometric_occlusion_settings_2_2020.png
  • Occlusion Intensity: This option presents the strength of the occlusion effect. The higher the value, the more occlusion effect is applied.

  • Occlusion Geometry: The different shapes of the occlusion object can be set here. For calculating the occlusion effect, the geometrical information of the occluder is necessary. The values for Occlusion Center, Size, and Radius need to be entered.

_images/audio_geometric_occlusion_settings_3.png
  • Occlusion Compute Mode: This is designed specifically for box-type geometric occlusion. It offers an option to select different computing modes to balance the coverage resolution and computation load.

    1. Very Light: In this mode, it requires the least amount of calculation for geometric occlusion. There is only one ray emitted from the source to the listener, then evaluates whether the ray collided with the occlusion object or not.

    2. Simple: There are 6 different rays emitted from the source to the listener in this mode. The occlusion ratio is evaluated by the number of rays that collide with the occlusion object.

    3. Normal: There are 11 different rays emitted from the source to the listener in this mode. The occlusion ratio is evaluated by the number of rays that collide with the occlusion object.

    4. Detailed: This is the most accurate mode for occlusion coverage calculation. It also takes subtle movements of the occlusion object into consideration when determining the extent of occlusion.

  • Occlusion Center: This option means the center coordinates relative to the attached object.

  • Occlusion size: This option means the dimension of the occluded region in meters. The factor, Scale in “Transform” of the attached object, is considered here.

Note

The Geometric Occlusion has its own collider calculator. The Unity collider component does not need to be added.

Raycast Occlusion

The VIVE 3DSP Raycast Occlusion component should be attached along with the object to enable the occlusion effect. This component can be selected via Add Component > VIVE > 3DSP_AudioOcclusion > Raycast.

_images/audio_raycast_occlusion_settings_1_2020.png
  • Occlusion Effect: This option turns on or off the occlusion effect on the attached object.

  • Occlusion Material: The material of an occluder can be changed here. There are some material presets in VIVE 3DSP. A customized material can be created via User Define by setting the High Freq. Attenuation (dB) and Low Freq. Attenuation Ratio.

_images/audio_raycast_occlusion_settings_2_2020.png
  • Occlusion Intensity: This option presents the strength of the occlusion effect. The higher the value, the more occlusion effect is applied.

  • Raycast Quantity: This feature represents the number of rays used in the occlusion effect. The higher the quantity, the better the occlusion effect quality. It ranges from 1 to 30.

Note

Raycast Occlusion only works when the Unity collider component is attached and enabled.


Audio Listener Settings

The VIVE 3DSP Audio Listener component should be attached along with the Unity Audio Listener to enable some 3D sound effects. This component can be selected via Add Component > VIVE > 3DSP_AudioListener.

_images/audio_listener_settings_1_2020.png
  • Global Gain: This option sets the overall volume changes of the audio output from VIVE 3DSP Audio Listener. It ranges from -24 dB to 24 dB.

  • Occlusion Mode: There are two types of occlusion modes that can be chosen. If Mono Occlusion is selected, the same occlusion effect would be applied in both ears; otherwise the engine will render the occlusion effect separately.

  • Headset Model (Android standard verison not support) : The option controls the target headset model. For the best audio experience, sound optimization for specific headset models is implemented.

    1. Generic: The option is suitable for general devices and headphones.

    2. VIVE Pro: Audio localization is tuned and optimized for the built-in headphones of the VIVE Pro.

_images/audio_listener_settings_2_2020.png
  • Record Audio Listener (Android standard verison not support) : This option is selected to record what the listener is hearing in the scene and exports it as an audio file. Set path is applied to set the file location. If the Record Audio Listener option is turned off during Play Mode or the Play Mode is stopped, an audio file will be exported in the WAV audio format.


Hi-Res Audio Settings

VIVE 3DSP supports Hi-Res audio file for the audio source. This feature can be set by:

  1. Set System Sample Rate to 96000 (maximum sample rate of Unity) by clicking Edit > Project Settings > Audio.

_images/hi_res_audio_settings_2_2020.png
  1. Modify the audio clip format by changing Compression Format to PCM in the Inspector window.

_images/hi_res_audio_settings_3_2020.png
  1. Modify the operating system’s audio settings. Change the sample rate to 96000 Hz in the system sound advanced settings.

_images/hi_res_audio_settings_4.png _images/hi_res_audio_settings_5.png

Troubleshooting

Memory consumption keeps increasing in Unity Editor

There is an issue in Unity editor in the Unity version 2018 series. If the game object with a Unity Audio Source in a scene has been chosen, and the scene has been shown in the Unity Editor, the memory usage of Unity will sometimes keep increasing. This happens because if the sphere object is shown in the Unity scene, it keeps generating dummy meshes. This issue can be fixed by restarting Unity to delete the dummy meshes. Unity will crash or audio clippings will likely occur if the memory usage is too high.

Unity Built-in Doppler Effect might cause audio distortion

Unity provides Doppler Effect and this can be found in the audio source object. However, the audio stream might be distorted by this built-in effect if the audio source object is moving very fast. For example, a quarter of the speed of sound, the audio stream could occasionally be rugged, therefore audio clipping occurs.

_images/troubleshooting_3_2020.png

Appendix

Material Preset List

Number

Material

1

Concrete

2

PorousConcrete

3

Marble

4

Brick

5

Glass

6

Wood

7

Gypsum

8

Curtain

9

Plywood

10

Steel

11

StoneWall

12

Plaster Absorber_Concrete

13

Plaster Absorber_Brick

14

Plaster Absorber_Wood

15

Plaster Absorber_Gypsum

16

Timberboard_Concrete

17

Timberboard_Brick

18

Timberboard_Wood

19

Timberboard_Gypsum

20

Ceramic Tile_Concrete

21

Ceramic Tile_Brick

22

Ceramic Tile_Wood

23

Ceramic Tile_Gypsum

24

Corkboard_Concrete

25

Corkboard_Brick

26

Corkboard_Wood

27

Corkboard_Gypsum

28

Soft Curtain_Concrete

29

Soft Curtain_Brick

30

Soft Curtain_Wood

31

Soft Curtain_Gypsum

32

Curtain_Concrete

33

Curtain_Brick

34

Curtain_Wood

35

Curtain_Gypsum

36

Cotton_Concrete

37

Cotton_Brick

38

Cotton_Wood

39

Cotton_Gypsum

40

Plaster_Concrete

41

Plaster_Brick

42

Plaster_Wood

43

Plaster_Gypsum

44

Plywood_Concrete

45

Plywood_Brick

46

Plywood_Wood

47

Plywood_Gypsum

48

Glazed Tile_Concrete

49

Glazed Tile_Brick

50

Glazed Tile_Wood

51

Glazed Tile_Gypsum

Note

Android standard version supports 11 materials


VIVE 3DSP Audio SDK Forum Shortcut

If you have suggestions or questions, you can go to the VIVE Audio SDK forum. It can be accessed by clicking Window > VIVE 3DSP Audio SDK > Visit VIVE 3DSP Forum.

_images/appendix_1_2020.png