Raytraced Audio In The Browser (Patreon)

After much experimentation I've managed to get Vercidium Audio working in the browser with multi-threading and a new JavaScript API.

JavaScript API

The JavaScript API is very similar to the C# API - create a context, add emitters and primitives, and update every frame. Everything can be edited in real time, and all the heavy logic runs in background threads in your browser.

The JavaScript API doesn't support a debug rendering view at the moment, but in the future it'll be able to render the scene to a canvas using WebGL.

How It Works

.NET 8 and onwards can compile to WebAssembly (WASM), a low-level instruction format that runs in browsers. WebAssembly is widely supported, having been available in all major browsers for the past seven years.

Since the Vercidium Audio C# SDK doesn't have any dependencies, it can compile to Web Assembly without much major work. The main issues I ran into were:

.NET WASM code is very slow if it's interpreted. For best performance you need to use AOT (ahead of time) compiling

AOT compiling doesn't work when running dotnet run or dotnet build - you need to run dotnet publish instead - else you'll face all sorts of strange issues in the browser

More details in the WASM setup are at the end of this post

Performance

For profiling, I set up a scene with:

1 listener emitter

1 source emitter

1024 occlusion rays with 8 bounces

128 permeation rays with 3 bounces

128 reverb rays with 64 bounces

8 threads

The raytracing times on my i7-10700F CPU are:

Desktop C#: 2.6ms

Desktop C# (after PGO): 0.7ms

Firefox: 3.15ms

Opera: 7.5ms

Chrome: 7.5ms

Edge: 3.3ms

Performance between Firefox and Desktop C# is comparable, but unfortunately the WASM library doesn't benefit from runtime PGO optimisations because it's AOT compiled.

Also note this profiling was performed with the ray cache disabled - it won't take 3.15ms in Firefox every frame. Instead it'll only re-cast the rays that are affected by moving primitives / emitters. If nothing changes, no rays are cast.

Detailed WASM Setup

To compile a .NET project to WASM, you need to add a few fields to your .csproj file:

To allow JS code to invoke functions and access data in the compiled WASM binary, we need to add some entry points. The [JSExport] attribute tells the C# compiler that we need to be able to invoke this function from JavaScript. The compiler then generates interop code for each function, which passes basic data types (not objects) between JS and C#.

When running dotnet publish, it'll produce a dotnet.js file that can be used in JavaScript to load and invoke the exported functions. When zipped it's all about 2.5 MB:

Example usage is below, with:

VAudioExports is the class name above

PrismPrimitive_Create is the name of one of the functions above

The threading config variables are explained below

Threading

Things get a bit more complicated if we want to run multi-threaded code in the browser. Adding <WasmEnableThreads>true</WasmEnableThreads> to your .csproj moves your C# code off the browser's UI thread, which means every [JSExport] call from JavaScript becomes async and must be awaited.

You can opt out of this by setting jsThreadBlockingMode: "DangerousAllowBlockingWait" in the dotnet config. But there's a catch - if Vercidium Audio spawns more threads than there are Web Workers available, the browser will freeze. To ensure this doesn't happen, adjust these two settings:

pthreadPoolInitialSize: the number of Web Workers spawned at startup, ready to be used by C# threads immediately

pthreadPoolUnusedSize: the number of Web Workers kept in reserve. As you consume them, the runtime tops the pool back up

However! You can't spawn 16 threads, dispose them, then immediately spawn another 16. The Web Workers don't return to the pool instantly — the runtime needs time to reclaim them. If you allocate more threads than are available, you'll lock up the browser.

For example my CPU has 16 threads, so the Vercidium Audio SDK will default the number of threads to 16 - 1 = 15 by default. I experimented with the numbers and found that the browser was sluggish unless I set pthreadPoolInitialSize to 24 or higher. I think this is because the .NET runtime uses its own threads for garbage collection / profiling / etc.

If a user with 64 threads then visits my website, the SDK will try to create 63 threads and their browser will freeze. To prevent this from happening, limit the maximum concurrency level for your raytracing context:

Next Steps

The new website now features an interactive sandbox at the top thanks to WASM! It's not live yet, but we're incredibly close to launch.

Thanks for reading - I can't wait for you to get your hands on Vercidium Audio.