converting multipass shadertoys to synesthesia

2024-03-23T03:26:15

spurred by this fb post:

Hey y’all. I am desperately looking for tutorials on coverting multipass shaders from Shadertoy into Synesthesia. I’ve got the basics of manually importing shaders into Synesthesia. I am absolutely stumped on how to properly configure shaders that use multiple buffers / passes to work in the software. Is there anyone out there that may be able to help walk me through how to import shaders such as the below? I know it involves setting up the JSON file for multiple passes, as well as adding some code into the main.glsl to call those passes. If I could figure out this process, I could simply repeat for other shaders I’m looking to import – but there’s a knowledge gap here for me that seems it would require many months of deep diving into GLSL to suss out.

using DotCamera shadertoy: https://www.shadertoy.com/view/dlKSDV

note: this was also posted as a comment reply on Facebook, I’m just copying it here for posterity

First it helps to understand a big fundamental difference between how Shadertoy and Synesthesia handle multipass shaders:
Shadertoy uses iChannels and Buffers. Each buffer is a separate piece of code that has its own iChannels assigned, which in turn contain some texture source. Synesthesia puts all buffers into one code file, using PASSINDEX to determine which block of code to run. This means in Shadertoy, each buffer can define its own version of the same function or variable, and each buffer has its own ‘iChannel0’. In Synesthesia all the variables and functions are available in all passes, and each pass has access to the same textures.

So putting it all together, the process of converting DotCamera looks like:

0. Enable Built-In Scene Editor in settings
   Set at least one custom scene folder
   Set ‘primary’ on the folder you’d like to create/import new scenes
1. Load DotCamera using the right sidebar import tab
   It will fail, click ‘create synScene in library’ anyway
2. Find the scene in your library and hit edit to load it into the scene editor
   It will still be broken but the format has been converted to fit Synesthesia.
   This should usually handle all of the iChannels, buffers, and passes for us, but sometimes there are still issues.
3. Fix the bugs
   Checking the Synesthesia console you’ll see an error message
   In this case the problem is on line 44, incompatible return type. This is because Synesthesia expects you to return a vec4 while Shadertoy expects you to set a certain variable (usually fragColor, but in this shadertoy it’s changed to O) to a vec4 and then return void.
   just change return; to return O;

And that’s all it took, it compiles! This actually turned out to be an easier one. There’s still a couple small issues that are easy to fix. Swapping syn_Media in for syn_UserImage on line 20 will correctly flip the image. Finally go to the Media Settings in the scene config sidebar and set filter mode to ‘mipmap’ and wrap mode to ‘clamp’ to match the settings used in Buffer A iChannel0 and it should be pretty much working! Let me know if you get it working and share a video if you make something cool out of it.