<h3 class="wp-block-heading"><a href="https://github.com/kylegrover/comfyui-python-cowboy">Python Cowboy</a> &#8211; Pretty .py for ComfyUI</h3>



Python Cowboy is my custom node for ComfyUI to allow executing custom Python code with a pretty wrapper. There are existing arbitrary python execution nodes for Comfy including <a href="https://github.com/nidefawl/ComfyUI-nidefawl">one by nidefawl</a> that I used until I slowly morphed it into this node. My node adds integration with CodeMirror to create a nicer Python editing environment that includes:



<ul class="wp-block-list">
<li>code highlighting</li>



<li>error reporting</li>



<li>print() display</li>



<li>common key commands</li>
</ul>



Inspiration borrowed from <a href="https://github.com/patriciogonzalezvivo/comfyui_glslnodes">https://github.com/patriciogonzalezvivo/comfyui_glslnodes</a>



<h4 class="wp-block-heading">⚠ STATUS: Incomplete ⚠</h4>



There are a few outstanding bugs that need to be resolved before Python Cowboy is ready for use.



<ul class="wp-block-list">
<li><s>implement python node</s></li>



<li><s>implement codemirror</s></li>



<li>synchronize codemirror and comfyui input state</li>



<li>correctly pass executed values</li>
</ul>



<a href="https://github.com/kylegrover/comfyui-python-cowboy">https://github.com/kylegrover/comfyui-python-cowboy</a>

🐍🤠

This is a loose collection of thoughts and notes I&#8217;ve gathered on generative music. I started making generative music years ago with Sonic Pi ♥ and later moved to using ToneJS in a web context so I could synchronize it to WebGL visuals. Lately the game has changed dramatically with AI which has introduced a ton of new tools. Seems to me there&#8217;s a huge unexplored space where these two things overlap.



<h2 class="wp-block-heading">Generative Code Music</h2>



<h4 class="wp-block-heading">Sonic Pi <a href="https://sonic-pi.net">https://sonic-pi.net</a></h4>



Sonic Pi rocks, I used this early on when I understood little about what I was doing and it made a lot of stuff fun and easy. It&#8217;s written in and based around Ruby, which is a language I have basically only used in this context, but it really can be a surprisingly good place to start for the code-curious.



I ultimately found the language a bit foreign and the context limiting for my skillset (possibly for a lot of the same reasons that make it great to start with).



This is my collection of sonic pi scripts, they require certain folders and must be run in a certain order and none of that is documented, but maybe someone will find helpful ideas within: <a href="https://github.com/kylegrover/sonicpi-algobop">https://github.com/kylegrover/sonicpi-algobop</a>



Using Sonic Pi:



Sonic Pi is a standalone environment, that&#8217;s part of what makes it so easy to start with. Simply download it and try it out using the built in project files. They also have a great tutorial: <a href="https://sonic-pi.net/tutorial.html#section-1">https://sonic-pi.net/tutorial.html#section-1</a>



<h4 class="wp-block-heading">ToneJS <a href="https://tonejs.github.io/">https://tonejs.github.io/</a></h4>



I switched to <a href="https://tonejs.github.io/">ToneJS</a> after Sonic Pi because it runs in the web context I&#8217;m familiar with and allowed me to tie it into other web actions. I still use ToneJS for example on my homepage <a href="https://ufffd.com/">https://ufffd.com/</a> to run the mouseover sounds based on interaction events from the 3D system. 



Here is a framework I built in ToneJS to generate songs for a few generative art projects I made: <a href="https://github.com/kylegrover/js-algobop">https://github.com/kylegrover/js-algobop</a>



You can see it in practice here: <a href="https://www.fxhash.xyz/generative/slug/bop">https://www.fxhash.xyz/generative/slug/bop</a>



Using ToneJS:



A lot has been written about ToneJS, it&#8217;s a popular library. Try starting here:



<ul class="wp-block-list">
<li><a href="https://github.com/Tonejs/Tone.js/wiki/Installation">https://github.com/Tonejs/Tone.js/wiki/Installation</a></li>



<li><a href="https://medium.com/dev-red/tutorial-lets-make-music-with-javascript-and-tone-js-f6ac39d95b8c">https://medium.com/dev-red/tutorial-lets-make-music-with-javascript-and-tone-js-f6ac39d95b8c</a></li>
</ul>



<h2 class="wp-block-heading">AI Generative Music</h2>



<h4 class="wp-block-heading">RAVE</h4>



<a href="https://github.com/acids-ircam/RAVE">RAVE</a>, a realtime generative model made by Antoine Caillon and Philippe Esling at IRCAM late 2021, is to this day my favorite model I&#8217;ve tried. I haven&#8217;t found anything more modern (other than RAVE v2) that is as fun to use or allows the same kind of control. Rather than prompting with text you manipulate some underlying latents and/or do realtime style transfer based on a tuned model to create bizarre musical sounds with a very human touch. I love this so much more than typing in &#8220;cool hihat loop 140bpm&#8221; waiting a few seconds and then receiving back maybe a hihat loop that might be cool and might be at 140bpm (or maybe something entirely unrelated) and then having no control over it beyond the normal sound manipulation means I can use on any other (higher quality!) sample.



Using RAVE:



Demo: <a href="https://caillonantoine.github.io/ravejs/">https://caillonantoine.github.io/ravejs/</a>



Try out a few pretrained models in a UI format. This shows you what it&#8217;s sonically capable of but it doesn&#8217;t demonstrate the awesome realtime possibilities&#8230;



Realtime usage: <a href="https://github.com/acids-ircam/RAVE#realtime-usage">https://github.com/acids-ircam/RAVE#realtime-usage</a> 



This is definitely harder and less documented than Sonic Pi or Tone Js, but it&#8217;s very doable on any hardware and moderate computer knowledge. Get some pretrained models to use here: <a href="https://acids-ircam.github.io/rave_models_download">https://acids-ircam.github.io/rave_models_download</a>



Training:



This is where it gets a little harder and requires either a beefy GPU or some money for cloud credits and a little extra patience. I trained a few rounds of RAVEv1 in the cloud successfully but it was a headache in many ways compared to simply having a decent modern GPU to train on.



[ I will update this section when I train a new RAVE model ]



someone elses notes on training v2 <a href="https://github.com/acids-ircam/RAVE/issues/312">https://github.com/acids-ircam/RAVE/issues/312</a>



colab for training v2 <a href="https://colab.research.google.com/drive/1ih-gv1iHEZNuGhHPvCHrleLNXvooQMvI?usp=sharing">https://colab.research.google.com/drive/1ih-gv1iHEZNuGhHPvCHrleLNXvooQMvI?usp=sharing</a>



I Wish It Had:



Nicer UI, for training and for inference. nn~ is cool but it would be great to bundle into a custom tool or VST. (I do think some VST work exists)



Better place to share trained models.



More checkpoints to start from.



Clip/clap/prompt training.



<h4 class="wp-block-heading">Stable Audio Tools &amp; Stable Audio Open 1.0</h4>



<a href="https://github.com/Stability-AI/stable-audio-tools">Stable Audio Tools</a> is a set of scripts and a UI for training and inferencing <a href="https://huggingface.co/stabilityai/stable-audio-open-1.0">Stable Audio Open 1.0 </a>



I&#8217;ve experimented with it only a bit. I think it&#8217;s the best that we&#8217;ve got locally but it feels a little restrained. My general feeling on AI audio is it&#8217;s held back in comparison to images and video because the music industry has a much more codified system of enforcing IP in the form of sampling and referencing. 



Using Stable Audio Tools:



<a href="https://huggingface.co/stabilityai/stable-audio-open-1.0/discussions/29">https://huggingface.co/stabilityai/stable-audio-open-1.0/discussions/29</a> quality setup info from mildly deranged internet bro Assbang 👍



I Wish It Had:



<ul class="wp-block-list">
<li>keyframing, timeline, prompt traveling</li>



<li>autolooping mode with variation</li>



<li>upres</li>



<li>demix</li>



<li>basically I want comfyui but for audio i think
<ul class="wp-block-list">
<li>there is a comfyui node&#8230;</li>
</ul>
</li>



<li>higher quality tuning (they cant release v2 because of IP shit)</li>
</ul>



<h4 class="wp-block-heading">Google Music FX</h4>



<a href="https://aitestkitchen.withgoogle.com/tools/music-fx">https://aitestkitchen.withgoogle.com/tools/music-fx</a>



meh. many great brains combined with corporate restriction and misguided financial interests to produce a tool that I find uninspiring and unhelpful for musicians. The best case it can serve is cutting into the market of royalty free music artists like Epidemic, Free Sounds, and Kevin MacLeod



<h4 class="wp-block-heading">Popular Tools I haven&#8217;t Used</h4>



Suno, Udio, elevenlabs, hugely popular tools that I haven&#8217;t really tried. I tend to not bother with paid closed services unless its on a pay per use api at a low cost. I do not want to subscribe, I do not want to purchase credits.



<a href="https://github.com/facebookresearch/audiocraft">https://github.com/facebookresearch/audiocraft</a> is also supposedly decent, I just haven&#8217;t tried it yet

AI &amp; generative music tools

mochi 1 video ai notes

i loved deforums steerable video capabilities but they were always glitchy, so when i found animatediff i was very pleased with the quality but very much missing the control. i had built out some tools to steer deforum using synesthesia&#8217;s audio engine even and wanted to make use of those with my beefy new gpu. i havent found anything newer that does this so my best idea is to fuse them.



so far my best attempts have come from running a deforum video and then using animatediff in a vid2vid process. i havent had any luck with hyper/lightning model quality so right now its running pretty slow, taking anywhere from 15 minutes to 2.5 hours to run just the vid2vid step for a 120 frame clip.



basic process:



<ul class="wp-block-list">
<li>process audio data from a song using synesthesia and other tools</li>



<li>use audio data to construct keyframes for comfyui</li>



<li>come up with a list of prompts to use at different calculated times in the song</li>



<li>render deforum with keyframed motion &amp; prompt</li>



<li>vid2vid with animatediffusion (im using 16 frame window with 8 frame overlap, using v3_sd15_mm)</li>



<li>optionally use hires fix upscaler (veerrry slow) or upscale frames afterwards</li>
</ul>



things to try:



<ul class="wp-block-list">
<li>start small and low framerate, scale up both later</li>



<li>rife frame interpolate at various steps</li>



<li>flux, sdxl, sd3 if possible (existing animatediff built for sd1.5)
<ul class="wp-block-list">
<li>maybe use in upscale</li>
</ul>
</li>



<li>cogvideo (not sure if vid2vid possible here)</li>
</ul>

high quality steerable ai videos

for the last year or two i&#8217;ve had a custom discord bot that i use to play around with the latest ai tools with friends in a convenient format i can reach from all my devices anywhere. self hosted discord bots are a ridiculously good service (maybe too good, i&#8217;m waiting for it to get ruined somehow) and a great project for practicing coding. 



currently my comfy bot repos are private, i&#8217;ll probably public them at some point but for now i can share the structure i used because i think it&#8217;s pretty flexible and helpful.



lightweight &#8216;always&#8217; bot



<ul class="wp-block-list">
<li>watches your discord for some trigger event
<ul class="wp-block-list">
<li>trigger could be: any message in a certain channel, message starting with a certain phrase, a slash command, adding a react to a message&#8230;</li>
</ul>
</li>



<li>when triggered it adds the message to a queue (ie by saving it in a db as a row)</li>



<li>stays on all the time so it doesn&#8217;t miss any events</li>



<li>write this whatever language you want, it&#8217;s not going to be doing much</li>
</ul>



&#8216;worker&#8217; bot/app



<ul class="wp-block-list">
<li>program to run locally that communicates with local ComfyUI API to generate images</li>



<li>fetches message/prompt from your queue, processes it with a comfyui workflow, then sends a response image straight to discord in reply to the message</li>
</ul>



this setup can be used for any heavy task. it excels for AI because a cloud model means you have to pay for storing the model and any involved tools/loras and when you spin up a machine you have to wait for those to download. this system can also work with multiple &#8216;workers&#8217; if you have a few machines you want to share work, or have friends that want to contribute to the rendering workload

comfyui discord bot

cornerstone post to link to all of my related GLSL ramblings and store some relevant links. will add more in time.



helpful tools:



<ul class="wp-block-list">
<li><a href="https://www.shadertoy.com/">shadertoy</a></li>



<li><a href="https://graphtoy.com/">graphtoy</a></li>



<li><a href="https://synesthesia.live/">synesthesia</a></li>



<li><a href="https://www.poshbrolly.net/">posh brolly</a></li>



<li><a href="https://thebookofshaders.com/">book of shaders</a></li>



<li>glsl docs
<ul class="wp-block-list">
<li><a href="https://www.khronos.org/opengl/wiki/Core_Language_(GLSL)">official wiki</a></li>



<li><a href="https://registry.khronos.org/OpenGL-Refpages/gl4/">official refpages</a></li>



<li><a href="https://docs.gl/sl4/all">docs.GL</a></li>
</ul>
</li>



<li><a href="https://iquilezles.org/articles/">inigo quilez&#8217;s blog</a></li>



<li><a href="https://github.com/patriciogonzalezvivo?tab=repositories">patricios gonzalez vivo&#8217;s repos</a></li>



<li>sdf builder tools&#8230;
<ul class="wp-block-list">
<li><a href="https://ephtracy.github.io/index.html?page=magicacsg">https://ephtracy.github.io/index.html?page=magicacsg</a></li>



<li></li>
</ul>
</li>
</ul>







Which GLSL version to use?



Unfortunately in 2024 I recommend 3.3. Almost every device has support for newer 4.x nowadays but if you want to support Macs (where OpenGL is deprecated 😬) then 3.3 seems the latest reliable version.



If you know you&#8217;re targeting newer Windows/Linux machines only you can play with 4.2+ and enjoy some newer capabilities, but most of them can also be &#8216;polyfilled&#8217; in a sense with workarounds in 3.3



<h2 class="wp-block-heading">Shadertoy</h2>



Shadertoy is an essential cornerstone to the online GLSL world and is where I&#8217;ve done so much of my GLSL learning.



my shadertoy: <a href="https://www.shadertoy.com/user/ufffd">https://www.shadertoy.com/user/ufffd</a>



breakdowns of my shaders:



to come. let me know if theres a particular one you want explained in greater depth







<h2 class="wp-block-heading">Synesthesia</h2>



breakdowns of my Synesthesia scenes:



to come. let me know if theres a particular one you want explained in greater depth

GLSL

my thoughts on programming languages, last updated Oct 2024



basically guaranteed to change



<ul class="wp-block-list">
<li>javascript: ⭐⭐⭐⭐⭐
<ul class="wp-block-list">
<li>goated, 5 stars, absolute chaotic trash that gets the job done. makes great sense at a high level, disgusting but impressive at lower levels</li>



<li>if you want to learn a programming language in 2024 you should almost definitely start here</li>
</ul>
</li>



<li>python: ⭐⭐⭐⭐
<ul class="wp-block-list">
<li>absolutely rules, makes sense in similar ways to js. teaches you to code clean, and the produced code just looks and feels noice</li>



<li>BUT&#8230; so much of what it does can be done with javascript, while you can&#8217;t do everything javascript does using python. lately i find myself phasing out my py projects for everything except ML, where pytorch gives python a near stranglehold.</li>



<li>do learn this if you want to do datascience or ML</li>
</ul>
</li>



<li>php: ⭐⭐⭐
<ul class="wp-block-list">
<li>my original love, the fuel behind WP and facebook. much maligned, but making a comeback . PHP does the job and does it well. if you&#8217;re making a website its a 4 or 5 really.</li>



<li>it was terrible in 5.6 but that&#8217;s the past</li>
</ul>
</li>



<li>typescript: ⭐⭐⭐⭐
<ul class="wp-block-list">
<li>yes, lower than javascript. i personally use it over js usually but it&#8217;s kind of a boat car in that you get all the hassle of types with none of the performance gains of a compiled language. still, it helps keep larger projects organized.</li>



<li>if you&#8217;ve been using javascript for a while, you should probably switch to ts at some point, its really easy to integrate in 2024</li>



<li>if youve switched entirely to typescript&#8230; maybe consider whether thats always necessary. modern javascript is also very capable</li>
</ul>
</li>



<li>C++: ⭐⭐⭐
<ul class="wp-block-list">
<li>amazing, powerful, tempted to give it 4 stars but it really does have its flaws and complications. </li>



<li>is it real time? you&#8217;re gonna end up using this</li>



<li>I will continue using it when performance matters and types are difficult, but lately i&#8217;ve been falling for&#8230;</li>
</ul>
</li>



<li>rust: ⭐⭐⭐⭐
<ul class="wp-block-list">
<li>the people love it and i agree</li>



<li>type safe, fast, looks cool, and man the ecosystem and compiler just rock. that might be the killer feature for me compared to the C++ anarchy</li>



<li>i think it could reach a 5 when the ecosystem evolves more. C++ still has a massive head start on things like libraries</li>
</ul>
</li>



<li>C: ⭐⭐⭐⭐⭐
<ul class="wp-block-list">
<li>i can&#8217;t claim any level of mastery but nobody can deny C its 5 stars</li>



<li>its here, it rocks, its so fundamental that i cant see it goin anywhere</li>
</ul>
</li>



<li>GLSL: ⭐⭐⭐⭐⭐
<ul class="wp-block-list">
<li>my true love language</li>



<li>truly we bicker enough that i would give it 4 stars, but there&#8217;s just nothing better</li>



<li>&#8220;simple&#8221;, versatile, and amazingly task suited. having the exact function you need, available with a simple name, is such a joy. need to reflect something? `reflect(something)`</li>



<li>surprisingly viable as a beginner language due to the limited &#8216;surface area&#8217;</li>
</ul>
</li>



<li>HTML &amp; CSS: ⭐⭐⭐⭐⭐
<ul class="wp-block-list">
<li>how can they not have 5 stars. where&#8217;s the competition?</li>



<li>seriously try writing raw html and css if you never have, it&#8217;s not complicated and it Just Works™</li>
</ul>
</li>



<li>sh/bash: ⭐⭐⭐
<ul class="wp-block-list">
<li>simultaneously underrated and overrated</li>



<li>you should learn some bash because it makes you a computer ninja</li>



<li>if you&#8217;re writing more than like 10 lines you should almost definitely just write a dang script in any other language</li>
</ul>
</li>



<li>lua: ⭐⭐⭐⭐⭐
<ul class="wp-block-list">
<li>haven&#8217;t actually written much if any lua, but its another language that just seems to do its job (scripting) perfectly</li>



<li>pretty please try this before writing your own scripting layer</li>
</ul>
</li>
</ul>



Languages I haven&#8217;t used enough to have strong thoughts on:



<ul class="wp-block-list">
<li>Go: seems cool, i&#8217;m trying it out for a project currently. seems great for efficient web services, fast but garbage collected is a cool niche</li>



<li>Gleam: definitely want to try this but it doesnt feel ready</li>



<li>Odin: same category as Gleam, especially seems great for visual heavy apps and games</li>



<li>C#: seems great but its just&#8230; never interested me</li>



<li>Java: same. these are some of the biggest languages but for some reason, meh</li>
</ul>

programming languages

i have a lot of thoughts and opinions about both of these things that i&#8217;ll start to accumulate here. to begin with here are brief thoughts on a few approaches you can take to making and managing websites:



<ul class="wp-block-list">
<li>nextjs &#8211; new kid on the block. pretty cool honestly, i&#8217;m using it for this website. but it&#8217;s quite complicated and even still, a little lacking when you compare it to something like WP or laravel from the php world. verdict is out on this one for me, i wouldn&#8217;t discourage it but i&#8217;d strongly encourage trying other things as well</li>



<li>wordpress &#8211; the mackdaddy, my original &#8220;love&#8221;, it got me into web development in a serious way and is imo a pinnacle of what community software can be and do. in many ways, it is hot garbage, but its some of the best hot garbage out of there and it really does empower small businesses and new devs in a way that jaded devs have a hard time relating to. in some drama recently with wp engine, we&#8217;ll see how that shakes out. use wordpress it rules. in fact this blog post is actually written in wordpress, i use it as a CMS for my nextjs website, just to make life more complicated! truly it has some benefits but plenty is written already just look up &#8220;headless wp&#8221;</li>



<li>laravel &#8211; rarely used this but it really seems like the way to go for any &#8216;serious&#8217; php dev. every language should aspire to have a framework like laravel</li>



<li>tina &#8211; really cool, edit in place and save to git, but i rarely actually use it, need to give it another try. might convert the wp cms to tina</li>



<li>markdown &#8211; the king, truly. just write all your notes and articles in markdown and save them in your repo, or host them elsewhere. markdown takes like 5 minutes to learn and it&#8217;s everywhere, learn it!</li>



<li>jekyll/11ty/hugo &#8211; static website builders are really sick and fun to play with, but at this point the major frameworks like nextjs are also capable of static export so these have lost a lot of their luster for me. 11ty is still cool if you want to go nearly barebones. you can learn a lot using these.</li>



<li>html &#8211; nearly as great as markdown imo &#8211; just write in friggin html! how hard is that? &lt;b>not hard&lt;/b>! it&#8217;s just a little more boilerplatey than markdown. that&#8217;s why you just write html in your markdown when and where its needed</li>



<li>htmx &#8211; have not used this but must say it looks 100% up my alley and has a great mindset</li>



<li>shopify &#8211; if you&#8217;re selling shit and you don&#8217;t want to think about web stuff, this is a slightly expensive but very good solution. i was a hater at first but they won me over with <a href="https://shopify.dev/docs/api/liquid">liquid</a> which I still sometimes use in projects today</li>
</ul>



<h3 class="wp-block-heading">my advice in a nutshell</h3>



start as simple as possible. probably html or markdown. then if that&#8217;s restrictive, start static building with next, or set up a wordpress or shopify site if you plan to sell anything. if your entire purpose is a blog and you want to do some nice formatting or host your own images, maybe try starting with wordpress or tina to begin with

CMS &amp; web frameworks

I&#8217;ve been really interested in the idea of algorithmically creating music for a while now. My first semi-serious experiments were in Sonic Pi. Here&#8217;s a public repo where I dumped my loose collection of Sonic Pi notebooks:



<a href="https://github.com/kylegrover/sonicpi-algobop">https://github.com/kylegrover/sonicpi-algobop</a>



<details class="wp-block-details is-layout-flow wp-block-details-is-layout-flow"><summary>Algobop Project Notes</summary>
<pre class="wp-block-code"><code>Algobop
Sample Based Song Generator
This is a really rough draft repo, just to get my previous work in sonic pi song generation up on git.

Running Algobop
Algobop runs in Sonic Pi. To run it you need to load all the relevant functions, then run a generator / template.

Algobop also requires samples, you'll need to point it at your sample libraries.

To Do / Plans / Ideas
Organize this repo lol
FX
Fills, Swells, Drops
Random "spice" samples
Add demo sample packs
More patterns/templates
Dynamic/generative patterns
Automate song generation. Make many songs at once.
Should this run in Sonic Pi? Maybe there's something better for my needs
License / Collaboration
Collaboration is very welcome, for the time being please ask ahead of time if you plan on using or modifying this code to create your own work. I might go GPL/MIT at some point but for now it's just a public personal project.</code></pre>




</details>



I still really like this approach and think it could serve as part of a good custom pipeline as it gives you a lot of direct control over the overarching structure, which other models aren&#8217;t great at.



OpenAI&#8217;s Jukebox came out some time after this and caught my attention a bit but wasnt flexible or &#8216;open&#8217; enough to play much with, but that&#8217;s the first moment I thought AI could really accomplish a lot here.



My next real attempt after Sonic Pi was using RAVE and it&#8217;s still one of my favorite approaches:



<a href="https://www.instagram.com/p/Cja4YexP-iw/?utm_source=ig_web_copy_link">https://www.instagram.com/p/Cja4YexP-iw/</a> live jamming with a custom finetuned BoC model



<a href="https://github.com/acids-ircam/RAVE">https://github.com/acids-ircam/RAVE</a> official repo



<a href="https://github.com/victor-shepardson/RAVE?tab=readme-ov-file#transfer-learning">https://github.com/victor-shepardson/RAVE?tab=readme-ov-file#transfer-learning</a> fork for training RAVEv2 (by victor shepardson who happens to be a <a href="https://www.shadertoy.com/user/victor_shepardson">killer glsl artist</a> as well)



somewhere around this timespan i also created a tonejs project very similar to the sonic pi work. i should probably pull that out of here into its own repo:



<a href="https://www.fxhash.xyz/generative/9492">https://www.fxhash.xyz/generative/9492</a> generative nft based on algobop js. click the buttons below the image and then click the image to hear endless variations



<ul class="wp-block-list">
<li>here&#8217;s one variation i like as a separate link <a href="https://gateway.fxhash2.xyz/ipfs/QmPtDAdXFbUCY7d8rPQbQSFyfHXK3hZgWThDeCeS9JaLPN/?fxhash=ooyCd3pJiS1CA24tYkvEqnpDPuHfm7RZvcpHSpyxT6KccnbXUaq&amp;fxchain=TEZOS&amp;fxiteration=5&amp;fxminter=tz1CenbLXcUV9xK97agvzCWT3FUqbLmR2nHh">https://gateway.fxhash2.xyz/ipfs/QmPtDAdXFbUCY7d8rPQbQSFyfHXK3hZgWThDeCeS9JaLPN/?fxhash=ooyCd3pJiS1CA24tYkvEqnpDPuHfm7RZvcpHSpyxT6KccnbXUaq&amp;fxchain=TEZOS&amp;fxiteration=5&amp;fxminter=tz1CenbLXcUV9xK97agvzCWT3FUqbLmR2nHh</a></li>
</ul>



other repos to try:



<ul class="wp-block-list">
<li><a href="https://github.com/Intelligent-Instruments-Lab/notochord">https://github.com/Intelligent-Instruments-Lab/notochord</a></li>



<li><a href="https://huggingface.co/stabilityai/stable-audio-open-1.0">https://huggingface.co/stabilityai/stable-audio-open-1.0</a></li>



<li><a href="https://huggingface.co/jadechoghari/openmusic">https://huggingface.co/jadechoghari/openmusic</a>
<ul class="wp-block-list">
<li><a href="https://github.com/ivcylc/qa-mdt?tab=readme-ov-file">https://github.com/ivcylc/qa-mdt?tab=readme-ov-file</a> ?</li>
</ul>
</li>



<li><a href="https://github.com/Intelligent-Instruments-Lab/notochord">https://github.com/Intelligent-Instruments-Lab/notochord</a></li>
</ul>

generative music

<a href="https://github.com/kylegrover/tomatwo">https://github.com/kylegrover/tomatwo</a>



<a href="https://github.com/itsKaspar/tomato">tomato.py</a> rewritten in rust



available as lib, cli, and gui:



download here: <a href="https://github.com/kylegrover/tomatwo/releases">https://github.com/kylegrover/tomatwo/releases</a>



<figure class="wp-block-image size-large"><img decoding="async" src="https://files.ufffd.com/ramblings/tomatwo/tomatwo_demo_screen.png" alt=""/></figure>



in action: <a href="https://www.instagram.com/p/C-ZjJvcxPYK">https://www.instagram.com/p/C-ZjJvcxPYK</a>



in my tests the rust version is ~5x faster for small files and ~50x faster for medium files



<hr class="wp-block-separator has-alpha-channel-opacity"/>



not sure how much i&#8217;ll push this forward, i may just attach a copyleft or open license and leave it for whoever wants to play with keyframes



i started a variation of this with a tauri web based ui, working title taurmatwo. not sure how i want to release or license that one:



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="932" height="759" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/10/image.png" alt="" class="wp-image-79" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/10/image.png 932w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/10/image-300x244.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/10/image-768x625.png 768w" sizes="(max-width: 932px) 100vw, 932px" /></figure>

tomatwo

not a lot here yet, may add some thoughts as they come up



teia is a fork of <a href="https://wordpress-production-3768.up.railway.app/hic-et-nunc/" data-type="post" data-id="71">hic-et-nunc</a>



see my art: <a href="https://teia.art/ufffd">https://teia.art/ufffd</a>



here are some repos ive worked on related to teia:



<ul class="wp-block-list">
<li><a href="https://github.com/teia-community/teia-ui">https://github.com/teia-community/teia-ui</a> of course</li>



<li><a href="https://github.com/teia-community/community-vote">https://github.com/teia-community/community-vote</a> fork of hicvote hicathon team project created to settle early community decisions publicly
<ul class="wp-block-list">
<li>now it&#8217;s outdated with new blockchain tallied and enacted votes created by jagracar and im glad for that</li>
</ul>
</li>



<li><a href="https://github.com/teia-community/teia-blog">https://github.com/teia-community/teia-blog</a> a simple next blog created quickly and early as a hub for teia communications.
<ul class="wp-block-list">
<li>comments are powered by a discourse instance i had been hosting for hicetnunc and cloned for teia. i hosted the blog and discourse for a while but its since been adopted by ziroh</li>
</ul>
</li>



<li><a href="https://github.com/kylegrover/tz-vertex">https://github.com/kylegrover/tz-vertex</a>
<ul class="wp-block-list">
<li>a project im starting to help tezos nft users, primarily on teia, to backup, host, and view nfts they own, view, created, or care about.</li>



<li>built with go, using ipfs (this is the first time ive touched go)</li>
</ul>
</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



teia fountain: great volunteer donation based project that&#8217;s been distributing funds to global artists to get started online since before teia. tezos nft world doesnt get enough credit for not only being the cheapest and most sustainable option early on, we were also out here giving people enough money to create a profile and mint a few artworks. a lot of artists got their start here:



<a href="https://github.com/teia-community/teia-fountain">https://github.com/teia-community/teia-fountain</a> (this is just the repo for the bot that helps manage the fountain)

teia

not much here yet, might write some thoughts here as they come up



<a href="https://github.com/hicetnunc2000/hicetnunc">https://github.com/hicetnunc2000/hicetnunc</a>



<a href="https://github.com/hicdex/hicdex">https://github.com/hicdex/hicdex</a>







here are a few pubic repos I made related to hicetnunc:







<ul class="wp-block-list">
<li><a href="https://github.com/kylegrover/ibi-et-tum">https://github.com/kylegrover/ibi-et-tum</a> i dont remember, might just be clone of og hic api</li>



<li><a href="https://github.com/kylegrover/alterhen">https://github.com/kylegrover/alterhen</a></li>



<li><a href="https://github.com/kylegrover/hicettum">https://github.com/kylegrover/hicettum</a> &#8212; private repo for a custom UI/fork, might make public. theres not much there really, i started just contributing to hicetnunc instead
<ul class="wp-block-list">
<li><a href="https://github.com/kylegrover/hic-et-tum">https://github.com/kylegrover/hic-et-tum</a> also</li>
</ul>
</li>



<li><a href="https://github.com/kylegrover/tzkthelper">https://github.com/kylegrover/tzkthelper</a></li>



<li><a href="https://github.com/kylegrover/henredirect">https://github.com/kylegrover/henredirect</a></li>



<li><a href="https://github.com/kylegrover/alterhen-gatsby">https://github.com/kylegrover/alterhen-gatsby</a></li>



<li><a href="https://github.com/kylegrover/hicvote">https://github.com/kylegrover/hicvote</a></li>



<li><a href="https://github.com/kylegrover/hicetnunc-art-redirect">https://github.com/kylegrover/hicetnunc-art-redirect</a></li>
</ul>

hic et nunc

spurred by <a href="https://www.facebook.com/groups/134453177040088/?multi_permalinks=1825418501276872">this fb post</a>:



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
Hey y&#8217;all. I am desperately looking for tutorials on coverting multipass shaders from Shadertoy into Synesthesia. I&#8217;ve got the basics of manually importing shaders into Synesthesia. I am absolutely stumped on how to properly configure shaders that use multiple buffers / passes to work in the software. Is there anyone out there that may be able to help walk me through how to import shaders such as the below? I know it involves setting up the JSON file for multiple passes, as well as adding some code into the main.glsl to call those passes. If I could figure out this process, I could simply repeat for other shaders I&#8217;m looking to import &#8211; but there&#8217;s a knowledge gap here for me that seems it would require many months of deep diving into GLSL to suss out.
</blockquote>



using DotCamera shadertoy: https://www.shadertoy.com/view/dlKSDV



note: this was also posted as a comment reply on Facebook, I&#8217;m just copying it here for posterity



First it helps to understand a big fundamental difference between how Shadertoy and Synesthesia handle multipass shaders: Shadertoy uses iChannels and Buffers. Each buffer is a separate piece of code that has its own iChannels assigned, which in turn contain some texture source. Synesthesia puts all buffers into one code file, using PASSINDEX to determine which block of code to run. This means in Shadertoy, each buffer can define its own version of the same function or variable, and each buffer has its own &#8216;iChannel0&#8217;. In Synesthesia all the variables and functions are available in all passes, and each pass has access to the same textures.



So putting it all together, the process of converting DotCamera looks like:



<ol class="wp-block-list" start="0">
<li>Enable Built-In Scene Editor in settings Set at least one custom scene folder Set &#8216;primary&#8217; on the folder you&#8217;d like to create/import new scenes</li>



<li>Load DotCamera using the right sidebar import tab It will fail, click &#8216;create synScene in library&#8217; anyway</li>



<li>Find the scene in your library and hit edit to load it into the scene editor It will still be broken but the format has been converted to fit Synesthesia. This should usually handle all of the iChannels, buffers, and passes for us, but sometimes there are still issues.</li>



<li>Fix the bugs Checking the Synesthesia console you&#8217;ll see an error message In this case the problem is on line 44, incompatible return type. This is because Synesthesia expects you to return a vec4 while Shadertoy expects you to set a certain variable (usually <code>fragColor</code>, but in this shadertoy it&#8217;s changed to <code>O</code>) to a vec4 and then return void. just change <code>return;</code> to <code>return O;</code></li>
</ol>



And that&#8217;s all it took, it compiles! This actually turned out to be an easier one. There&#8217;s still a couple small issues that are easy to fix. Swapping syn_Media in for syn_UserImage on line 20 will correctly flip the image. Finally go to the Media Settings in the scene config sidebar and set filter mode to &#8216;mipmap&#8217; and wrap mode to &#8216;clamp&#8217; to match the settings used in Buffer A iChannel0 and it should be pretty much working! Let me know if you get it working and share a video if you make something cool out of it.

converting multipass shadertoys to synesthesia

<figure class="wp-block-embed is-type-link is-provider-sponsored-by-embedez wp-block-embed-sponsored-by-embedez"><div class="wp-block-embed__wrapper">
https://www.instagramez.com/p/C3v7o-ysLve/
</div></figure>



midichonk is my first custom MIDI controller. Its just 4 arcade buttons and 4 sliders in a custom enclosure but it&#8217;s compact, expandable/repairable, and I definitely learned some stuff putting it together. I&#8217;m far from a professional but I thought I&#8217;d document what I learned in the process in case somebody wants to use it as a jumping off point.



<h2 class="wp-block-heading">My Process</h2>



<h3 class="wp-block-heading">Planning</h3>



I&#8217;ve thought of making a MIDI controller before but when I saw my friend (shout out <a href="https://digitalintrospect.com/">Digital Introspect</a>) starting to make his own I thought it was a good time to join along. I followed his lead and bought an Arduino Leonardo which rocks for this task for a few reasons:



<ul class="wp-block-list">
<li>Native USB MIDI support</li>



<li>Common + Open-Source = Resources Galore</li>



<li>Capable chip with a good number of digital and analog IO pins</li>



<li>Power and program over micro USB</li>
</ul>



I also decided to buy a few components before I landed on a final design &#8211; arcade buttons, linear potentiometers, and some toggle switches that I ended up not using (yet). You could probably plan your design first but it ended up being convenient to be able to measure the components and recreate them in 3D before finalizing.



I had some much more complicated ideas than I even drew out, but I ultimately decided on the best simple starting point being 4 sliders and 4 buttons. For a first device I recommend staying within 6 analog inputs (sliders, xy pads) and 12 digital IOs (buttons, toggles, LEDs). You could do more than that on a Leonardo but it involves getting fancy.



<h3 class="wp-block-heading">Sketching</h3>



Once I have a direction I like to switch to a good old doodle. I sketched some options out in Procreate before switching into <a href="https://www.autodesk.com/products/fusion-360/personal">Fusion360</a> (free for personal use!):



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="768" height="1024" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-768x1024.png" alt="" class="wp-image-40" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-768x1024.png 768w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-225x300.png 225w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-1152x1536.png 1152w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image.png 1536w" sizes="(max-width: 768px) 100vw, 768px" /></figure>



Once I had a better sense of the layout I wanted, I measured all my components and recreated them in Fusion, including the Arduino. I laid them out and designed a simple enclosure to fit them by extruding a rounded block, splitting it in 2 parts, and using the &#8216;shell&#8217; modifier on both halves. I added cutouts and supports for the sliders and buttons, and some spots to screw the case together.



<div class="wp-block-group is-nowrap is-layout-flex wp-container-core-group-is-layout-1 wp-block-group-is-layout-flex">
<figure class="wp-block-image size-full is-resized"><img loading="lazy" decoding="async" width="948" height="712" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1925.png" alt="" class="wp-image-43" style="width:313px;height:auto" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1925.png 948w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1925-300x225.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1925-768x577.png 768w" sizes="(max-width: 948px) 100vw, 948px" /></figure>



<figure class="wp-block-image size-full is-resized wp-duotone-unset-1"><img loading="lazy" decoding="async" width="973" height="731" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-2.png" alt="" class="wp-image-42" style="width:319px;height:auto" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-2.png 973w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-2-300x225.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-2-768x577.png 768w" sizes="(max-width: 973px) 100vw, 973px" /></figure>
</div>



I also needed a mounting location for the Arduino. I decided to stick it to the bottom and include enough wire length to where it could be opened, which I plan on doing. I initially tried a way too detailed pin design but landed on a simplified system of bent nubs that create a satisfying press fit. 



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="870" height="477" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-3.png" alt="" class="wp-image-44" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-3.png 870w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-3-300x164.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-3-768x421.png 768w" sizes="(max-width: 870px) 100vw, 870px" /></figure>



<figure class="wp-block-video"><video controls src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/427909095_7139947052789765_6848021408976118183_n.mp4"></video></figure>



Don&#8217;t forget to cut a clearance for the Arduino USB and Power. I might have missed the mark by a mm or two my first time and fixed it with a knife. The Fusion file should be updated now but it never hurts to check against your Arduino.



<h2 class="wp-block-heading">Testing</h2>



For testing I used a breadboard and some alligator clips and jumpers to try wiring everything together in different ways. It&#8217;s all 5v so you can kind of just fuck around. The wiring is incredibly simple, with the <a href="https://docs.arduino.cc/learn/microcontrollers/digital-pins/">built in pull-up resistors</a> enabled with a line of code all that you need to do to wire up a button is connect one lead to your chosen input pin and the other lead to ground. Toggles work about the same. Sliders require VCC in, ground, and a third lead connects to your analog input pin (A0-A5)



Once you&#8217;re wired up you can plug the Leonardo into your computer and boot up <a href="https://www.arduino.cc/en/software">Arduino IDE</a> and connect to your Leonardo.



Here&#8217;s some code you can use with a button connected to pin 2 to send a MIDI note:



<pre class="wp-block-code has-small-font-size"><code>#include "MIDIUSB.h" // this will do the heavy lifting for us

// set up variables
const int buttonPin = 2; // we'll be using the pin labeled 2 on the board
bool lastButtonState = HIGH; // HIGH is the off state with the pull-up resistor

void setup() { // this runs once
 pinMode(buttonPin, INPUT_PULLUP); // Set button pins as input with pull-up
}

void loop() { // this runs repeatedly
 bool reading = digitalRead(buttonPin); // read the state of the button
 if (reading != lastButtonState) { // if the button state has changed
 lastButtonState = reading; // save the new state
 if (reading == LOW) { // if the button is pressed
 noteOn(0x90, 60, 127); // send a note on message
 } else { // if the button is released
 noteOn(0x80, 60, 0); // send a note off message
 }
 }
}

// Helper function to send MIDI messages
void noteOn(byte cmd, byte data1, byte data2) {
 midiEventPacket_t event = {0x09, cmd, data1, data2};
 MidiUSB.sendMIDI(event);
 MidiUSB.flush();
}</code></pre>



You can test your MIDI controller in this stage as well by connecting to a MIDI app of your choice. I&#8217;ve been getting great use from <a href="https://benjamin.kuperberg.fr/chataigne/en">Chataigne</a>, with its MIDI monitor mode.



You can also use the Serial output in Arduino IDE with <code>Serial.println("")</code> or lit up some LED on the Arduino to make sure components worked.



<h2 class="wp-block-heading">Building the Case</h2>



To actually build the case I fired up the Ender 3 and printed the two halves in a nice Sakura Pink PLA+, with the flat top or bottom facing down. Settings will vary based on your printer but a pretty default setting on 0.4 nozzle with 0.2 height should fit the design well if you want to recreate it. I had a few failures with the edges curling, brims helped for adhesion.



If you don&#8217;t have a 3D printer you have some options: &#8211; Ask a friend, you probably know someone that has one &#8211; Buy one: they&#8217;re cheap! The <a href="https://amzn.to/3T5REJ8">Ender 3 V3 SE</a> is fantastic for 200 new. If you have more to spend the Bambu P1P is butter smooth. And if you want to save money you can probably find an old Ender 3 for 100. The less you spend the more time you&#8217;ll spend learning about 3D printers, for better or worse.



<div class="wp-block-group is-nowrap is-layout-flex wp-container-core-group-is-layout-2 wp-block-group-is-layout-flex">
<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="1003" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-4-1024x1003.png" alt="" class="wp-image-46" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-4-1024x1003.png 1024w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-4-300x294.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-4-768x752.png 768w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image-4.png 1074w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="975" height="977" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1927.png" alt="" class="wp-image-48" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1927.png 975w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1927-300x300.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1927-150x150.png 150w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1927-768x770.png 768w" sizes="(max-width: 975px) 100vw, 975px" /></figure>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="954" height="750" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1928.png" alt="" class="wp-image-49" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1928.png 954w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1928-300x236.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1928-768x604.png 768w" sizes="(max-width: 954px) 100vw, 954px" /></figure>
</div>



<h2 class="wp-block-heading">Assembly</h2>



The assembly is pretty simple, screw the buttons in and align them. push all the sliders in and either screw them in from the top or just glue them from the bottom if you&#8217;re lazy (I&#8217;m lazy). Solder all the grounds and powers together (maybe you could solder before installing hardware but i didn&#8217;t) then connect the sliders to the analog pins, and connect the buttons and button LEDs to pins 2 through 9 in pairs, going Button A, LED A, Button B, LED B so on



I suck at soldering and you can too! Just tin the iron, tin the wire, use dupont wires for the Arduino end if it comes with headers. and test everything well before you close it all up or you&#8217;ll kick yourself



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="576" src="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1929-1024x576.png" alt="" class="wp-image-50" srcset="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1929-1024x576.png 1024w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1929-300x169.png 300w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1929-768x432.png 768w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1929-1536x864.png 1536w, https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/image_1929.png 1920w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<h2 class="wp-block-heading">Everything You Need To Make Your Own</h2>



Tools:



<ul class="wp-block-list">
<li>soldering iron &#8211; you can get decent ones for as little as 10 bucks. i&#8217;ve heard the battery ones are nice for simple stuff like this</li>



<li>3D Printer (The <a href="https://amzn.to/3T5REJ8">Ender 3 V3 SE</a> is a great deal currently in Feb 2024)</li>



<li>wire strippers (you don&#8217;t need them need them but they sure are handy and cheap)</li>
</ul>



Testing:



<ul class="wp-block-list">
<li><a href="https://amzn.to/3OSfYeQ">breadboard</a></li>



<li><a href="https://amzn.to/48pvGoP">alligator clips</a></li>



<li><a href="https://amzn.to/49kxQHz">alligator to dupont</a></li>



<li><a href="https://amzn.to/3wnRjZk">dupont ribbon/peel cable</a></li>
</ul>



Parts:



<ul class="wp-block-list">
<li>Any 3D Printing Filament, ~100g
<ul class="wp-block-list">
<li>I used <a href="https://amzn.to/49DCzDL">Sakura Pink PLA+</a>: $28 per roll, approximately $2.80 used</li>
</ul>
</li>



<li>Optional: second filament for slider caps, or buy slider caps</li>



<li><a href="https://amzn.to/3uEOu5s">Arcade Buttons</a>: $14.50 for set of 6, around $9.67 for 4 used</li>



<li><a href="https://amzn.to/48mieCa">Sliders</a>: $14 for set of 10, about $5.60 for the 4 I used</li>



<li><a href="https://amzn.to/42NiVD5">Leonardo Arduino</a> $15 you can get these from any supplier they&#8217;re all about the same, this one was cheap</li>



<li>22 AWG Wire: Prices vary greatly, let&#8217;s consider a conservative estimate of $2.00 for the portion used in this project.</li>



<li>3x 2.5 screws. you can drill the screw holes to size and use any screws that fit. I used <a href="https://amzn.to/42JGn4e">this kit</a> $10, used parts cost pennies</li>



<li>Could also use <a href="https://amzn.to/42JG4GC">heat set inserts</a> for the screws. I didn&#8217;t</li>
</ul>



parts used cost: ~$35



3D Model:



Use .f3d to load in Fusion 360 or choose the file type you need



<a href="https://files.ufffd.com/ramblings/midichonk/midichonk+v21.f3d">.f3d</a> &#8211; <a href="https://files.ufffd.com/ramblings/midichonk/midi+controller+v21.step">.step</a> &#8211; <a href="https://files.ufffd.com/ramblings/midichonk/midichonk+v21.3mf">.3mf</a> &#8211; <a href="https://files.ufffd.com/ramblings/midichonk/midichonk+v21.obj">.obj</a> &#8211; <a href="https://files.ufffd.com/ramblings/midichonk/midichonk+v21.skp">.skp</a>



Exported parts:



<div class="wp-block-file"><a id="wp-block-file--media-9806f885-50fc-4708-90aa-58141468f979" href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/midichonk-L1body-bottom.stl">midichonk-L1body-bottom</a><a href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/midichonk-L1body-bottom.stl" class="wp-block-file__button wp-element-button" download aria-describedby="wp-block-file--media-9806f885-50fc-4708-90aa-58141468f979">Download</a></div>



<div class="wp-block-file"><a id="wp-block-file--media-ef673c4b-2180-47ec-971b-06e4ebe5034c" href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/midichonk-L1-body-top.stl">midichonk-L1-body-top</a><a href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/midichonk-L1-body-top.stl" class="wp-block-file__button wp-element-button" download aria-describedby="wp-block-file--media-ef673c4b-2180-47ec-971b-06e4ebe5034c">Download</a></div>



<div class="wp-block-file"><a id="wp-block-file--media-ea6f96be-5c2a-48c8-9130-039fa3eb478b" href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/slider-knob-1.stl">slider-knob-1</a><a href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/slider-knob-1.stl" class="wp-block-file__button wp-element-button" download aria-describedby="wp-block-file--media-ea6f96be-5c2a-48c8-9130-039fa3eb478b">Download</a></div>



And the final code I used for my MIDI controller, which handles 4 sliders, 4 arcade buttons that light up, and also incorporates some debouncing and other checks to avoid misfires:



<div class="wp-block-file"><a id="wp-block-file--media-3dce0444-7d25-45a8-a193-08f7870d4b3c" href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/LeonardoMidichonk.ino">LeonardoMidichonk</a><a href="https://wordpress-production-3768.up.railway.app/wp-content/uploads/2024/02/LeonardoMidichonk.ino" class="wp-block-file__button wp-element-button" download aria-describedby="wp-block-file--media-3dce0444-7d25-45a8-a193-08f7870d4b3c">Download</a></div>



<h3 class="wp-block-heading">That&#8217;s it! That&#8217;s all it takes</h3>



Let me know if you have any questions, I&#8217;d be glad to help you get your project going. I might do a follow up with some of my ideas for a V2 (OSC, joysticks, xy, endless rotating encoders, built in screens, multiplexing, modulation&#8230;), though maybe that will have to wait until I actually make a V2.

How I Built My Custom MIDI Controller

absolute hodge podge of urls



<a href="https://cybercult.net/">https://cybercult.net/</a> buy my clothes



<a href="https://jizz.lol/ecoliving">https://jizz.lol/ecoliving</a> ai generated geocities pages



<a href="https://www.shadertoy.com/user/ufffd">https://www.shadertoy.com/user/ufffd</a> some of my shader art



<a href="https://bardmagic.netlify.app/">https://bardmagic.netlify.app/</a> automagical website 4 ur band (not released yet)

linky mclinkerton

Console logging 60 times per second isn\&#8217;t really useful to humans, use this function to only log to the javascript console once per second:



<pre class="wp-block-code"><code>var logtime = 0;
function update() {
 logtime++;
 if (logtime > 60) {
 logtime = 0;
 console.log(something);
 }
 ...</code></pre>

Smarter Console Logging in Synesthesia

The JavaScript layer in Synesthesia is ridiculously useful, and the least familiar aspect for those coming from ShaderToy. In the future I\&#8217;d like to do a full article and/or video on this, but for now here\&#8217;s a simple recipe to follow for your script.js file:



<pre class="wp-block-code"><code>// declaring an object
Something() {
 this.x = 0
 this.y = 0
 this.z = 0
}
// giving the object a function to call each frame
Something.prototype.tick = function() {
 // determine how the variables should change in each frame
 delta = {
 x: syn_level,
 y: syn_BassLevel,
 z: syn_HighHits
 }
 // apply changes
 this.x += delta.x
 this.y += delta.y
 this.z += delta.z
}
// instantiate our object
var something = new Something();
// pass our object instance to the built in variable `uniforms`
uniforms.something = something;

// the `update()` function runs once each frame, Synesthesia requires this
function update(data) {
 // call our `tick()` function on our object instance
 something.tick();
}</code></pre>



Now in the main.glsl file we can reference these variables as a vec3 like so:



<pre class="wp-block-code"><code>float something_x = something.x;
vec2 something_xy = something.xy;
// using a sin() fxn to keep the results within the &#91;0-1] range
vec3 color = sin(something); </code></pre>



This really only scratches the surface but if you have JavaScript knowledge/experience then this should open up whole worlds for you 🙂

Synesthesia Quick Tip: JavaScript Variables

Note: This is outdated, check <a href="https://app.synesthesia.live/docs/ssf/functions.html" data-type="link" data-id="https://app.synesthesia.live/docs/ssf/functions.html">the docs</a> for modern methods.



This simple function returns user media if any is selected, or if not it returns black.



To change the fallback color just change <code>vec3(0)</code> to the color of your choice



<pre class="wp-block-code"><code>vec3 usermedia_or_black() {

    vec3 color = vec3(0);

  if (_exists(syn_UserImage)){

    color = _loadUserImage().rgb;

  }

  return color;
}</code></pre>

Synesthesia Quick Tip: Load Media

So you&#8217;ve downloaded <a href="https://synesthesia.live/" data-type="link" data-id="https://synesthesia.live/">Synesthesia</a>, you&#8217;re playing around with the scenes and having fun, but you want to control something and the slider just isn&#8217;t there. Maybe you want to control the speed of an effect, or more control over the audio reactivity, I see a lot of people asking questions about Synesthesia scenes that ultimately boil down to needing this skill, so I figured I&#8217;d type it out real nice one time to share again and again.



This guide aims to take an absolute beginner through all the steps needed without requiring or diving into the fundamentals of coding, GLSL, JS, or JSON, but those are all great fundamentals you&#8217;ll want to learn at some point. If you&#8217;d like to build your understanding of GLSL Code Art I highly recommend the free online <a href="https://thebookofshaders.com/" data-type="link" data-id="https://thebookofshaders.com/">Book of Shaders</a> and of course <a href="https://www.synesthesia.live/docs/index.html" data-type="link" data-id="https://www.synesthesia.live/docs/index.html">Synesthesia&#8217;s Official Docs</a>.



<h3 class="wp-block-heading">A Quick Overview of the Process</h3>



I&#8217;m going to break this down into two basic steps: first we&#8217;ll create a slider, then we&#8217;ll modify the code so our slider modifies something. Once you&#8217;re comfortable with that it&#8217;s just rinse and repeat until something cool happens.



But before we start editing files let&#8217;s take a step back and look at how a scene is structured and how the program reads it. Synesthesia scenes have 4 main files, each in a different format:



<ul class="wp-block-list">
<li>main.glsl: graphics processing code lives here</li>



<li>scene.json: settings are written here in JSON (I&#8217;ll talk more about JSON below)</li>



<li>script.js*: an optional file that allows you to do some extra processing in JavaScript before each frame runs</li>



<li>some_name.png/jpg: the scene thumbnail, as listed in the scene.json file</li>
</ul>



Note: Synesthesia Scenes can also include any number of additional images to read from within your GLSL code.



Thankfully we only need to work with 2 or 3 of those files, depending on the scene. First we&#8217;ll focus on the most common scenario where we&#8217;re only editing the GLSL file and the JSON file, after that we&#8217;ll deal with the JavaScript file.



<h3 class="wp-block-heading">Adding a Slider to the Scene</h3>



Let&#8217;s start out by adding our own slider to the control panel. It won&#8217;t do anything yet, but it&#8217;s a good place to start. I&#8217;ll be working with the default scene Glassier if you&#8217;d like to work along.



&lt;ol&gt;



  &lt;li>If you don&#8217;t have a scene ready to edit, choose one from the library view and click the &#8220;duplicate scene&#8221; button indicated by two documents, then find that new scene (ie &#8220;Glassier_dup&#8221;) and click the &#8220;edit scene&#8221; button indicated by a pencil.&lt;/li>



  &lt;li>You should be greeted by the scenes folder. Open the file &#8220;scene.json&#8221; in your &lt;a href=&#8221;https://mikkegoes.com/finding-the-best-text-editor-for-coding/&#8221; target=&#8221;_blank&#8221; rel=&#8221;noopener noreferrer&#8221;>favorite code editor&lt;/a> (it may appear as just &#8220;scene&#8221; depending on your OS preferences)&lt;/li>



&nbsp; &lt;li&gt;Note all the existing settings, and the format they&#8217;re written in. Try to figure out what they&#8217;re doing. If you&#8217;re editing Glassier like I am, note that it&#8217;s made by the fantastic &lt;a href=&#8221;https://www.shadertoy.com/user/Flexi&#8221; target=&#8221;_blank&#8221; rel=&#8221;noopener noreferrer&#8221;&gt;Felix Woitzel&lt;/a&gt;.&lt;/li&gt;



&nbsp; &lt;li&gt;



&nbsp; &nbsp; Find the section wrapped in <code>CONTROLS: [...]</code> and add this after the <code>[</code> but before the next <code>{</code>



<pre class="wp-block-code"><code>{

&nbsp; "NAME": "my_new_slider",

&nbsp; "TYPE": "slider",

&nbsp; "UI_GROUP": "my custom controls",

&nbsp; "DEFAULT": 0,

&nbsp; "MAX": 1,

&nbsp; "MIN": 0,

&nbsp; "DESCRIPTION": "This controls whatever I want it to!"

},</code></pre>



&nbsp; &lt;li&gt;Save the file and reload your scene in Synesthesia. If you did everything right, you should see your slider show up in the control panel. It won&#8217;t do anything yet, but we&#8217;re getting somewhere! If you see loads of red text in the Console Output at bottom right, you probably goofed up your syntax, as I expertly demonstrate in the screenshot below. Code editors with color highlighting are your best friend for spotting these types of errors.&lt;/li&gt;



&lt;/ol&gt;



&lt;img src=&#8221;https://res.cloudinary.com/kylegrover/image/upload/./v1555781631/goofed-up-json-syntax-synesthesia.png&#8221; alt=&#8221;&#8221; class=&#8221;align-center&#8221;&gt;



<h2 class="wp-block-heading">Connecting Your Slider to a GLSL Variable</h2>



Okay, we have our very own slider &#8211; rad. The configurations you added to your scene.json file created a slider that goes from 0 at the bottom to 1 at the top, and we can access that number with the variable name &lt;code class=&#8221;highlighter-rouge&#8221;&gt;my_new_slider&lt;/code&gt;. Now, how do we make it do something?



Put simply, we&#8217;re going to use our variable to modify another variable. Every bit of information in a shader is stored as a variable. All of these variables serve to drive functions and logic that ultimately output each pixel&#8217;s color values each frame. We&#8217;re going to: choose a variable to manipulate, choose a way to manipulate it, and then test it out!



<h4 class="wp-block-heading">Choosing a Variable or Number</h4>



Your options here are truly limitless, but to avoid diving into the specifics of how one shader works I&#8217;ll play with something that&#8217;s relevant to every scene: color. If you don&#8217;t know what you want to modify, just pick a random variable or number and see what happens when you modify it! If you do have a specific idea in mind, you might luck out by searching for a relevant variable name or exploring &lt;a href=&#8221;https://www.synesthesia.live/docs/uniforms/index.html&#8221; target=&#8221;_blank&#8221; rel=&#8221;noopener noreferrer&#8221;>Synesthesia&#8217;s built in &#8220;Uniforms&#8221;&lt;/a>, but you might need to spend a while getting know the code before you find that magic variable.



I&#8217;m going to bypass all of that for now by jumping right to the end of the GLSL file and finding the final &lt;code class=&#8221;highlighter-rouge&#8221;>return&lt;/code> line &#8211; for Glassier that was line 783: &lt;code class=&#8221;highlighter-rouge&#8221;>return fragColor;&lt;/code>. This line says &#8220;okay, we&#8217;re done here, set the current pixel to this RGBA color&#8221;. The variable I&#8217;ll be modifying is fragColor, or more specifically fragColor.r &#8211; the red value.



<h4 class="wp-block-heading">Choosing an Operator</h4>



Now we have two variables, our custom slider that goes from 0 to 1, and our chosen variable &#8211; mine is a red channel value that also ranges from 0 to 1. All that&#8217;s left is to choose what to do with these numbers, and again your options are as endless as the field of mathematics, but let&#8217;s try two basic examples &#8211; multiplying, overriding &#8211; and then one weirder example.



Multiplying: this one&#8217;s simple, I&#8217;ll just add this line before the return line: <code>fragColor.r *= my_new_slider;</code> In english: Take the current red channel value and multiply it by my slider value. With this saved, the slider has no effect when it&#8217;s at the top, and at the bottom it removes the red entirely. Here&#8217;s how it looks around 0.3:



&lt;img src=&#8221;https://res.cloudinary.com/kylegrover/image/upload/./v1555785135/glassier-slider-red-times-equals.png&#8221; alt=&#8221;&#8221; class=&#8221;align-wide&#8221;&gt;



Not bad, one line of code gives us a new color scheme!



Overriding: even easier, instead of <code>*=</code> we&#8217;ll just use <code>=</code>. When the slider&#8217;s at the bottom this will look just like when we multiplied &#8211; no red &#8211; but as you dial it up it&#8217;ll add a flat red tinge to everything:



&lt;img src=&#8221;https://res.cloudinary.com/kylegrover/image/upload/./v1555785568/glassier-slider-red-override.png&#8221; alt=&#8221;&#8221; class=&#8221;align-wide&#8221;&gt;



Weird Stuff: Let&#8217;s branch out. What if we raise the red value to the power of the slider, or vice versa? How about if we wrap it in a modulo operator? Let&#8217;s try both with <code>fragColor.r = mod(pow(my_new_slider, fragColor.r), 0.1) * 10.0;</code>



&lt;img src=&#8221;https://res.cloudinary.com/kylegrover/image/upload/./v1555784909/glassier-slider-red-mod-pow.png&#8221; alt=&#8221;&#8221; class=&#8221;align-wide&#8221;&gt;



Wicked! And there&#8217;s no end to the depth of control you can configure. In that line above I included a few hard-coded numbers, 0.1 and 10.0 &#8211; those could both also be sliders instead.



<h3 class="wp-block-heading">The End, For Now</h3>



This post has really grown beyond my initial plans and I&#8217;m going to call it for now and post what I have online. Hopefully this helps someone out, check back in for an updated version where I tackle the scripts.js file as well as a few tips for smoothly modifying your time variables.

�'s Ramblings