Input texture into GL3 - Shader Toy Loader
Hello,
Been struggling to get a texture inside the shader toy example, or render the noise function that's in the example*. I've tried addressing iChannel0 thinking that's where the noise should be coming through.
*(Extras>GL-3>ShaderToy loader (install Gl-3 and enable in prefs))
I've tried with a video piping to a gl.texture and naming the output, then putting a metronome and banging the name of the file into the slab but I must be doing something wrong. That seems to be the way I've seen others in patchers do it, but I must be missing something.
and this is the part in the pfft (not important for the functionality of testing, just if anyone's interested):
The end goal is stated in the comments. Getting a texture into the slab and addressing it as iChannel0 is really the first step.
I've tried naming the the gl.texture as iChannel0, however after searching on the forum I've found you can't or shouldn't name the texture the same as the variable inside the shader.
If anyone has the slightest clue as to how to effectively get a texture inside this shader I've be very thankful! Been banging my head against the wall, I know it's got to be a relatively small issue.

I've figured getting input now, seems I'm learning the quirks of jitter, disabling the texture output made it work. FFT.
I've now got an FFT texture from jitter into shadertoy, now to make it look or format the data like IQ's, which may be a little harder.
Currently it's printing a texture where the x axis is time, like a histogram, I don't think the one on shadertoy works like that. I've tried to visualise what's going on in the shadertoy audio FFT by mapping the FFT bins to the resolution of the screen like so:
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/iResolution.xy;
vec4 col2 = texture(iChannel0, uv);
// Output to screen
fragColor = col2;
}
Where iChannel0 is set to a sound buffer or sc input, the spectrum looks like:

So I'm guessing the lines are the spectrum bins and the faded bits are the waveform.
I'm not sure how to do the waveform part yet, so I'm just going to focus on the mapping the bins correctly. If anyone has any ideas or thoughts on how this could be achieved succinctly please let me know!
This is another test, but as you can see the output is not the same as Shadertoy's:

Ok massively close, but not a direct replica or drop in replacement for the sound iChannel0 just yet.
Here's the results from my progress so far:
https://youtu.be/rriN2uJQ3ao
and here's the patch:
I may have to call it a night now and stop working on it. It's functional, but the mapping is not perfect. (upside down, and haven't worked out the waveform part either yet)
If anyone wants to take the reigns or has anything to add please feel free to leave a comment.
And the code in the jxs patch is here (as I'm unsure if it attaches the code inside the compressed maxpat):
<jittershader name="stripes">
<param name="modelViewProjectionMatrix" type="mat4" state="MODELVIEW_PROJECTION_MATRIX" />
<param name="textureMatrix0" type="mat4" state="TEXTURE0_MATRIX" />
<param name="position" type="vec3" state="POSITION" />
<param name="texcoord" type="vec2" state="TEXCOORD" />
<param name="iResolution" type="vec2" state="TEXDIM0" />
<param name="iMouse" type="vec4" default="0 0 0 0" />
<param name="iTime" type="float" default="0" />
<param name="dummytex" type="int" default="0" />
<param name="iChannel0" type="int" default="1" />
<param name="iChannel1" type="int" default="2" />
<param name="iChannel2" type="int" default="3" />
<param name="iChannel3" type="int" default="4" />
<param name="ffttex" type="int" default="1" />
<param name="music1" type="float" default="0.0" />
<param name="music2" type="float" default="0.0" />
<language name="glsl" version="1.5">
<bind param="modelViewProjectionMatrix" program="vp" />
<bind param="textureMatrix0" program="vp" />
<bind param="position" program="vp" />
<bind param="texcoord" program="vp" />
<bind param="iTime" program="fp" />
<bind param="iMouse" program="fp" />
<bind param="iResolution" program="fp" />
<bind param="dummytex" program="fp" />
<bind param="iChannel0" program="fp" />
<bind param="iChannel1" program="fp" />
<bind param="iChannel2" program="fp" />
<bind param="iChannel3" program="fp" />
<bind param="music1" program="fp" />
<bind param="music2" program="fp" />
<bind param="ffttex" program="fp" />
<program name="vp" type="vertex" >
<![CDATA[
#version 330 core
in vec3 position;
in vec2 texcoord;
out jit_PerVertex {
vec2 texcoord;
} jit_out;
uniform mat4 modelViewProjectionMatrix;
uniform mat4 textureMatrix0;
void main(void) {
gl_Position = modelViewProjectionMatrix*vec4(position, 1.);
jit_out.texcoord = vec2(textureMatrix0*vec4(texcoord, 0., 1.));
}
]]>
</program>
<program name="fp" type="fragment" >
<![CDATA[
#version 330 core
in jit_PerVertex {
vec2 texcoord;
} jit_in;
out vec4 outColor;
uniform float iTime;
uniform float music1;
uniform float music2;
uniform vec2 iResolution;
uniform vec4 iMouse;
uniform sampler2DRect dummytex;
uniform samplerJit0 tex;
//uniform samplerBuffer iChannel0;
uniform sampler2DRect iChannel0;
uniform sampler2DRect iChannel1;
uniform samplerBuffer ffttex;
//uniform sampler iChannel2;
//uniform sampler iChannel3;
// <<<<<<<<< START SHADER CODE:
vec3 B2_spline(vec3 x) { // returns 3 B-spline functions of degree 2
vec3 t = 3.0 * x;
vec3 b0 = step(0.0, t) * step(0.0, 1.0-t);
vec3 b1 = step(0.0, t-1.0) * step(0.0, 2.0-t);
vec3 b2 = step(0.0, t-2.0) * step(0.0, 3.0-t);
return 0.5 * (
b0 * pow(t, vec3(2.0)) +
b1 * (-2.0*pow(t, vec3(2.0)) + 6.0*t - 3.0) +
b2 * pow(3.0-t,vec3(2.0))
);
}
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
// create pixel coordinates
vec2 uv = fragCoord.xy / iResolution.xy;
float fVBars = 128.;
float fHSpacing = 1.00;
float fHFreq = (uv.x * 3.14);
float squarewave = sign(sin(fHFreq*fVBars)+1.0-fHSpacing);
float x = floor(uv.x * fVBars)/fVBars;
float fSample = texture( iChannel0, vec2(abs(2.0 * x - 1.0), 0.25)).x;
float fft = squarewave * fSample* 0.5;
float fHBars = 100.0;
float fVSpacing = 0.180;
float fVFreq = (uv.y * 3.14);
fVFreq = sign(sin(fVFreq * fHBars)+1.0-fVSpacing);
vec2 centered = vec2(1.0) * uv - vec2(1.0);
float t = iTime / 100.0;
float polychrome = 1.0;
vec3 spline_args = fract(vec3(polychrome*uv.x-t) + vec3(0.0, -1.0/3.0, -2.0/3.0));
vec3 spline = B2_spline(spline_args);
float f = abs(centered.y);
vec3 base_color = vec3(1.0, 1.0, 1.0) - f*spline;
vec3 flame_color = pow(base_color, vec3(3.0));
float tt = 0.3 - uv.y;
float df = sign(tt);
df = (df + 1.0)/0.5;
vec3 col = flame_color * vec3(1.0 - step(fft, abs(0.3-uv.y))) * vec3(fVFreq);
col -= col * df * 0.180;
// output final color
fragColor = vec4(col,1.0);
}
// <<<<<<<<< END SHADER CODE.
void main(void) {
mainImage(outColor, jit_in.texcoord.st);
}
]]>
</program>
</language>
</jittershader>


So close now, another few tweaks need to be made.
The update through a bang is a draw back here, it's jittery when updating using a metro.
If the shader could just grab the texture when needed that would be great, updating through a bang when using shaders seems counter intuitive.
patch so far:
Please god someone take the reigns!
If you find this and want to copy along when you take a shader it may be inverted/upside down you may have to flip the uv's by uv.y = 1 - uv.y; and also the audio texture going into the the slab you need to change the uv lookup and scale by the frag coords, bit hacky, but seems to work, so instead of using the full tex uv's just use a quarter so fragCoord.x/4.
Think I'm 90% there. If anyone wants to chime in please feel free. Just randomly talking to myself here like a mad person.
It's ok Rich you're not going crazy, you're just having a bit of a jitter.
Thanks Rich, means a lot coming from you.
Ok actually I think 94.5394265% there.
I swear I read a forum post saying this couldn't be done. I think it's almost done. You're welcome in advance to any seekers on a similar path. Also balls bags; the lot of you, stop reading this and go do some work now.
Thanks to everyone that helped me.
You're welcome Rich, now get some sleep.
OK BOOM.
I think I've done it. I used catch and graph following this tutorial to try and make the waveform: https://docs.cycling74.com/max7/tutorials/14_analysischapter02
However I can't find the patches in the help>examples, is there another place they're hiding?
Basically catch>graph into a jit martix 512 x 1, then pack in jitter matrix together with the fft texture. I'm hoping they don't just add together on the same 512x1, I think that could be the case though.
Maybe jit.concat could be a better choice? Anyone have any thoughts?
Good thought on the jit.concat to concatenate matrixes Rich, I'll look into that thanks, for a start I guess I have understand what concatenate means, as apparently it has nothing to do with fruit juice.
I think i'm 99.283% there now.
I have FFT and waveform going through as a 512x2 texture almost identically to the way shadertoy's music channel is parsed. I reverse engineered the protocol.

Here it is for the general max community:
Hope this helps someone on the same voyage trying to work out how to input sound into a shadertoy shader from jitter inside max/msp.
I hope this opens up some creative possibilities!
There are some draw backs as noted in this thread, scaling by frag-coords doesn't seem like the best approach. I think the world co-ords are different from shadertoy's.
Also I ported this shader: https://www.shadertoy.com/view/Xds3Rr
if you want to repeat this process, go to Gl3 examples > shader toy loader, and paste in the code in the place where it says paste shader toy code, the errors are texel fetch, just use texture instead, and delete the last variable, so you're only giving a vec4. Also there's no iVec so just change them all to floats and similarly change the casted ints above. Also scale the sound texture by scaling the uv's.
Or you could just use the code I changed below:
Edit:
// Created by inigo quilez - iq/2013
// License Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
//Edited by your handy local ballbag
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
// create pixel coordinates
vec2 uv = (fragCoord.xy) / iResolution.xy;
vec2 uv2 = (fragCoord.xy) / (iResolution.xy/0.5);
uv.y = 1 - uv.y; // Co-ord system is different in max to shader toy??
// the sound texture is 512x2
float tx = float((uv2.x/2.)*(512.0));
// first row is frequency data (48Khz/4 in 512 texels, meaning 23 Hz per texel)
float fft = texture( iChannel0, vec2(tx,0.)).x;
// second row is the sound wave, one texel is one mono sample
float wave = texture( iChannel0, vec2(tx,1.)).x;
// convert frequency to colors
vec3 col = vec3( fft, 4.0*fft*(1.0-fft), 1.0-fft ) * fft;
// add wave form on top
col += 1.0 - smoothstep( 0.0, 0.15, abs(wave - ((uv2.y*4.0)-1.)) );
// output final color
fragColor = vec4(col,1.0);
}
There's probably a few kinks that still need to be ironed out, if you find or fix any bugs any please post revised code/patch.
Done. You're welcome weary traveller.
Hey Richard great job!
Did you do the FFT algorithm yourself or did you find it somewhere?
Is it a standard one or did you tweak it?
Hey thanks Federico! It’s literally the FFT object inside max set to 512 frames, I’m guessing that the frame size is equivalent to fft bins or partials. I haven’t done anything special it was just formatting the fft data inside max to match the way shadertoy packs their sound buffer into a texture. I still haven’t finished, I reckon the uv mapping is off, I’ll hopefully make a patch soon with the fixes and post here. (Should mostly work with a few tweaks!)
If you find some audible shader toy code and get stuck trying to import it, slap it in here and I’ll see if I notice anything that can help.
(Not sure the gl slab code comes across in compress copy, so maybe attach link to the shader you’re trying to port)