Convert an image into music

Blake's icon

I want to create a patch that will allow me to accurately generate music based on an image, and when the music is played in a sonograph the spectrogram will reveal the image. I have never really delved too deep into using FFT and STFT beyond the MSP tutorials, and have absolutely no experience using FFT in Jitter. I'm sure there are many people here who have done this type of thing before, so I was wondering if anybody would mind giving me a quick rundown of what I need to do, point me in the right direction, suggest good resources to use etc.

I really appreciate anyones help, and will share the patch once it's complete.

Here is an example of Venetian Snares - "Look" when played through a sonograph, Aphex Twin did something similar with the track "Mathematical Equation" from Windowlicker.

Giorgio's icon

I think you should look for metasynth. I guess that Richard did it that way.

lewis lepton's icon

it was right at the end, when 'windowlicker' is teetering out, it does this squelchy, squealing sound. that was also done with metasynth.

great sound...

Blake's icon

Hi guys,

I'm still trying to do this, for the purposes of learning. Can you use jit.buffer in a pfft~, and use the jit.buffer to compute as audio, maybe through converting a char matrix to float32?

div's icon

Hey Blake,

Any luck with this?

I had a fiddle around but can't seem to get it working.

Tim

dal_kas's icon

Hi Guys

This might be a start - one of Baz's youtube patch-a-days: