[SHARE] Sonographic Sound Processing Diss.

Mar 26, 2012 at 11:02pm

[SHARE] Sonographic Sound Processing Diss.


Finally the time has come to give something back to this precious forum…

I wrote a dissertation in October 2011 entitled “STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter”. I am sharing here the paper and all the accompanying patches (around 40 patches):


And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.


Table of Contents


Introduction 8
1.1 Summary of problem 9
1.2 Aim of project 9
1.3 Project scope 10
1.4 Methods 10
1.5 Chapter overview 11


Background 13
2.1 Graphical Modifications of Visually Presented Audio Data in the Context of Sound Synthesis 13
2.2 Placing the Analysis Driven Sonographic Sound Processing in Wider Context That Made It Possible 16
2.3 History of Analysis Driven Sonographic Sound Procesing and Overview of Its Most Important Implementations 20
2.4 The Reasons Behind Choosing Max/MSP and Jitter 22


From Fourier Transform to Phase Vocoder and Sonogram 24
3.1 Introduction to the Concepts of Fourier Analysis and Its Offspring 24
3.2 Explaining the Fourier Analysis 27
3.2.1 Fourier Series 27
3.2.2 Fourier Transform 28
3.2.3 The Link Between Complex Numbers, Circular Functions and Exponential Functions 30
3.2.4 Magnitude Detection in Fourier Transform 36
3.2.5 Magnitude Detection in Special Cases 36
3.2.6 Maximum Magnitude Values 39
3.2.7 Fourier Integral of the Incoming Sine Wave 41
3.2.8 Magnitude Detection in Harmonic Signals 43
3.2.9 Phases 45
3.2.10 DFT and FFT 48
3.2.11 STFT 56
3.2.12 Phase Vocoder and Spectrogram in Max/MSP 63
3.2.13 Interacting With Sonogram 70


Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface 78
4.1 Main Patch – Time Domain 79
4.2 Creating the Spectral Matrix – Inside the object pfft~ – Frequency Domain 82
4.3 Reading the Transformed Spectral Matrix – Inside the Object pfft~ – Frequency Domain 84
4.4 Mapping the Display Window to Spectral Matrix 86
4.5 Default Horizontal Speed of Reading the Spectrogram 88

Chapter5 90
Implementation and Critical Evaluation of Existing and Unexisting Real-time ADSSP Techniques in Max/MSP 90
5.1 Optimization 91
5.1.1 Moving Area of ADSSP According to Reading Position 91
5.1.2 Horizontal Blur 94
5.1.3 Frame Interpolation 96
5.1.4 Interpolate – Concatenate – Blur (ICB) 99
5.1.5 Poking Holes in Interpolated Spectrum 101
5.2 Global Transformations 103
5.2.1 Blur/Water Effect 103
5.2.2 Spectral Smear 111
5.2.3 Lock to Grid 112
5.2.4 Spectrum Shift, Scale and Rotation with jit.rota 112
5.2.5 Time Scramble 114
5.2.6 Slice Fade 116
5.2.7 Sound Rider 117
5.2.8 Saturation / Limiter 118
5.2.9 Denoiser 119
5.2.10 Compression 120
5.3 Local Transformations 121
5.3.1 Masks 121 Filter Masks 122 Applying a Mask to a Spectrum 133 Rectangular Masks 137 Arbitrary Masks 143
5.3.2 Repositions 146 Creating Spatial Maps with Multislider Object 146 Creating Spatial Maps with Itable Object 148 Frequency Warp 152 Spectral Delay 158 Spectral Rolling Pin 165
5.4 Spectral Interactions 170
5.4.1 Cross-Synthesis Using Masks 170
5.4.2 Centroid Based Spectral Morphing 174
5.4.3 Indirect Spectral Interaction 176
5.4.4 Target-Based Concatenative Synthesis 177
5.5 FFT Data Processing on Graphic Cards 181

Chapter 6

Conclusions and Future Work 185
6.1 Future Work 186
References 187

And special thanks to J.F.Charles that made this dissertation possible with his priceless article in Computer Music Journal entitled “A tutorial on spectral sound processing using Max/MSP and Jitter”.


Tadej Droljc

Mar 27, 2012 at 9:52am

Thanks so much for sharing this, I just had a peek and it looks fascinating.

Mar 27, 2012 at 10:05am

Thanks for sharing, looks very interesting. unfortunatly opening the word doc I get an erro message ( see attached).
Would you mind posting the papaer as .pdf ?


  1. Worderrormess.tiff
Mar 27, 2012 at 10:47am

PDF version (without patches):


Mar 27, 2012 at 1:17pm

> And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

> http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

great! i’ve been doing related spectrum-to-3d stuff lately. will have an in-depth look at your system. tanx!

Mar 28, 2012 at 2:24am

This is a fabulous resource– may I suggest you publish it through a print-on-demand website– it would be something good to buy and have on the bookshelf… although I imagine there are probably embargo issues with your university…

Mar 28, 2012 at 1:32pm

Thank you for your replies! Print-on-demand release would actually be a good idea, but probably I would add a chapter about 3D spectrograms before I do that. Also a poof reading is needed.

Hopefully I will do that soon and in the meantime, please let me know if you find any mistakes or nonsenses…


Apr 1, 2012 at 5:10am

Thanks t this looks incredible.

Apr 2, 2012 at 7:09am

What a great work !!
Bravo !

Apr 18, 2012 at 4:45am

It seems that the link expired. Could you please post the files again?
Best regards and many thanks in advance,

Apr 20, 2012 at 2:22pm
Apr 26, 2012 at 1:14am

Thanks for the new link! I have already start to read your dissertation and I am very impressed, congratulations!!!
Best regards,

May 19, 2012 at 5:30pm

Would you mind uploading the link once more? It is expired again. I am really interested and would really appreciate it.


May 19, 2012 at 11:11pm
Nov 2, 2012 at 2:02am

I just finished reading your thesis and going thru all the accompanying Max patch files. Well done and thanks! It has greatly aided my postgraduate research.

Nov 3, 2012 at 12:16am

Whoa, this is phenomenal. Thanks for sharing!

Nov 3, 2012 at 10:59pm

Thanks for sharing. It looks really helpful.
Especially because I’m looking into FFT techniques right now.

Apr 24, 2013 at 5:34pm

Sorry to bring this topic up again, but I moved the paper and the patches to my site, so here is the new link:


Also I wrote an article on that subject, that could be found here:


May 11, 2013 at 12:18pm

Great! This is amazing! Thanks for sharing.

Jun 10, 2013 at 6:11am

Hey T, would you give me a bit of advice on this issue? http://cycling74.com/forums/topic/sonify-matrix-through-pfft-please-clarify/
I’d send this over PM but those are still not activated on the new forum…

Feb 2, 2014 at 6:55am

Nice work, thanks for sharing :)

Feb 2, 2014 at 9:10am

Thanks so much. Great work !!!

Feb 2, 2014 at 9:17am


I get this error when trying to open the .zip file
Any ideas?

  1. Screenshot-2014-02-02-17.15.24
Feb 2, 2014 at 9:26am

Its fine now thanks… !!!


You must be logged in to reply to this topic.