Finally the time has come to give something back to this precious forum...

I wrote a dissertation in October 2011 entitled "STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter". I am sharing here the paper and all the accompanying patches (around 40 patches):

https://www.wetransfer.com/dl/3Ytc3XyO/7f7f74b5c25a3c67b02189e6c10c995d7b29ad731318edc256ea7a21844f61d83b8e071e2d74724

And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

2.1 Graphical Modifications of Visually Presented Audio Data in the Context of Sound Synthesis    13

2.2 Placing the Analysis Driven Sonographic Sound Processing in Wider Context That Made It Possible    16

2.3    History of Analysis Driven Sonographic Sound Procesing and Overview of Its Most Important Implementations    20

2.4    The Reasons Behind Choosing Max/MSP and Jitter    22

From Fourier Transform to Phase Vocoder and Sonogram    24

3.1 Introduction to the Concepts of Fourier Analysis and Its Offspring    24

3.2 Explaining the Fourier Analysis    27

3.2.3 The Link Between Complex Numbers, Circular Functions and Exponential Functions    30

3.2.4 Magnitude Detection in Fourier Transform    36

3.2.5 Magnitude Detection in Special Cases    36

3.2.7 Fourier Integral of the Incoming Sine Wave    41

3.2.8 Magnitude Detection in Harmonic Signals    43

3.2.12 Phase Vocoder and Spectrogram in Max/MSP    63

3.2.13 Interacting With Sonogram    70

Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface    78

4.2 Creating the Spectral Matrix – Inside the object pfft~ – Frequency Domain    82

4.3 Reading the Transformed Spectral Matrix – Inside the Object pfft~ – Frequency Domain    84

4.4 Mapping the Display Window to Spectral Matrix    86

4.5 Default Horizontal Speed of Reading the Spectrogram    88

Implementation and Critical Evaluation of Existing and Unexisting Real-time ADSSP Techniques in Max/MSP    90

5.1.1 Moving Area of ADSSP According to Reading Position    91

5.1.4 Interpolate – Concatenate – Blur (ICB)    99

5.1.5 Poking Holes in Interpolated Spectrum    101

5.2.4 Spectrum Shift, Scale and Rotation with jit.rota    112

5.3.1.2 Applying a Mask to a Spectrum    133

5.3.2.1 Creating Spatial Maps with Multislider Object    146

5.3.2.2 Creating Spatial Maps with Itable Object    148

5.3.2.2.3 Spectral Rolling Pin    165

5.4.1 Cross-Synthesis Using Masks    170

5.4.2 Centroid Based Spectral Morphing    174

5.4.3 Indirect Spectral Interaction    176

5.4.4 Target-Based Concatenative Synthesis    177

5.5 FFT Data Processing on Graphic Cards    181

And special thanks to J.F.Charles that made this dissertation possible with his priceless article in Computer Music Journal entitled "A tutorial on spectral sound processing using Max/MSP and Jitter".

Finally the time has come to give something back to this precious forum... 

Introduction	8
1.1 Summary of problem	9
1.2 Aim of project	9
1.3 Project scope	10
1.4 Methods	10
1.5 Chapter overview	11

Background	13
2.1 Graphical Modifications of Visually Presented Audio Data in the Context of Sound Synthesis	13
2.2 Placing the Analysis Driven Sonographic Sound Processing in Wider Context That Made It Possible	16
2.3	History of Analysis Driven Sonographic Sound Procesing and Overview of Its Most Important Implementations	20
2.4	The Reasons Behind Choosing Max/MSP and Jitter	22

From Fourier Transform to Phase Vocoder and Sonogram	24
3.1  Introduction to the Concepts of Fourier Analysis and Its Offspring	24
3.2  Explaining the Fourier Analysis	27
3.2.1 Fourier Series	27
3.2.2 Fourier Transform	28
3.2.3 The Link Between Complex Numbers, Circular Functions and Exponential        Functions	30
3.2.4   Magnitude Detection in Fourier Transform	36
3.2.5  Magnitude Detection in Special Cases	36
3.2.6  Maximum Magnitude Values	39
3.2.7  Fourier Integral of the Incoming Sine Wave	41
3.2.8  Magnitude Detection in Harmonic Signals	43
3.2.9  Phases	45
3.2.10  DFT and FFT	48
3.2.11  STFT	56
3.2.12  Phase Vocoder and Spectrogram in Max/MSP	63
3.2.13  Interacting With Sonogram	70

Chapter4                                                                     

Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface	78
4.1  Main Patch – Time Domain	79
4.2  Creating the Spectral Matrix – Inside the object pfft~ – Frequency Domain	82
4.3  Reading the Transformed Spectral Matrix – Inside the Object pfft~ – Frequency Domain	84
4.4  Mapping the Display Window to Spectral Matrix	86
4.5  Default Horizontal Speed of Reading the Spectrogram	88

Chapter5                                                                                                                                      90
Implementation and Critical Evaluation of Existing and Unexisting Real-time ADSSP Techniques in Max/MSP	90
5.1  Optimization	91
5.1.1  Moving Area of ADSSP According to Reading Position	91
5.1.2  Horizontal Blur	94
5.1.3  Frame Interpolation	96
5.1.4  Interpolate – Concatenate – Blur (ICB)	99
5.1.5  Poking Holes in Interpolated Spectrum	101
5.2  Global Transformations	103
5.2.1  Blur/Water Effect	103
5.2.2  Spectral Smear	111
5.2.3  Lock to Grid	112
5.2.4  Spectrum Shift, Scale and Rotation with jit.rota	112
5.2.5  Time Scramble	114
5.2.6  Slice Fade	116
5.2.7  Sound Rider	117
5.2.8  Saturation / Limiter	118
5.2.9 Denoiser	119
5.2.10  Compression	120
5.3  Local Transformations	121
5.3.1  Masks	121
5.3.1.1  Filter Masks	122
5.3.1.2  Applying a Mask to a Spectrum	133
5.3.1.3  Rectangular Masks	137
5.3.1.4  Arbitrary Masks	143
5.3.2 Repositions	146
5.3.2.1 Creating Spatial Maps with Multislider Object	146
5.3.2.2 Creating Spatial Maps with Itable Object	148
5.3.2.2.1 Frequency Warp	152
5.3.2.2.2 Spectral Delay	158
5.3.2.2.3 Spectral Rolling Pin	165
5.4  Spectral Interactions	170
5.4.1  Cross-Synthesis Using Masks	170
5.4.2 Centroid Based Spectral Morphing	174
5.4.3 Indirect Spectral Interaction	176
5.4.4 Target-Based Concatenative Synthesis	177
5.5 FFT Data Processing on Graphic Cards	181

Conclusions and Future Work                185
6.1 Future Work	186
References	187

share-sonographic-sound-processing-diss

Thanks so much for sharing this, I just had a peek and it looks fascinating.

Thanks for sharing, looks very interesting. unfortunatly opening the word doc I get an erro message ( see attached).

Would you mind posting the papaer as .pdf ?

Thanks for sharing, looks very interesting. unfortunatly opening the word doc I get an erro message ( see attached).
Would you mind posting the papaer as .pdf ?
Thx
Andreas

> And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

great! i've been doing related spectrum-to-3d stuff lately. will have an in-depth look at your system. tanx!

> http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

great! i've been doing related spectrum-to-3d stuff lately. will have an in-depth look at your system. tanx!


This is a fabulous resource-- may I suggest you publish it through a print-on-demand website-- it would be something good to buy and have on the bookshelf... although I imagine there are probably embargo issues with your university...

This is a fabulous resource-- may I  suggest you publish it through a print-on-demand website-- it would be something good to buy and have on the bookshelf... although I imagine there are probably embargo issues with your university...


Thank you for your replies! Print-on-demand release would actually be a good idea, but probably I would add a chapter about 3D spectrograms before I do that. Also a poof reading is needed.

Hopefully I will do that soon and in the meantime, please let me know if you find any mistakes or nonsenses...

It seems that the link expired. Could you please post the files again?

Hi!
It seems that the link expired. Could you please post the files again?
Best regards and many thanks in advance,
Prego


Thanks a lot for your feedback! Here is the link again:

https://www.wetransfer.com/dl/d9W1fhba/0617f92e9df20e474407b5c94b1310822897a58db6d630de7094ac91121ac0d8a64ab7573248a03

Thanks for the new link! I have already start to read your dissertation and I am very impressed, congratulations!!!

t,
Thanks for the new link! I have already start to read your dissertation and I am very impressed, congratulations!!!
Best regards,
Prego


Would you mind uploading the link once more? It is expired again. I am really interested and would really appreciate it.

Hi!
Would you mind uploading the link once more? It is expired again. I am really interested and would really appreciate it.

I just finished reading your thesis and going thru all the accompanying Max patch files. Well done and thanks! It has greatly aided my postgraduate research.

Tadej,
I just finished reading your thesis and going thru all the accompanying Max patch files. Well done and thanks! It has greatly aided my postgraduate research.
TMConnors


Whoa, this is phenomenal. Thanks for sharing!

Thanks for sharing. It looks really helpful.

Especially because I'm looking into FFT techniques right now.

Thanks for sharing. It looks really helpful.
Especially because I'm looking into FFT techniques right now.


Sorry to bring this topic up again, but I moved the paper and the patches to my site, so here is the new link:

http://www.tadej-droljc.org/td-zip/SSP-Dissertation.zip

Also I wrote an article on that subject, that could be found here:

http://designingsound.org/2013/04/sonographic-sound-processing/

Great! This is amazing! Thanks for sharing.

Hey T, would you give me a bit of advice on this issue? 

https://cycling74.com/forums/sonify-matrix-through-pfft-please-clarify/

I'd send this over PM but those are still not activated on the new forum...

Hey T, would you give me a bit of advice on this issue? https://cycling74.com/forums/sonify-matrix-through-pfft-please-clarify/
I'd send this over PM but those are still not activated on the new forum...


I get this error when trying to open the .zip file

I get this error when trying to open the .zip file
Any ideas?


Thanks so much for sharing this dude. Very interested and extremely helpful. Great work!

Seems like a really cool project! Any way you could repost again? It appears neither the paper nor the patches are downloadable at this point

You can d/l from Tadej' website. It really is a fantastic piece of work.

http://www.tadej-droljc.org/portfolio/sonographic-sound-processing-article/gallery/max-related-articles/

Can I get shared too? all of the links above doesn't work, probably because it's 2019...

McDonald’s has organized an online survey known as McDonald’s Survey Canada at its official website McDVoice. Here, at McDonald’s Customer.

I just discovered this amazing piece of work, thank you Tadej for sharing! And inspiring! I am just going through your thesis as writing this.

I've been trying to access the website and patches, but without any luck.

Any chance someone here has those patches and is keen to share them on a valid link?

I just discovered this amazing piece of work, thank you Tadej for sharing! And inspiring! I am just going through your thesis as writing this.
I've been trying to access the website and patches, but without any luck.
Any chance someone here has those patches and is keen to share them on a valid link?

Thanks you so much for sharing this. Its Very 

Thanks you so much for sharing this. Its Very interested and extremely helpful. Nice work!

http://gofile.me/2a2kr/bQaLysaiD will be valid for one week

I am delighted to announce that I have a valuable contribution to make to this esteemed community. Back in October 2011, I completed a dissertation titled "STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter." Today, I am thrilled to share not only the dissertation itself but also approximately 40 accompanying patches that were developed during the research process 

I am delighted to announce that I have a valuable contribution to make to this esteemed community. Back in October 2011, I completed a dissertation titled "STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter." Today, I am thrilled to share not only the dissertation itself but also approximately 40 accompanying patches that were developed during the research process click here 

this is amazing! but is there anyway i could access these links again? could anybody share them?

https://web.archive.org/web/20160531055319/http://www.tadej-droljc.org/td-zip/SSP-Dissertation.zip

[SHARE] Sonographic Sound Processing Diss.