Forums > Jitter

[SHARE] Sonographic Sound Processing Diss.


t
March 26, 2012 | 11:02 pm

Hi,

Finally the time has come to give something back to this precious forum…

I wrote a dissertation in October 2011 entitled "STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter". I am sharing here the paper and all the accompanying patches (around 40 patches):

https://www.wetransfer.com/dl/3Ytc3XyO/7f7f74b5c25a3c67b02189e6c10c995d7b29ad731318edc256ea7a21844f61d83b8e071e2d74724

And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

Table of Contents

Chapter1

Introduction 8
1.1 Summary of problem 9
1.2 Aim of project 9
1.3 Project scope 10
1.4 Methods 10
1.5 Chapter overview 11

Chapter2

Background 13
2.1 Graphical Modifications of Visually Presented Audio Data in the Context of Sound Synthesis 13
2.2 Placing the Analysis Driven Sonographic Sound Processing in Wider Context That Made It Possible 16
2.3 History of Analysis Driven Sonographic Sound Procesing and Overview of Its Most Important Implementations 20
2.4 The Reasons Behind Choosing Max/MSP and Jitter 22

Chapter3

From Fourier Transform to Phase Vocoder and Sonogram 24
3.1 Introduction to the Concepts of Fourier Analysis and Its Offspring 24
3.2 Explaining the Fourier Analysis 27
3.2.1 Fourier Series 27
3.2.2 Fourier Transform 28
3.2.3 The Link Between Complex Numbers, Circular Functions and Exponential Functions 30
3.2.4 Magnitude Detection in Fourier Transform 36
3.2.5 Magnitude Detection in Special Cases 36
3.2.6 Maximum Magnitude Values 39
3.2.7 Fourier Integral of the Incoming Sine Wave 41
3.2.8 Magnitude Detection in Harmonic Signals 43
3.2.9 Phases 45
3.2.10 DFT and FFT 48
3.2.11 STFT 56
3.2.12 Phase Vocoder and Spectrogram in Max/MSP 63
3.2.13 Interacting With Sonogram 70

Chapter4

Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface 78
4.1 Main Patch – Time Domain 79
4.2 Creating the Spectral Matrix – Inside the object pfft~ – Frequency Domain 82
4.3 Reading the Transformed Spectral Matrix – Inside the Object pfft~ – Frequency Domain 84
4.4 Mapping the Display Window to Spectral Matrix 86
4.5 Default Horizontal Speed of Reading the Spectrogram 88

Chapter5 90
Implementation and Critical Evaluation of Existing and Unexisting Real-time ADSSP Techniques in Max/MSP 90
5.1 Optimization 91
5.1.1 Moving Area of ADSSP According to Reading Position 91
5.1.2 Horizontal Blur 94
5.1.3 Frame Interpolation 96
5.1.4 Interpolate – Concatenate – Blur (ICB) 99
5.1.5 Poking Holes in Interpolated Spectrum 101
5.2 Global Transformations 103
5.2.1 Blur/Water Effect 103
5.2.2 Spectral Smear 111
5.2.3 Lock to Grid 112
5.2.4 Spectrum Shift, Scale and Rotation with jit.rota 112
5.2.5 Time Scramble 114
5.2.6 Slice Fade 116
5.2.7 Sound Rider 117
5.2.8 Saturation / Limiter 118
5.2.9 Denoiser 119
5.2.10 Compression 120
5.3 Local Transformations 121
5.3.1 Masks 121
5.3.1.1 Filter Masks 122
5.3.1.2 Applying a Mask to a Spectrum 133
5.3.1.3 Rectangular Masks 137
5.3.1.4 Arbitrary Masks 143
5.3.2 Repositions 146
5.3.2.1 Creating Spatial Maps with Multislider Object 146
5.3.2.2 Creating Spatial Maps with Itable Object 148
5.3.2.2.1 Frequency Warp 152
5.3.2.2.2 Spectral Delay 158
5.3.2.2.3 Spectral Rolling Pin 165
5.4 Spectral Interactions 170
5.4.1 Cross-Synthesis Using Masks 170
5.4.2 Centroid Based Spectral Morphing 174
5.4.3 Indirect Spectral Interaction 176
5.4.4 Target-Based Concatenative Synthesis 177
5.5 FFT Data Processing on Graphic Cards 181

Chapter 6

Conclusions and Future Work 185
6.1 Future Work 186
References 187

And special thanks to J.F.Charles that made this dissertation possible with his priceless article in Computer Music Journal entitled "A tutorial on spectral sound processing using Max/MSP and Jitter".

Regards,

Tadej Droljc


March 27, 2012 | 9:52 am

Thanks so much for sharing this, I just had a peek and it looks fascinating.


March 27, 2012 | 10:05 am

Thanks for sharing, looks very interesting. unfortunatly opening the word doc I get an erro message ( see attached).
Would you mind posting the papaer as .pdf ?
Thx
Andreas

[attachment=189530,3601]

Attachments:
  1. Worderrormess.tiff


t
March 27, 2012 | 10:47 am

PDF version (without patches):

http://wtrns.fr/aIDtlDr_s1bVCS



dtr
March 27, 2012 | 1:17 pm

> And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

> http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

great! i’ve been doing related spectrum-to-3d stuff lately. will have an in-depth look at your system. tanx!


March 28, 2012 | 2:24 am

This is a fabulous resource– may I suggest you publish it through a print-on-demand website– it would be something good to buy and have on the bookshelf… although I imagine there are probably embargo issues with your university…



t
March 28, 2012 | 1:32 pm

Thank you for your replies! Print-on-demand release would actually be a good idea, but probably I would add a chapter about 3D spectrograms before I do that. Also a poof reading is needed.

Hopefully I will do that soon and in the meantime, please let me know if you find any mistakes or nonsenses…

Cheers!


April 1, 2012 | 5:10 am

Thanks t this looks incredible.


April 2, 2012 | 7:09 am

What a great work !!
Bravo !


April 18, 2012 | 4:45 am

Hi!
It seems that the link expired. Could you please post the files again?
Best regards and many thanks in advance,
Prego



t
April 20, 2012 | 2:22 pm

Thanks a lot for your feedback! Here is the link again:

https://www.wetransfer.com/dl/d9W1fhba/0617f92e9df20e474407b5c94b1310822897a58db6d630de7094ac91121ac0d8a64ab7573248a03


April 26, 2012 | 1:14 am

t,
Thanks for the new link! I have already start to read your dissertation and I am very impressed, congratulations!!!
Best regards,
Prego


May 19, 2012 | 5:30 pm

Hi!
Would you mind uploading the link once more? It is expired again. I am really interested and would really appreciate it.

Thanks
Tina



t
May 19, 2012 | 11:11 pm

here you go:

http://www.sendspace.com/file/m90wr3

enjoy:)


November 2, 2012 | 2:02 am

Tadej,
I just finished reading your thesis and going thru all the accompanying Max patch files. Well done and thanks! It has greatly aided my postgraduate research.
TMConnors


November 3, 2012 | 12:16 am

Whoa, this is phenomenal. Thanks for sharing!


November 3, 2012 | 10:59 pm

Thanks for sharing. It looks really helpful.
Especially because I’m looking into FFT techniques right now.



t
April 24, 2013 | 5:34 pm

Sorry to bring this topic up again, but I moved the paper and the patches to my site, so here is the new link:

http://www.tadej-droljc.org/td-zip/SSP-Dissertation.zip

Also I wrote an article on that subject, that could be found here:

http://designingsound.org/2013/04/sonographic-sound-processing/


May 11, 2013 | 12:18 pm

Great! This is amazing! Thanks for sharing.



dtr
June 10, 2013 | 6:11 am

Hey T, would you give me a bit of advice on this issue? http://cycling74.com/forums/topic/sonify-matrix-through-pfft-please-clarify/
I’d send this over PM but those are still not activated on the new forum…


February 2, 2014 | 6:55 am

Nice work, thanks for sharing :)


February 2, 2014 | 9:10 am

Thanks so much. Great work !!!


February 2, 2014 | 9:17 am

hi,

I get this error when trying to open the .zip file
Any ideas?

Attachments:
  1. Screenshot-2014-02-02-17.15.24

February 2, 2014 | 9:26 am

Its fine now thanks… !!!


Viewing 24 posts - 1 through 24 (of 24 total)