[SHARE] Sonographic Sound Processing Diss.

Mar 26, 2012 at 11:02pm

[SHARE] Sonographic Sound Processing Diss.

Hi,

Finally the time has come to give something back to this precious forum…

I wrote a dissertation in October 2011 entitled “STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter”. I am sharing here the paper and all the accompanying patches (around 40 patches):

https://www.wetransfer.com/dl/3Ytc3XyO/7f7f74b5c25a3c67b02189e6c10c995d7b29ad731318edc256ea7a21844f61d83b8e071e2d74724

And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

Table of Contents

Chapter1

Introduction 8
1.1 Summary of problem 9
1.2 Aim of project 9
1.3 Project scope 10
1.4 Methods 10
1.5 Chapter overview 11

Chapter2

Background 13
2.1 Graphical Modifications of Visually Presented Audio Data in the Context of Sound Synthesis 13
2.2 Placing the Analysis Driven Sonographic Sound Processing in Wider Context That Made It Possible 16
2.3 History of Analysis Driven Sonographic Sound Procesing and Overview of Its Most Important Implementations 20
2.4 The Reasons Behind Choosing Max/MSP and Jitter 22

Chapter3

From Fourier Transform to Phase Vocoder and Sonogram 24
3.1 Introduction to the Concepts of Fourier Analysis and Its Offspring 24
3.2 Explaining the Fourier Analysis 27
3.2.1 Fourier Series 27
3.2.2 Fourier Transform 28
3.2.3 The Link Between Complex Numbers, Circular Functions and Exponential Functions 30
3.2.4 Magnitude Detection in Fourier Transform 36
3.2.5 Magnitude Detection in Special Cases 36
3.2.6 Maximum Magnitude Values 39
3.2.7 Fourier Integral of the Incoming Sine Wave 41
3.2.8 Magnitude Detection in Harmonic Signals 43
3.2.9 Phases 45
3.2.10 DFT and FFT 48
3.2.11 STFT 56
3.2.12 Phase Vocoder and Spectrogram in Max/MSP 63
3.2.13 Interacting With Sonogram 70

Chapter4

Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface 78
4.1 Main Patch – Time Domain 79
4.2 Creating the Spectral Matrix – Inside the object pfft~ – Frequency Domain 82
4.3 Reading the Transformed Spectral Matrix – Inside the Object pfft~ – Frequency Domain 84
4.4 Mapping the Display Window to Spectral Matrix 86
4.5 Default Horizontal Speed of Reading the Spectrogram 88

Chapter5 90
Implementation and Critical Evaluation of Existing and Unexisting Real-time ADSSP Techniques in Max/MSP 90
5.1 Optimization 91
5.1.1 Moving Area of ADSSP According to Reading Position 91
5.1.2 Horizontal Blur 94
5.1.3 Frame Interpolation 96
5.1.4 Interpolate – Concatenate – Blur (ICB) 99
5.1.5 Poking Holes in Interpolated Spectrum 101
5.2 Global Transformations 103
5.2.1 Blur/Water Effect 103
5.2.2 Spectral Smear 111
5.2.3 Lock to Grid 112
5.2.4 Spectrum Shift, Scale and Rotation with jit.rota 112
5.2.5 Time Scramble 114
5.2.6 Slice Fade 116
5.2.7 Sound Rider 117
5.2.8 Saturation / Limiter 118
5.2.9 Denoiser 119
5.2.10 Compression 120
5.3 Local Transformations 121
5.3.1 Masks 121
5.3.1.1 Filter Masks 122
5.3.1.2 Applying a Mask to a Spectrum 133
5.3.1.3 Rectangular Masks 137
5.3.1.4 Arbitrary Masks 143
5.3.2 Repositions 146
5.3.2.1 Creating Spatial Maps with Multislider Object 146
5.3.2.2 Creating Spatial Maps with Itable Object 148
5.3.2.2.1 Frequency Warp 152
5.3.2.2.2 Spectral Delay 158
5.3.2.2.3 Spectral Rolling Pin 165
5.4 Spectral Interactions 170
5.4.1 Cross-Synthesis Using Masks 170
5.4.2 Centroid Based Spectral Morphing 174
5.4.3 Indirect Spectral Interaction 176
5.4.4 Target-Based Concatenative Synthesis 177
5.5 FFT Data Processing on Graphic Cards 181

Chapter 6

Conclusions and Future Work 185
6.1 Future Work 186
References 187

And special thanks to J.F.Charles that made this dissertation possible with his priceless article in Computer Music Journal entitled “A tutorial on spectral sound processing using Max/MSP and Jitter”.

Regards,

Tadej Droljc

#62615
Mar 27, 2012 at 9:52am

Thanks so much for sharing this, I just had a peek and it looks fascinating.

#226127
Mar 27, 2012 at 10:05am

Thanks for sharing, looks very interesting. unfortunatly opening the word doc I get an erro message ( see attached).
Would you mind posting the papaer as .pdf ?
Thx
Andreas

[attachment=189530,3601]

Attachments:
  1. Worderrormess.tiff
#226128
Mar 27, 2012 at 10:47am

PDF version (without patches):

http://wtrns.fr/aIDtlDr_s1bVCS

#226129
Mar 27, 2012 at 1:17pm

> And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

> http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

great! i’ve been doing related spectrum-to-3d stuff lately. will have an in-depth look at your system. tanx!

#226130
Mar 28, 2012 at 2:24am

This is a fabulous resource– may I suggest you publish it through a print-on-demand website– it would be something good to buy and have on the bookshelf… although I imagine there are probably embargo issues with your university…

#226131
Mar 28, 2012 at 1:32pm

Thank you for your replies! Print-on-demand release would actually be a good idea, but probably I would add a chapter about 3D spectrograms before I do that. Also a poof reading is needed.

Hopefully I will do that soon and in the meantime, please let me know if you find any mistakes or nonsenses…

Cheers!

#226132
Apr 1, 2012 at 5:10am

Thanks t this looks incredible.

#226133
Apr 2, 2012 at 7:09am

What a great work !!
Bravo !

#226134
Apr 18, 2012 at 4:45am

Hi!
It seems that the link expired. Could you please post the files again?
Best regards and many thanks in advance,
Prego

#226135
Apr 20, 2012 at 2:22pm
#226136
Apr 26, 2012 at 1:14am

t,
Thanks for the new link! I have already start to read your dissertation and I am very impressed, congratulations!!!
Best regards,
Prego

#226137
May 19, 2012 at 5:30pm

Hi!
Would you mind uploading the link once more? It is expired again. I am really interested and would really appreciate it.

Thanks
Tina

#226138
May 19, 2012 at 11:11pm
#226139
Nov 2, 2012 at 2:02am

Tadej,
I just finished reading your thesis and going thru all the accompanying Max patch files. Well done and thanks! It has greatly aided my postgraduate research.
TMConnors

#226140
Nov 3, 2012 at 12:16am

Whoa, this is phenomenal. Thanks for sharing!

#226141
Nov 3, 2012 at 10:59pm

Thanks for sharing. It looks really helpful.
Especially because I’m looking into FFT techniques right now.

#226142
Apr 24, 2013 at 5:34pm

Sorry to bring this topic up again, but I moved the paper and the patches to my site, so here is the new link:

http://www.tadej-droljc.org/td-zip/SSP-Dissertation.zip

Also I wrote an article on that subject, that could be found here:

http://designingsound.org/2013/04/sonographic-sound-processing/

#226143
May 11, 2013 at 12:18pm

Great! This is amazing! Thanks for sharing.

#226144
Jun 10, 2013 at 6:11am

Hey T, would you give me a bit of advice on this issue? http://cycling74.com/forums/topic/sonify-matrix-through-pfft-please-clarify/
I’d send this over PM but those are still not activated on the new forum…

#252256
Feb 2, 2014 at 6:55am

Nice work, thanks for sharing :)

#279856
Feb 2, 2014 at 9:10am

Thanks so much. Great work !!!

#279865
Feb 2, 2014 at 9:17am

hi,

I get this error when trying to open the .zip file
Any ideas?

Attachments:
  1. Screenshot-2014-02-02-17.15.24
#279866
Feb 2, 2014 at 9:26am

Its fine now thanks… !!!

#279868

You must be logged in to reply to this topic.