[SHARE] Sonographic Sound Processing Diss.

Hi,

Finally the time has come to give something back to this precious forum...

I wrote a dissertation in October 2011 entitled "STFT Analysis Driven Sonographic Sound Processing in Real-Time using Max/MSP and Jitter". I am sharing here the paper and all the accompanying patches (around 40 patches):

https://www.wetransfer.com/dl/3Ytc3XyO/7f7f74b5c25a3c67b02189e6c10c995d7b29ad731318edc256ea7a21844f61d83b8e071e2d74724

And here you can watch an excerpt from the performance I had in February this year (video and especially sound quality are extremely bad). Almost everything beside 3D version of a spectrogram is covered in the dissertation and the patches.

http://www.youtube.com/watch?v=LzBPr3Hc6CA&feature=youtu.be

Table of Contents

Chapter1

Introduction    8
1.1 Summary of problem    9
1.2 Aim of project    9
1.3 Project scope    10
1.4 Methods    10
1.5 Chapter overview    11

Chapter2

Background    13
2.1 Graphical Modifications of Visually Presented Audio Data in the Context of Sound Synthesis    13
2.2 Placing the Analysis Driven Sonographic Sound Processing in Wider Context That Made It Possible    16
2.3    History of Analysis Driven Sonographic Sound Procesing and Overview of Its Most Important Implementations    20
2.4    The Reasons Behind Choosing Max/MSP and Jitter    22

Chapter3

From Fourier Transform to Phase Vocoder and Sonogram    24
3.1 Introduction to the Concepts of Fourier Analysis and Its Offspring    24
3.2 Explaining the Fourier Analysis    27
3.2.1 Fourier Series    27
3.2.2 Fourier Transform    28
3.2.3 The Link Between Complex Numbers, Circular Functions and Exponential Functions    30
3.2.4 Magnitude Detection in Fourier Transform    36
3.2.5 Magnitude Detection in Special Cases    36
3.2.6 Maximum Magnitude Values    39
3.2.7 Fourier Integral of the Incoming Sine Wave    41
3.2.8 Magnitude Detection in Harmonic Signals    43
3.2.9 Phases    45
3.2.10 DFT and FFT    48
3.2.11 STFT    56
3.2.12 Phase Vocoder and Spectrogram in Max/MSP    63
3.2.13 Interacting With Sonogram    70

Chapter4

Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface    78
4.1 Main Patch – Time Domain    79
4.2 Creating the Spectral Matrix – Inside the object pfft~ – Frequency Domain    82
4.3 Reading the Transformed Spectral Matrix – Inside the Object pfft~ – Frequency Domain    84
4.4 Mapping the Display Window to Spectral Matrix    86
4.5 Default Horizontal Speed of Reading the Spectrogram    88

Chapter5 90
Implementation and Critical Evaluation of Existing and Unexisting Real-time ADSSP Techniques in Max/MSP    90
5.1 Optimization    91
5.1.1 Moving Area of ADSSP According to Reading Position    91
5.1.2 Horizontal Blur    94
5.1.3 Frame Interpolation    96
5.1.4 Interpolate – Concatenate – Blur (ICB)    99
5.1.5 Poking Holes in Interpolated Spectrum    101
5.2 Global Transformations    103
5.2.1 Blur/Water Effect    103
5.2.2 Spectral Smear    111
5.2.3 Lock to Grid    112
5.2.4 Spectrum Shift, Scale and Rotation with jit.rota    112
5.2.5 Time Scramble    114
5.2.6 Slice Fade    116
5.2.7 Sound Rider    117
5.2.8 Saturation / Limiter    118
5.2.9 Denoiser    119
5.2.10 Compression    120
5.3 Local Transformations    121
5.3.1 Masks    121
5.3.1.1 Filter Masks    122
5.3.1.2 Applying a Mask to a Spectrum    133
5.3.1.3 Rectangular Masks    137
5.3.1.4 Arbitrary Masks    143
5.3.2 Repositions    146
5.3.2.1 Creating Spatial Maps with Multislider Object    146
5.3.2.2 Creating Spatial Maps with Itable Object    148
5.3.2.2.1 Frequency Warp    152
5.3.2.2.2 Spectral Delay    158
5.3.2.2.3 Spectral Rolling Pin    165
5.4 Spectral Interactions    170
5.4.1 Cross-Synthesis Using Masks    170
5.4.2 Centroid Based Spectral Morphing    174
5.4.3 Indirect Spectral Interaction    176
5.4.4 Target-Based Concatenative Synthesis    177
5.5 FFT Data Processing on Graphic Cards    181

Chapter 6    

Conclusions and Future Work 185
6.1 Future Work    186
References    187

And special thanks to J.F.Charles that made this dissertation possible with his priceless article in Computer Music Journal entitled "A tutorial on spectral sound processing using Max/MSP and Jitter".

Regards,

Tadej Droljc