wannabeafrog

I've been posting a lot lately circling around an idea I've been trying to develop, finally I think I've found the correct words and hopefully this time I will find the right answers.

I'm interested in finding methods to translate light spectra into tristimulus values within max/msp, the idea is to process real time spectral information to output colors. Does anyone knows any works that have done anything like this before with Max? Do I have to go through all the steps spectra->tristimulus values->color pixel? or are there any objects or works that can help me skip one of the steps?

light-spectral-information-into-color-synthesis

Don't take this the wrong way but Max is a programming environment... so program it dude! If you want some other thing to do it for you there is plenty of software out there. The good news is that what you are talking about is pretty straightforward. If I undersand you correctly, you want to convert intensity/wavelength data into a renderable color format for your computer monitor or lighting equipment, right?

The best way to handle this is to first convert spectral data to XYZ data.

 The XYZ system was specifically designed for this sort of thing. It all comes down to defining the reflective context of those given spectral values. Considering that there are lots of things that can reflect light, this value is usually generalized as a fixed reference known as an illuminant.

wavelength values of visible light have already been converted to XYZ and are available in tabular format.

 In other words.. for every wavelength value, there is a corresponding X, Y and Z value.

Ideally, you want to sample your spectral data in at least 5-10nm steps. Here is a link that will output a csv file for all visible and some nonvisible wavelengths and the corresponding XYZ values to .1nm resolution.

All you need to do is load that data into tables, colls or similar Max objects so that you can input wavelengths and output XYZ values. It's very straightforward. If you don't know how to do that, search in the tutorials.. or ask for specific help in getting that data into a table or coll.

Next step is to specify a color space and convert those XYZ values. Keep it simple and convert to sRGB..

 which will work with monitors, lighting equipment, cameras, etc. I recently made some abstractions to calcuate RGB to XYZ but not the other way around... But it's not difficult. It's a simple linear conversion followed by nonlinear companding (for uniformity). Numbers in, numbers out.

If you start working on the spectral to XYZ conversion, I will make an abstraction that converts XYZ to sRGB (and then Roman Thilenius will probably come around and make a better one).

So that's how I would generate RGB values from spectrographic data in real time. XYZ/RGB can also easily be converted to hue, saturation, luminance, chroma, intensity and so on. I don't recommend using jit.colorspace as it seems more geared for video work, and I have found some of the output values to be grossly incorrect in several of the conversion formats. Stick with good old mathematical expressions and tables, collections, sliders, number boxes, etc.

Whatever you decide to do.. good luck with your project!

Don't take this the wrong way but Max is a programming environment... so program it dude!  If you want some other thing to do it for you there is plenty of software out there.  The good news is that what you are talking about is pretty straightforward.  If I undersand you correctly, you want to convert intensity/wavelength data into a renderable color format for your computer monitor or lighting equipment, right?

The best way to handle this is to first convert spectral data to XYZ data.  The XYZ system was specifically designed for this sort of thing.  It all comes down to defining the reflective context of those given spectral values. Considering that there are lots of things that can reflect light, this value is usually generalized as a fixed reference known as an illuminant.  

The good news is that wavelength values of visible light have already been converted to XYZ and are available in tabular format.  In other words.. for every wavelength value, there is a corresponding X, Y and Z value.

Ideally, you want to sample your spectral data in at least 5-10nm steps.   Here is a link that will output a csv file for all visible and some nonvisible wavelengths and the corresponding XYZ values to .1nm resolution.

All you need to do is load that data into tables, colls or similar Max objects so that you can input wavelengths and output XYZ values.  It's very straightforward. If you don't know how to do that, search in the tutorials.. or ask for specific help in getting that data into a table or coll.

Next step is to specify a color space and convert those XYZ values.  Keep it simple and convert to sRGB.. which will work with monitors, lighting equipment, cameras, etc.  I recently made some abstractions to calcuate RGB to XYZ but not the other way around...  But it's not difficult.  It's a simple linear conversion followed by nonlinear companding (for uniformity).   Numbers in, numbers out.   

If you start working on the spectral to XYZ conversion, I will make an abstraction that converts XYZ to sRGB (and then Roman Thilenius will probably come around and make a better one).  

So that's how I would generate RGB values from spectrographic data in real time.   XYZ/RGB can also easily be converted to hue, saturation, luminance, chroma, intensity and so on.    I don't recommend using jit.colorspace as it seems more geared for video work, and I have found some of the output values to be grossly incorrect in several of the conversion formats.  Stick with good old mathematical expressions and tables, collections, sliders, number boxes, etc.  

Whatever you decide to do.. good luck with your project!


Thank you so much again for your time and interest. Your advice has been most useful and enlightening.

I already started the spectral to XYZ conversion, but I'm stuck with the list step since the csv file I download comes with an odd format that [coll] doesn't recognize. I'm now trying to solve it through XML external objects but so far I haven't been able to recall the data properly, do you have any ideas to solve this? Im working with 1nm steps so manually reformatting the csv is not a good option.

On another subject, do you have such XYZ to SRGB at hand? I would like to do some concept testing with a small sample list.

Open the csv file in excel (or any program that can display the csv file as a table). Copy a single column and paste into a .txt file. Use the included [p load-a-coll] to load your colls with the contents of the .txt file. Be sure to select 'save data with patcher' in the coll inspector.

The included subpatch will also load a coll with multiple columns if you save the csv as a 

 file (all columns separated by spaces, not commas) and then change the extension back to .txt.

XYZ to sRGB conversion patch [p xyz2rgb] is included.

109136.3oM6cusbbckkte+Z0OEYv1QrsKqhUNGi4XNly9Ju21Nr8s16Hb2UW

Un.hDRD6FDfa.vRR8N76jeF7SlyC.T7zevwm3AQohczkjvg7CYNVe4JW472J

y7+1+vW8fu8xe7zqevl+oM+qa9pu5+1+vW8UG9V6+Fe0se8W8fmdxO9nyO45

C+ZO3QW9zmd5E27fu93O6lS+waN78exM27r+o+ze5G9ge3ge6Ie6om+nKO+x

qd3te6+ziu7Gt37KO4w+o+iat5z+1Ym9Cat7617+4+a+m1b3WYy0O6jGc50O

7YO96tKyu6pu+a2m41Gt81uyEO+omcw4mdygqBsa+lO6jadzSN6hu+at5zGc

ywaB803gV70aZssOb2+xm4tH1X6BZye81K0YO9vU2K+1+K+w1Z6K9Sd4E2bw

IO8zC+r+iWc1Im+fe9O8kO+l69au8k98u9r+8C+9M6ga2+c++4e3eX++3qeO

GjO84meyYWe9YO9zq1b10at9Qmb9oOdyMWt4o6u8t4lmb5lyO6omcy06Gi6+

pS91K+amt4rmdx2e5GfI3199QVeqeX.1l26.bN+PM.u6ZzUmd8tgxI2b1kW7

xWq7sGuVc3ec2+Xye8C3X+lmraX+Cmb8lu+pS91uc2D+6t5xmt4jMe+kW98m

e63cy0mdxUO5IO7ga9fMn8wZ+f1F2eSclepFz4Cy8a3Sa+UqOJS5WZOEO5uc

04O7xq99+zidz0O7I27z2+w5nu+13lw119q+sXb3lCOWGeZlqSqu+5yvGezJ

v+mext+x62m5465wmc94a1ck9YO+lC6hX+NQN4pM+y+K+4c+BWd0iO6hSt4z

q273y1eM77eZye73NW1MJO7q+2N47me5K1+x+5o+3yt5utY27Z2u70aN6hMW

eyoOai8v6tJb9YWb5it74Wb35g+duILxCaBige3tD4g+UOgsfi3SzVvsiCaA

2t889NFWb5Or6p9arA7Or6gslObtc1V4CdayLimY97vi6Y1gqcscs+6auIwR

clc2lqieqa9omc5w+tO3618H66tN7W+v83hvzYewdy+92b3nEdf59Zy0KsSg

b9v6aWBg7AEzogyC95W5+gyn2Qu6vNvZd6Uqc2dgt9j+1oO9aNdWyu4jat4p

y91meywCn6qdw78qdvoOc2incn6e3abXKxGxsK+z601kHGE1tzWeVscocX2R

sseVuc4Ge+1tDG1MRd+GCXe9Y01kCGGU9ZOL6m9sJziR+G9C+gcOP8l+4udy

+xlSt3wa9y28fsmb0o2cr9We4tG28jiOx8462jcxEO5z6dz3Ge10O67S9oM+

vSN6lS+iO6xyt3l8OMg1tGWZy+26+d6+p8+h6t5b1Se9SOl+9K8omr6oOb74

b8nmbxEWb54ObylCWY1cXA6d9EGeJYW72N8pq2MR2Gy2dx06t5r6+ded+uLh

M6NrhiWetY+e+cw8jcGv7gqIaNdM4kO5hc+IWwC21yu9v0tsaOb+kcOB2z28

Gdy+6mt6V0WeH58WAt+wvsGKyc2lNbzJmd71I7m+sezIw68QmXwgcHGYt+N9

c63cQ5vcNZ1mliNwFwgGga9w63K2sQ6vFlM+y6pt+4M6aLu7neWw9C2ye+ti

iYduGGS7I5P+FGlmlu9nMa+C+gM29rhN8hG9Cm8uc1yN8wmcxgmYz9u5O8+7

+G+u9Msk29lC2883N0+G2+M+we5adzS18DU2cU9QmcyO8M6tT6d9qO8a1sqk

uY2cN9li+N+Ku7k6AentKvHN7TWG81wsYvVI8G29W1VoC+F6ZNe71JseQrNt

bUad7oGVClc6k5g27i2r46Na28K1cGhSN77udvGfmBz53S543pBzV26yhs+I

ZwAr73Shc0+nMh6O71GB5la2MysOtz+8We5tGX6hqO6w6d3gm+sGFYa9tcOR

1iO8lSN67q+e3C39dF8iK808uzA8OQqbvwV838egufiS7Ya1Wq+im7Ge4l6q

Ly19NOPwa6o2+dq68OzCrCGDvUu9Qus+9h2dLLuzV7u5Am7rm8Re6u5ktH6G

T+Wt7PPiu9Eeqyt332p8hu09Uu9tKu+hu69UDb2dBdzMO+pi2h9w43AG+g61

37Ob2E8muSzw6SOiiG3vwkaMrednsaC42e9kO5e61iA8tu4kO6zKN6hWtk7J

+3Ge52cxyO+lu4sOrd0e92s+gAnK7acKyW8fu+pyd7kWr+JwqbI2+su6O2t8

Ae3F09+4Oei4vuwEm7r2xE95c2Nd90e6IWset9sme5KsHG6ZnWd44u5O5EaJ

2UzO4hy18fdmdyYGuxZaeQnm8zmc0YGOZuW78N8hS1kwSt9QWsuj+xQc7m72

dK+jGuaq8iN8GN6w27jWYGJ6+I690O6Y2sI3AuXF83y99Su9lW86cyIe+0u5

24UHWd4Z3KeO3W46eu677Uu67Oc4y27nStXySu7wm8c+zlCqp8w8VtaOo2b0

IWb82c5UGehDmbyI+7J.t6mt6hd0lc6kX2WsuC73COV1tiTe2cxdz+19i.+v

E6+vUmtKg+Cad5oWe8OKN7lGos8R+f2Xmx3NloczzVG12hm2tu4iek+R8rWa

uMyW9JFWqwc5bO634dW+7kc7Z1aYUB+4mK4Wc29s+Pu0+ju6lcac2uC86VJ2

WrIe+Fyudyiu746p6adz4mseS5tmj2Me8lqO87cW+2bx9e9it7Y+zgmW5yN4

58OUxK3dwm3s7Gmn1wi8391x6+Jsk+Xmb+wps8izV9at76+9yO8A+BGj1Zdj

Dwt6Qoe0Gq90miVmGUu5epcOgiS2069li6K80Gkuwp7bXmyeHmKukcE8h6Qb

bsbnQlUajc6xgcXjsWDEGY1uztW69FXO3U9681Vh8OA6Z4+z9C79pCOKmiOf

v98t7WNrjZGenje3radxlaOlr+xC1b4gGebuVz9ewcG59y1MV2c.669+e5IW

Open the csv file in excel (or any program that can display the csv file as a table).  Copy a single column and paste into a .txt file.  Use the included [p load-a-coll] to load your colls with the contents of the .txt file.  Be sure to select 'save data with patcher' in the coll inspector.

The included subpatch will also load a coll with multiple columns if you save the csv as a space delimited file (all columns separated by spaces, not commas) and then change the extension back to .txt.

Here is an alternative approach using human cone spectral sensitivity functions (

 This approach doesn't require any XYZ to RGB conversion but it's also outputting a different 'interpretation' of wavelength values based directly on the range of our cone receptors. Notice how the values are black at the extremes rather than red and blue.

And just so you're aware.. this is all closely related to my own interests and work, so it was no big deal to share it with you. Your question actually motivated me to investigate all of this more closely so thanks!

btw.. you will notice that all of the values in both patches are at .1nm resolution. That's for my own use. You can easily replace the colls and tweak everything to 1nm resolution.

Here is an alternative approach using human cone spectral sensitivity functions (http://www.cvrl.org/cones.htm).  This approach doesn't require any XYZ to RGB conversion but it's also outputting a different 'interpretation' of wavelength values based directly on the range of our cone receptors.  Notice how the values are black at the extremes rather than red and blue. 

And just so you're aware.. this is all closely related to my own interests and work, so it was no big deal to share it with you.  Your question actually motivated me to investigate all of this more closely so thanks!   

btw.. you will notice that all of the values in both patches are at .1nm resolution. That's for my own use.  You can easily replace the colls and tweak everything to 1nm resolution.

This is me again, abusing your knowledge :P. I've been doing a lot of thinking into how to approach the sound-color mapping, and now this NM - XYZ thing is resolved, this is what I have in mind:

(I know you are already aware of the introduction information, but I want to be as clear as possible with my ideas to avoid confusions so here we go)

In sound, Frequency is perceived as pitch, amplitude as loudness, while the amount of waves sounding simultaneously and their relationship between each other is perceived as timbre; the higher the amount of simultaneous frequencies, the more 'chaotic' the timbre (e.g: white noise), the lower the amount, the purest the timbre (eg: sine tone)

In Color, oversimplifying, we could say frequencies are perceived as spectral hues, amplitudes as brightness, while the amount of waves vibrating simultaneously and the relationship between each other determines mostly saturation, the higher the ammount of simultaneous frequencies, the less saturated the color (eg: white color), the lower the ammount, the purer the color (eg: spectral color), BUT the thing is that simultaneity also could be perceived as the mixing of different colors, it all depends on some sort of relation between amplitude and ¿harmony?

And this is were my questions begin, I would like to read a sound spectrum as if it were an electromagnetic spectrum (for now, please forget about the octaves issue), mapping frequency as spectral hue, amplitude as brightness and the amount of individual waves, and their relationship among each other to determine the saturation, but in order to do this, I need to know how color is synthesized from a spectrum.

how do I know when a frequency with certain amplitude should be considered as part of the synthesis or disregarded?

how many peaks do I take into consideration?

Again, any pointers would be most appreciated.

And this is were my questions begin, I would like to read a sound spectrum as if it were an electromagnetic spectrum (for now, please forget about the octaves issue), mapping frequency as spectral hue, amplitude as brightness and the amount of individual waves, and their relationship among each other to determine the saturation, but in order to do this, I need to know how color is synthesized from a spectrum.
how do I know when a frequency with certain amplitude should be considered as part of the synthesis or disregarded?
how many peaks do I take into consideration?
Are there any methods around to do this?

The correlation between pitch and colour is a tricky one, and as you point out its complicated by perception not being easy to simplify - understatement of the year!

Having said that, my approach for this project would be to do a spectral analysis of the sound and use the FFT data to map to hue / saturation / brightness. Depending on how confident you are with spectral processing, you may find it easier to look around for FFT objects to do some of the "heavy lifting". AFAIK, there is no official way of doing this, and you will most likely have to make the connections between sound and light a personal creative choice. There are some general principles that you outlined, but I have not yet seen or heard of a "scientific" approach in this regard.

Thanks for your answer. I'm actually already doing this you are telling me, I'm taking a signal, analyzing it in real time and producing colors out of pitches (thanks to the help of Mr. Metamax).

Regarding my last post, perhaps I wasn't clear enough, but the thing is that I'm not tracking a single pitch but rather a harmonic noise, and this is why I want to take different simultaneous pitches into account. The idea is to take different pitches and group them according to their harmonic relations, take that spectral data and then synthesize different colors according to the present frequencies in a given group. My current question being: In the spectrum of visible light, how is simultaneity handled? We already know the coexistence of red and blue will give us magenta, but whats the mathematic analysis behind this? Whats the wave behavior I have to look for in order to produce such a mixture?

Thanks for your answer. I'm actually already doing this you are telling me, I'm taking a signal, analyzing it in real time and producing colors out of pitches (thanks to the help of Mr. Metamax).
Regarding my last post, perhaps I wasn't clear enough, but the thing is that I'm not tracking a single pitch but rather a harmonic noise, and this is why I want to take different simultaneous pitches into account. The idea is to take different pitches and group them according to their harmonic relations, take that spectral data and then synthesize different colors according to the present frequencies in a given group. My current question being: In the spectrum of visible light, how is simultaneity handled? We already know the coexistence of red and blue will give us magenta, but whats the mathematic analysis behind this? Whats the wave behavior I have to look for in order to produce such a mixture?

Light spectral information into Color synthesis