ygreq

I work mainly with Max and Jitter so this might sound stupid for the musicians, but I am trying to figure out what to look for in sounds that I can interpret as data. 

Imagine someone saying "I love you'. This not a simple unmodified sound. It changes as you say it. So I would like to know what really changes. Is it the pitch, the amplitude, the volume? Something else?

I would really like to extract that information as data in real time. 

I work mainly with Max and Jitter so this might sound stupid for the musicians, but I am trying to figure out what to look for in sounds that I can interpret as data. 

Imagine someone saying "I love you'. This not a simple unmodified sound. It changes as you say it. So I would like to know what really changes. Is it the pitch, the amplitude, the volume? Something else?

I would really like to extract that information as data in real time. 

I hope it makes sense

Hey there!
I work mainly with Max and Jitter so this might sound stupid for the musicians, but I am trying to figure out what to look for in

what-makes-a-sound

Nothing else than frequency, amplitude and timing.

The mystery are the combinations, patterns and dependencies. Let's say frequency modulations and dynamic behavior. On top of these basic elements you have the formants, which are peaks of energy in certain frequency ranges and they form the typical sound. A complex description is the timbre, a combination of root tones, partials (overtones) and noise.

Even if you are not interested in the voice iself, you need parts of the timbre for analysis. For example, think about the the vowels "a" and "o". Pitch, volume and time can be the same, so there must be something else that makes the difference. Formants. Timbre.

Once upon a time, in front of a new sculpture, a famous sculptor was asked how he could make such a wonderful lion out of a rough piece of stone. He answered, "This is easy. I just take off what doesn’t look like a lion." The ironic answer bears the truth. You cannot extract "that" information. You can extract many components and interpret them in the way you want.

Back to speech. If you want to electronically understand what was said, you aim at a kind of reverse process of speech synthesis. Acoustic translation programs do it and they concentrate on typical aspects of languages and emphasis.

For deconstruction, re-synthesis or control data you may focus just on certain aspects. Envelope following is a starting point (volume), then the same technique for frequency (pitch). Frequency- and volume splitting, rise- and fall times of dynamics, frequency- and dynamic patterns etc. It goes complicate with the grade of detail you want. Amount and use of transients (short volume bursts) are typical for human speech. Roughly spoken: French people don’t know consonants, Polish people have only consonants. Frequency modulations at certain points (syllables) are typical for some languages, sometimes they change the meaning of a word.

Does this help? If you tell us what you want to do with your de-composed speech, some people here may be able and willing to help you. But we need specific questions.

Nothing else than frequency, amplitude and timing. 

Once upon a time, in front of a new sculpture, a famous sculptor was asked how he could make such a wonderful lion out of a rough piece of stone. He answered, "This is easy. I just take off what doesn’t look like a lion." The ironic answer bears the truth. You cannot extract "that" information. You can extract many components and interpret them in the way you want. 

Back to speech. If you want to electronically understand what was said, you aim at a kind of reverse process of speech synthesis. Acoustic translation programs do it and they concentrate on typical aspects of languages and emphasis.

For deconstruction, re-synthesis or control data you may focus just on certain aspects. Envelope following is a starting point (volume), then the same technique for frequency (pitch). Frequency- and volume splitting, rise- and fall times of dynamics, frequency- and dynamic patterns etc. It goes complicate with the grade of detail you want. Amount and use of transients (short volume bursts) are typical for human speech. Roughly spoken: French people don’t know consonants, Polish people have only consonants. Frequency modulations at certain points (syllables) are typical for some languages, sometimes they change the meaning of a word.

Thank you, Peter! It's gonna be a hard one for me in this case :)

The simplest thing to start with might be to do pitch and envelope-following on the speech. More complex approaches could make use of audio descriptors such as zsa.

Thank you, Kenneth! As I have zero experience in MSP, I will look on forums to see some examples. 

What I would want in the end is to show some data on the screen in real time about the sound

Thank you, Kenneth! As I have zero experience in MSP, I will look on forums to see some examples. 

What I would want in the end is to show some data on the screen in real time about the sound 

What makes a sound?