Arshia

For many obvious reasons I did not initially develop a Midi reader 

for Antescofo, the most basic one being that you can not even write 

trills correctly and things become very messy on the time level if 

the Midi file is not correctly prepared! On the development agenda of 

Antescofo however is the capability of atleast reading Midi and 

MusicXML, but this won't be a priority! My development priorities 

these days are ensemble coordination, augmenting the FWD semantics 

and also making the system robust for heavy polyphonic stuff 

Up to now, whenever I was confronted with a Midi score I did one of 

- Use NoteAbilityPro (gotta buy! I understand your reluctance!)

- Use an intermediate parser that reads MIDI and spits out Antescofo 

scores in text. Serge Lemouton of Ircam (in CC here) has already done 

this on the FTM Scoob editor that can read and parse MIDI, and then 

he parses it out for Antescofo using his java script program. I used 

his system for a whole piece of about 45 minutes long and used it in 

concert. It dramatically reduces edit time! Maybe you can ask him 

- If you have patience, write down the score by hand!!! Time 

consuming but for some situations the best solution so far.. . 

Moreover, you can make use of full odd event functionalities of 

Antescofo semantics (Trills, Multis etc.) and also put your cues 

directly in the score using beat-time delays.

The current version can be used in a polyphonic situation (as the 

examples suggest). But for 'heavy' polyphonic pieces (piano music 

with pedal, orchestra etc.) I am in the process of making it as 

robust as possible suitable for concert situations. This should be 

Also, soon you'll be able to synchronize pitched events (like midi 

scores!) as well as video (or anything else) concurrently. The main 

dilemma for that today is not the synchronization system itself but 

HOW to represent such data together in a coherent (multi-purpose, 

multi-dimensional) container in Max... . Non existing up to now!

For many obvious reasons I did not initially develop a Midi reader  
for Antescofo, the most basic one being that you can not even write  
trills correctly and things become very messy on the time level if  
the Midi file is not correctly prepared! On the development agenda of  
Antescofo however is the capability of atleast reading Midi and  
MusicXML, but this won't be a priority! My development priorities  
these days are ensemble coordination, augmenting the FWD semantics  
and also making the system robust for heavy polyphonic stuff  
(orchestra etc.).

Up to now, whenever I was confronted with a Midi score I did one of  
the following:
- Use NoteAbilityPro (gotta buy! I understand your reluctance!)
- Use an intermediate parser that reads MIDI and spits out Antescofo  
scores in text. Serge Lemouton of Ircam (in CC here) has already done  
this on the FTM Scoob editor that can read and parse MIDI, and then  
he parses it out for Antescofo using his java script program. I used  
his system for a whole piece of about 45 minutes long and used it in  
concert. It dramatically reduces edit time! Maybe you can ask him  
directly for that... .
- If you have patience, write down the score by hand!!! Time  
consuming but for some situations the best solution so far.. .  
Moreover, you can make use of full odd event functionalities of  
Antescofo semantics (Trills, Multis etc.) and also put your cues  
directly in the score using beat-time delays.

The current version can be used in a polyphonic situation (as the  
examples suggest). But for 'heavy' polyphonic pieces (piano music  
with pedal, orchestra etc.) I am in the process of making it as  
robust as possible suitable for concert situations. This should be  
ready before the end of November.

Also, soon you'll be able to synchronize pitched events (like midi  
scores!) as well as video (or anything else) concurrently. The main  
dilemma for that today is not the synchronization system itself but  
HOW to represent such data together in a coherent (multi-purpose,  
multi-dimensional) container in Max... . Non existing up to now!

Arshia Cont
http://cosmal.ucsd.edu/arshia/

re-live-audio-video-synchronization

I already prepared the score file manually, which isn't too problematic for short pieces. For the actual sound, I used some inbuilt drums from Ableton. The wav file was used as audio input together with the score text file.

For some odd reason, the recognition wasn't too good. Is there any specific hints/documentation when using antescofo for percussive stuff (drums etc.) ? maybe some pointers what could cause issues ?

and another question: What's the relevance of the beat duration if the tones are not sustained ?

Note that Antescofo is a reactive system based on recognition. Therefore the AUDIO QUALITY of the analysis is of extreme importance and has nothing to do with the audio quality you'd want to achieve (for example) when using a harmonizer! Therefore to be able to answer your question:

- What kind of mic are you using (Cardio, Omni)? and in what environment (noisy with lots of background electronics etc)?

- What is the instrument in question? How do you mic it for score following?

- Do you use any hardware/software audio compressor?

To be more useful, in every concert situation we have a separate adc for any recognition algorithm such as Antescofo. Sometimes you'd need to 'reduce' audio quality (by compressing etc) to achieve more robust results. If you give me some hints on the above questions I can be more useful!

Your second question was "and another question: What's the relevance of the beat duration if the tones are not sustained?"

Realtime tempo detection in Antescofo is based on IOI (or Inter-Onset-Interval). So as far as you have the same score in Antescofo as for the musician, things should go with no problem.

Note that Antescofo is a reactive system based on recognition. Therefore the AUDIO QUALITY of the analysis is of extreme importance and has nothing to do with the audio quality you'd want to achieve (for example) when using a harmonizer! Therefore to be able to answer your question:
- What kind of mic are you using (Cardio, Omni)? and in what environment (noisy with lots of background electronics etc)?
- What is the instrument in question? How do you mic it for score following?
- Do you use any hardware/software audio compressor?
To be more useful, in every concert situation we have a separate adc for any recognition algorithm such as Antescofo. Sometimes you'd need to 'reduce' audio quality (by compressing etc) to achieve more robust results. If you give me some hints on the above questions I can be more useful!

we are not sure yet on what drums to use exactly, potentially the bass drum and something else (to be more robust).

In terms of mic'ing it, again the option to use standard mic's (I am not a specialist, don't know about the specifics) or trigger microphones. If I am not mistaken,trigger mic's don't provide audio signals but rather impulses (on/off). In that case, the whole observation system wouldn't need to be tuned regarding noisiness, compressors etc. The question is whether antescofo would then be appropriate as there wouldn't be any pitch or midi information anymore to analyse.

How robust is antescofo in terms of missed notes/drum hits etc. ?

In terms of mic'ing it, again the option to use standard mic's (I am not a specialist, don't know about the specifics) or trigger microphones. If I am not mistaken,trigger mic's don't provide audio signals but rather impulses (on/off). In that case, the whole observation system wouldn't need to be tuned regarding noisiness, compressors etc. The question is whether antescofo would then be appropriate as there wouldn't be any pitch or midi information anymore to analyse. 

It seems to me that you don't really need a score follower! What you really need is a tempo tracker that works smoothly on onsets (that you get from Bonk~ or others).

Ofcourse you can always use Antescofo in MIDI mode in order to get a pretty fair tempo curve. But you'd go to the hassle of having a score for it! 

If you don't care about PITCH, then try putting fake notes (let's say all with pitch 60) but correct durations (in beats) and then simply use NEXTEVENT during the performance.. You'll notice that the tempo is being updated.

We have used Antescofo Midi synchronized with an orchestra just weeks ago with BBC Scotish orchestra and it worked smoothly.

Antescofo is pretty smooth in terms of missed notes/jumps etc. If you notice anything strange let me know!

But once again.. you gotta figure out whether your problem is really a synch problem or just tempo tracking... .

Ofcourse you can always use Antescofo in MIDI mode in order to get a pretty fair tempo curve. But you'd go to the hassle of having a score for it! 
If you don't care about PITCH, then try putting fake notes (let's say all with pitch 60) but correct durations (in beats) and then simply use NEXTEVENT during the performance.. You'll notice that the tempo is being updated.

the requirements are shifting quite a bit, the more one finds out :-). As it doesn't matter whether sync'ing visual effects or firing off audio effects at specific times, it should be more than just tempo "following".

I used bonk the other day, loading it with 8 different drum sounds (bd, tom, snare, etc.). Then I streamed some audio from my sequencer, and bonk successfully recognized the different instruments, returning the respective id.

I shifted the values by 60, thereby making it appear like midi notes to antescofo (using standard mode converting midi to frequency).

For some reason, if there was the same note (instrument) played twice after another, it didn't recognize the second occurence.

I shifted the values by 60, thereby making it appear like midi notes to antescofo (using standard mode converting midi to frequency). 

For some reason, if there was the same note (instrument) played twice after another, it didn't recognize the second occurence. 

I'm happy that things are starting to move... .

For the case of "repeated notes", make sure you somehow send in a NOTE OFF (pitch ZERO) to Antescofo, otherwise it'll think you're still on the same event without moving to the next event with the same pitch tag. That's all!

For firing stuff, try the FWD, PFWD and LFWD commands.. . They couple with tempo in realtime and have proven to be quite musical and useful in performances... .

I already used noteoff's between repeated notes/drum hits. Still, at one point, antescofo recognizes the repeated note, the second occurence though is not recognized.

It's hard to "debug" over the forum, I could send you a zip with :

Let me know whether I shall send it, don't want to occupy you too much.

I already used noteoff's between repeated notes/drum hits. Still, at one point, antescofo recognizes the repeated note, the second occurence though is not recognized. 

- the wav file
- the bonk patterns
- the antescofo score
- the patch
- (the mid file for the sequence)

Just to follow up Nick's problem with Antescofo... The problem was mysteriously solved! So the current distribution is for now solid. Do not hesitate to bug me with bugs!

A note: You do not need to put fake silences between notes! Antescofo automatically puts dummy silences between notes even if there's no silence indicated in the score... .

Re: live audio - video synchronization