Can I pitch track real vibraphone in a live situation and convert sound into microtonal in max ?

If the vibraphone plays monophically (one note at a time), then yes, you can do pitch tracking in realtime quite easily with [fzero~], [retune~], [sigmund~] or many others. If it's playing in polyphony though, then maybe another kind of tracking would be more efficient (piezzo for each note, computer vision...).