Something like buffir~ in the phase domain

Ok, that's difficult.
I want to implement a phase equalizer, a filter which leave the gain of each frequency bin at 1 and alter JUST the phase referring to a buffer.

I've tried with a fft-->cartopol-->phase transformation-->poltocar-->ifft chain, but at very high frequency resolution (i.e. 1Hz) it has a latency equal to the frequency resolution period
(I think it's caused by the indetermination principle). So if i have a 1Hz frequency resolution i will have a overall latency of 1 second!

There's nothing like buffir~ which acts in the way i've described above?
Any suggestion would be appreciated.
Thanks