Phase drift does matter when you’re mixing waveforms for audio (rather than control). Try the attached example, using the -1 and +1 messages to flip the phase of the saw~ signal. Of course it doesn’t matter if the saw~ signal is on its own, but when mixed with the signal from rect~, it makes a huge difference. This means that if the different waveform generators drift out of phase, then the synth could sound different after you’ve left it on for a while! Of course 180 degrees is the most extreme phase shift, but any shift definitely matters.
I keep thinking there must be a "standard" Max way to keep the various anti-aliased waveform generator objects sync’d in phase, unless it’s absolutely impossible (or would take a very long time), for the waveform generators to drift out when they’re being controlled by the same frequency signal.