You could try to use a gate and a gate~ to switch between both inputs. But it won't work with all objects (maybe it even never works...) as once a signal is connected to an input, it will have priority over control rate (a signal connection always sends values).

Another option is to script the patch cords connection, but you will hear it as the dsp chain will be recreated.