Why does my standalone crash the first time it runs but runs perfectly thereafter? Or, trying to manage the idiosycracies of UDP/OSC in Windows using Behringer/Midas mixers!

Andy Maskell's icon

Some brief background (as this is a loooooong post):

1) My application is huge but hopefully well organised with about 25+ sub-patchers and bpatchers. It runs perfectly every time I run it from the Max 8.52 editor.

2) As soon as I build it into a standalone, the executable always crashes on the first, and only the first, time it is run. Every subsequent run is entirely successful.

3) If I encapsulate the standalone in a VirtualBox, if the original standalone has not been run previously, the VirtualBox version also crashes repeatedly. If I run the standalone at least twice before encapsulating it (always one crash followed by any number of successful runs), the VirtualBox version works perfectly every time!

4) The crash takes the form of the top level patch window showing the spinning dial followed by the window greying out with a Windows system generated "not responding" message. I have to click the close box and then select the shutdown option to get out. If I do nothing, this apparent hang-up continues indefinitely with no measurable CPU activity or messages in the Max Console.

So what is going on?

The crash happens during an initialisation phase. I have gone to great lengths to ensure that this all happens in a very specific order as most of the modules require some form of initialisation routine and certain modules are dependant on others to have been initialised first.. I have painstakingly tested all manner of orders of initialising the modules and have been obsessive about eliminating [delay] and [pipe] objects in the initialisation routines to ensure that these processes are not colliding or conflicting with each other due to deferred scheduling. I also don't use Overdive.

After hours of testing, I have determined that the problem lies somewhere in this piece of code and almost certainly has something to do with the Max [udpreceive] object:

Some more explanation is useful here:

1) Max does not provide any means to request Windows to allocate a free UDP port as many applications choose to do - you must supply a port number.

2) Loads of people have reported problems with [udpreceive] failing to work in apps that are run repeatedly using the same port number until Windows is rebooted and I have found the same thing.

3) This routine creates two almost identical portals though which I communicate with a Behringer X32 Rack mixer and a Behringer XR18. Both Midas and Behringher have adopted a very strange protocol for using UDP. Rather than recieving and sending messages on adjecent ports, as is the common method, they have chosen to both send and receive on the SAME port. By default, this port 10023 for the X32/M32 series mixers and 10024 for the XR/MR 12/16/18 series mixers. I don't know about the Wave series but I expect it suffers in the same way.

4) Aside from the problems with [udpreceive] failures mentioned above, [udpreceive] also fails to work if it is configured with the same port number as the [udpsend] command. In other words, it simply doesn't work with these mixers. Yes, I have heard some people talking about broadcasting the sent messages but in reality it doesn't stand up robustly for any length of time.

5) Having opened a UDP port in Max, there is no way to close it again and I think this explains why patches don't work when run repeatedly - neither Max nor Windows are releasing the port.

However, using the equivalent Sadam UDP objects DO work and they work reliably and persistently even when the send and receive ports are configured to be identical. It also works on repeated runs of the same patch. So what are the Sadam routines doing that the Max ones don't?

The one problem with Sadam is that it does everything in OSC binary bundles and "streams" and regardless of how many times I read he help pages, I can't figure out how to convert normal, text based OSC commands into the correct format.

Then I discovered a workaround published elsewhere - send the textual OSC commands with [udpsend] to [sadam.udpReceiver] via the internal loopback (localhost or 127.0.0.1) and send the replies via [sadam.udpSender] to [udpreceive] - and blow me it works! Perfectly! Every time and for hours on end!

So how have I tackled the problem with [udpreceive]/Max/Windows blocking up ports? My solution has been to reconfigure the port numbers randomly every time the routine is reset. I have chosen to use adjactent even/odd ports in a rarely used block from 15000 to 20000.

As stated previously, the [udpreceive] object MUST be defined with a port number. I've tried all manner of port numbers in different ranges and it seems to make no difference to the crashes mentioned above so before anybody suggests that it's because I've used [udpreceive 0] as a starting point, I can assure you that it makes no difference what port number I choose. Once the object is defined though, the port number can then be changed any number of times, which is what I do with my randomiser.

So does some mega-guru out there have any suggestions? Why does this "one-time" crash occur, why doesn't it happen on subsequent runs, and how come I can get a reliable VirtualBox package to work provided that the executable has been allowed to run and crash once before it is encapsulated? Is it something that has been introduced and/or made worse with the Max 8.5+ upgrade?

And for all you Behringer/Midas users out there who are pulling your hair out trying to interface to your mixers using Max, this method does work!!!

Source Audio's icon

few very short answers to this very long mail ....
1- sadam udp send - receive : insert atoi / itoa at both ends.
Simple as that.


------
2- that one time crashes might indicate preferences problem,
but to keep it simple I would reboot windows after compiling standalone,
to remove any loaded dependencies from Max itself,
and after reboot launch standalone to see if it makes any difference.

It would be helpfull if you upload a screenshot showing ALL
settings of standalone inspector.

And - your standalone is an unknown variable -
without knowing all involved objects and routines
it is impossible to advice anything .

P.S.
do you compile that complete standalone every time ?
One can build it once excluding externals, edit resources, trimm it down,
and on updates compile only updated mxf , also excluding externals.

Andy Maskell's icon

Thanks @Source Audio.

I have reworked my routine a bit and performed large numbers of tests removing and isolating various objects and actions and confirmed without any shadow of doubt that it is the Max [udpreceive] object that is causing the problem. I attach the latest version of my routine here if it is of interest:

Max Patch
Copy patch and select New From Clipboard in Max.

And here is the screenshot of my [standalone] settings:

As I have said, my app is large and complex but it all works fine apart from this one initialisation routine. All the app does in effect is to process MIDI and OSC data streams between several devices making changes in various different ways to add functionality. Without the hardware, it would be pretty unrewarding to look at!

As for the OSC binary data structure, I have fathomed it out but I don't have a solution for decoding it yet. The general structure of the messages is:

oscaddress 0 0 ,sfii 0 0 dddddddddd 0 0 dddd 0 0 dddd ....

The OSC address is simple to convert using [itoa]. However the "0 0" delimiter causes most things to stop at that point. The next segment is a comma followed by a variable number of data type identifiers, such as "s" for text symbol, "f" for 32-bit float, "i" for 32-bit integer, and so on. What follows is then a series of data values that match the number and type of the identifiers, again separated by "0 0" delimeters. I'm sure that I could write a decoder to unravel it all but I'm going to hunt around to see if anybody has already written something suitable. If I can find/write a de-coder then I can get rid of the troublesome Max [udpreceive] objects.

As for rebooting, it makes no difference irrespective of when a reboot is done, i.e. after building the standalone, after running the standalone once or more times, after encapsulating the standalone into VirtualBox, or after running the VirtualBox!

Andy Maskell's icon

@SourceAudio - one other thing I don't think I mentioned is that I have a Master Reset button on my user interface that performs a complete reset by running all the initialisation routines, including this one, in the event that any part of the external network, hardware or software fails. It seems to be only the first encounter with the [udpreceive] object that causes the crash. Thereafter, all subsequent encounters work exactly as expected, whether it is by running the app repeatedly or doing multiple resets during a single app run. Really odd!

Source Audio's icon

You disable overdrive and use that many midi messages ?
Enabling overdrive and all windows active is first thing I set.
If you keep same preferences name as Max itself - that will
probably also produce problems.

at the bottom I see you use OneDrive.
I am not really sure, but I think using max files on virtual drives
might also create problems.

now you seem to nail it to that RESET routine.
I had a short look at that patch.
Could you try qmetro and slower count ?

Andy Maskell's icon

Thanks @SourceAudio.

I have disabled overdrive temporarily as it interferes with another initialisation routine that goes before this one. I need to work out why but it certainly doesn't relate to this problem and I'll be turning on overdrive again once i've fixed that.

I'll try All Windows Active although I only ever have one interface Window visible when my app is running.

I'll certainly try using a separate preferences file and see if that helps.

I've already tried metro values up to 2000ms and it makes no difference. I'll try qmetro but I doubt if it will help as my rigid flow means that this is the only active process running in the scheduler at the time.

Whilst I'm using OneDrive, I only use it as a backup system. All my files are stored locally all the time. It's not caused me any problems with Max for the 2 years that I've been using it.

Andy Maskell's icon

@SourceAudio - I have fixed my other routine so I have enabled overdrive again.

I have applied all the other suggestions too and yet the problem still remains unchanged! 😒

I've not found an OSC encoder/decoder that I can use so I'm off to write one!

Andy Maskell's icon

Well, I'm not exactly sure how I've fixed it but I now have a working program again!

As Ken Mattingly (Gary Sinise) said in Apollo 13:

"It's all in the sequencing, John."

I've made adjustments to the sequence of events during the initialisation process and moved the launching of the Mixing Station software that I interface to until after the UDP/OSC ports are created, just in case any flurry of activity between the software and the mixer as they synchronised was swamping Max at the same time. I've added most of the tips given by @SourceAudio as well - thank you.

Phew! 😉

Source Audio's icon

another thing > udp ports should not have that low numbers
like 1 or 4, at least on mac that throws errors and ports do not work.
ports > 1024 have no problems

Andy Maskell's icon

Normally I would totally agree with you. I originally had them set to 12020/1/2/5/6 but I was desperately trying everything I could think of to get it to work; and my randomiser picks 4 sequential numbers from 15000 to 20000 always starting with an even number.

My experiments appear to suggest that [udpreceive] needs an almost silent machine and network when it is first invoked but quite why that is I have no idea. It certainly doesn't appear to be coded very well and I'm far from being alone with this problem.

Andy Maskell's icon

I've set the default [updreceive] port numbers to 14996/7/8/9 now.

I know from previous discussions with the developer, David Schumann, that the MixingStation software I use is constantly having to request free UDP ports from Windows as the mixers it interfaces to use UDP servers to transmit the live level meter data. Each server request requires that allocation of a new UDP port is made by the host and passed to the mixer in the server request. I'll message him and ask if there could possibly be anything going on in his software that could be causing a snarl up.

I certainly know that interfacing to these mixers via UDP has caused no end of problems for other Max users out there. However, my solution now works again!!! Yeahhh! 😁