Patcher outputs
Dear All,
We have a subpatch defined with "p". This subpatch has multiple outputs. What is the best way to specify the order in which the outputs trigger? When using this subpatch, it became obvious that the order in which outputs send data is not deterministic. Sometimes it is the left-most outlet that triggers first and sometimes it is the right-most.
Thanks!
Atmapuri
It depends on what your subpatcher is actually doing. With no example of what is inside we can't hep precisely.
This said, [buddy] or [bondo] objects might be what you are looking for.
Unlike most Max objects that send their output in a predictable order from right to left, sub-patchers are totally dependent on the code that you put in them. They will send output to any outlet as soon as the data is available. Therefore, if you need the outputs to send their messages out in a predictable order, you need to add the code necessary to make this happen. Quite what is the best way to do this will be entirely dependent on what the sub-patcher is doing, what data/message(s) is being output and the time intervals involved.
the message order in custom max code is 100% your responsibility, and it is a great gift that you are free to organize things like you want.
however, in most cases you want to make sure that from outside your subpatcher or abstraction looks and works like most externals do, too: i.e. with a strict order from signal to message, video to message, and with inlets of "hot to cold" oder and outlets "last to first" order.
feel free to post 2 or 3 examples of your patches, then we can discuss.
Here is my patch. The goal is for the outputs to trigger in the order 4,3,2,1.
I know nothing about sadam objects, but it *seems* that for one message from the topmost sadam.stream, you would get one message per outlet. If it is indeed the case, then as I said [buddy] is your friend. If some of the outlets receive more messages than the other, you can try to keep outputs synchronized by sending the "clear" message to buddy when a new message gets output from the topmost sadam.stream.
If in the rest of your patch you can handle the possibility of having some outlets updated several times while the others get updated only once, then [bondo] might be simpler to work with.
The sadam passes a list of bytes, which is a UDP packet message. The message has a header and depending on the header the patcher will decode the content from bytes to int, float or char array. The output will be always only one of the outputs 2,3,4 and also 1. The goal is to have 2,3 or 4 to trigger first, before the #1 is triggered.
This is the kind of precision we would like to have from the very first post!
Then I would do something like that (added objects with red borders):
If you can't organise processing of post header to ensure output order,
this simple fix could help:
I have tried the suggestion by "source audio", but this is still not working. Namely, The outputs 2,3,4 will send output, and trigger #1 before the unpack will send data to the integer storage.
Did you try my solution? [buddy] waits to get an input in each of its inlets before outputting anything, so it will always output in the same order a pair of an integer and a 32/64/char array regardless their order of arrival. It might still get out of sync depending on your patch, but it seems that it shouldn't be an issue here.
Also, the top part of your [p readHeader] makes no sense to me. Like [expr $i1*4] with always receive a 3, so will always output a 12, so your "getarray $1 $2" message will always be "getarray 1 12" regardless of what's coming from the patcher's inlet (unless sadam.stream can send messages such as "set 15" which could be interpreted by message objects, but it doesn't seem to be the case).
About pReadHeader: This is a setup for potentially variable size header with extra fields. It was just a study how to do stuff.
About buddy: One of the requirements is that only that "data array" type triggers, which actually has new data and not all of them.
What is also hampering my ability to efficiently debug this is that even "print" commands depend on on the right to left, top down logic.
Is there some recommended approach how to monitor the sequence of execution?
Thanks!
Atmapuri
you stated that ony 1 type of data gets detected an output,
in which case my solution must work.
"The output will be always only one of the outputs 2,3,4 and also 1. The goal is to have 2,3 or 4 to trigger first, before the #1 is triggered."
if they all output something, then of course not.
Yes, the spec is correct. For now I have implemented this:
But it is not sufficient, that this patcher is outputting in correct order. It is also the calling logic, which is causing me trouble. How can you see in the calling logic which output triggered first? The print command evaluation moment depends on the position of its box.
I can't use sadam objecs - being on the road.
but you did no impement what I suggested.
you say that only 1 of outputs 2,3 an 4 triggers, depending
on type you try to detct.
so wha is the problem ?
maybe you have more than 1 output from 2,3 & 4 ?
It seems that this patcher is wreaking havoc on sequence:
It looks as if though it is a thread blocking call which is buffering incoming messages.
You can get a better sense of output order by using a single print and prepend some identifiers to each incoming signal, like such:
I feel like using watchpoints and the debugger might help you as well.
But without further explicit details or examples I feel like we basically just move air and add sweat on our keyboard by trying to help you.
For example:
- could we get an example of incoming data stream so we can actually test the patcher? One example for each of the int32 float64 and char types woould be awesome.
- if [p readHeader] was just a study, what's going on now in your current patcher? From the version you posted here, [p readHeader] has a major role in execution order as it basically triggers all inputs (even though in its "study" version it will always output the same thing regardless of its input).
- What device or program is sending data to the top sadam.stream? In which frequency is it updated?
About buddy: One of the requirements is that only that "data array" type triggers, which actually has new data and not all of them.
I agree with Source Audio, this is confusing with what you said earlier. Maybe you might need [change] and/or [zl.change] to filter out repetitions? (which would also prevent you from receiving two times the same message in a row, if that kind of thing can happen).
What has this path problem to do with previous patch,
for which you asked help for ?
this should be a better way to combine path with arbitrary name.
you must use symout, because path will get broken if it contains space.
what for do you use absolutepath for ?
only sense would make to check if file exists.
But if file does not exist it outputs notfound, which makes no sense in that patch.
if you need to resolve path to ~/ (user directory)
including user name, take conformpath as shown above.
Dear All, I appreciate your help a lot. Still I cant figure out what the problem is. Here is the complete patch which is using both of the specified above.
The error is that the rFileInt triggers bang on select 16 before zl.reg received list of chars. This makes no sense to me. First the UDP Package is unpacked to command and data and for some reason the bang from command always reaches the zl.reg before the zl.reg is filled with the data from that UDP pacakge.
You will get no help with that bunch of stuff in the patch,
and still not a singe example or explanation of udp message
which has to be routed, detected or whatever.
wipe all unneeded stuff from the patch, and post it again
with example or explanation of expected UDP message.
where is udpreceive in that last patch ?
what for are report 1 and change patchers with named steam which does not get used anywhere ?
Let's take it that you receive udp packed starting with that 3
header values
1- messge type
2- data type
3- array length
as you seem to parse them as int32, you have 12 bytes in the header.
split that from the packet and after evaluating that 3 values, send the rest of the packet where it belongs, or split it, whatever.
that is so easy once you know what to expect.
P.S. what for do you need array length ?
if data type is int32 you split 4 bytes and
send rest to itoa - if that is your "path"
if data is float64 then you slice 8 bytes.
Here is the simplified patcher:
print rFileSelAmpltChange shows bang before "char array" reaches zl.reg.
where is udp string example that needs to be parsed ?
Two commands for example:
(integer) cmd 15
(Integer) Type 2
(Integer) Length 12
(list of char) Data: "image15.png"
(integer) cmd 16
(Integer) Type 2
(Integer) Length 12
(list of char) Data: "image16.png"
The goal is that each cmd will have its own processing path, but currently image16.png lands in the image object for image15.png. And even more complex mixups with more commands can happen.
And here is the list of bytes for the two commands:
Cmd15ByteList: 15 0 0 0 2 0 0 0 57 0 0 0 67 58 47 85 115 101 114 115 47 65 116 109 97 47 68 111 99 117 109 101 110 116 115 47 76 105 102 116 80 101 111 112 108 101 47 82 101 112 111 114 116 49 65 109 112 108 116 67 104 97 110 103 101 46 112 110 103
Cmd16FileByteList: 16 0 0 0 2 0 0 0 57 0 0 0 67 58 47 85 115 101 114 115 47 65 116 109 97 47 68 111 99 117 109 101 110 116 115 47 76 105 102 116 80 101 111 112 108 101 47 82 101 112 111 114 116 49 80 104 97 115 101 67 104 97 110 103 101 46 112 110 103
Currently, it sometimes works and sometimes not. Which means, that somehow the sequence of execution is not deterministic. (not always the same).
Atmapuri
split header from data.
you don't need length.
use zl.slice 8 to get cmd number and data type to open gate
to route the rest
I thought that your packet is combining header , int or float and ascii string
have a look at this example, you will see what I mean.
Its not combining. It is decoding data and cmd out of the received packet. And then using Cmd to know what to do with the data.
Sometimes this is the output:
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic "/Users/janezmakovsek/Documents/LiftPeople/Session page not active!"
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Dummy.png
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Dummy.png
rFileCmd: 51
rFileCmd: 50
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Report1AmpltChange.png
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Report1PhaseChange.png
And sometimes this:
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic "/Users/janezmakovsek/Documents/LiftPeople/Session page not active!"
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Dummy.png
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Dummy.png
rFileCmd: 51
rFileCmd: 51
rFileCmd: 15
rFileSelAmpltChange: bang
rFileAmpltChange: pic /Users/janezmakovsek/Documents/LiftPeople/Report1AmpltChange.png
rFileCmd: 16
rFileSelPhaseChange: bang
rFilePhaseChange: pic /Users/janezmakovsek/Documents/LiftPeople/Report1PhaseChange.png
There is somewhere some race condition. For this patch:
Note that rFileCmd is different between the two runs for the same data. I send both commands in sequence. First cmd15 and then cmd16 (byte list). But the output varies from run to run.
that is of no help to show your wrong result.
post the string which you send.
you have problems to parse and output it properly.
main problem is definitely in that readHeader pacher
and the wy you rute dt to that 3 read array patchers.
You still don't show who is sendin data, where is it received,
why are you not using udpsend and receive or even OSC, and so on.
You shoud know by now how to route different types of data or
simpify your data strings, maybe insert delimiter to know where your data string starts or ends, there are so many ways ...
if you don't need more than 256 message and data types,
why send them as int32 ?
where 2 int8 bytes would do ?
and why send full path to known folder containing some png files,
instead of recallig them using a single byte from preloaded umenu etc.
I am asking all this, because it kind of makes no sense to make this complication for few messages, no mater where they come from.
I just saw 2 examples you posted yesterday...
you send header as int32, but the rest as int8
why do you try to parse data type 2 in such complicated way ?
this is all it needs :
I am completely ignoring here your length value - no need for it.
P.S.
here is a beter way to route your messages
>post the string which you send.
But you have the list of bytes. There is string included.
>and the wy you rute dt to that 3 read array patchers.
Because sometimes the data received is string, sometimes it is a list of float64 and for some commands it is int32. It depends on the command.
>You still don't show who is sendin data, where is it received,
The receiver is only this:
The data is sent by a C style language.
>why are you not using udpsend and receive or even OSC, and so on.
Because they are specific to MaxMSP and are not cross-language capable.
>simpify your data strings
Data is not always a string. It is a list of bytes. The list of bytes posted above is what gets to be transmitted over the network. Sometimes it is a (C) array of float64 or int32 and sometimes unicode char array.
>I am completely ignoring here your length value - no need for it.
The length is there to avoid packet splitting and verify that a complete UDP packet was received.
Sadam UDP Receiver has a bug here. It does not implement its own message queue. As soon as you send more than 64kB of anything the packets are just dropped. So I had to add 100ms sleep periods on the sender side, to be able to semi-reliably copy data bigger than 64kB. Luckily the png files are small. Usually smaller than 64kB. But still I have to wait between sends of images, so that the previous data is processed by MaxMSP.
>and why send full path to known folder containing some png files
Because those png files were transmitted over network to the computer on which MaxMSP is running. I used a javascript object in MaxMSP to be able to save the binary files. The recepient needs to know which png files were copied to be able to update the fpic object. The fpic may not lock the files during the copying process either. Here is the logic:
a.) In the first part you send "Dummy.png" file names to unlink fpic from the local png files and prevent write lock error.
b.) Copy the files over the network.
c.) Relink fpic objects to the newly copied files.
>this is all it needs :
You are joining together the command and the string in to one message. The whole logic is there just to get them separated. Depending on the type of the message and the command in the message, the string needs to be routed to different fpic object.
Thanks!
Atmapuri
>here is a beter way to route your messages
Ok, Using "forward" and then catching with "receive". I dont understand what "t set" does. There is no info in the help file. Just converts input to bang?
Additionally, it is not obvious if you can also send list of int32 and list of float as a part of the message instead of string for example.
t set just clears messages to only show new data...
test this, just few examples, including primitive data length check
you can still simplify the header if it needs no larger values than 0-255
list with 6 foat64 values is only 6 x 8 = 48 bytes
When I am copying the png files over the network, the list is 1024 int32 items long and 4096 bytes.
What you sent looks very promising. It is nearly a complete program.
The "t set" clears all message boxes, which are not updated. However, I would like them to remain unchanged. Not cleared. I am clearing fpic content explicitly one by one by sending some dummy string, which fpic cant convert to image and then send valid image names to those fpic objects that needs to be updated. This I still dont know how to do from your example without introducing a whole lot of extra logic. (maybe doubling up on what you have now).
you can set alpha 0 to hide loaded pic
and again alpha 1 after loading new one
if you don't clear messages that monitor data, at least set restorecontents atribute to 0.
This is not about hiding, but unlocking the file on the file system to allow overwriting the file with new content.
what do you use to wipe the file ?
or another question - if you anyway send image over network,
can you use some jiter stuff to display it without need to store it to disk ?
>what do you use to wipe the file ?
I simply overwrite the file. But before I overwrite, I need to dereference the file from MaxMSP fpic object so that MaxMSP is no longer blocking and preventing the file from being written to. This how my javascript looks like:
autowatch = 1;
inlets = 1;
var f;
function createFile(fileName) {
f = new File(fileName, "write", []);
post("Filename: " + fileName +"\n");
}
function closeFile() {
f.eof = f.position;
f.close();
post("File closed\n");
}
function writeFile() {
var intArray = arrayfromargs(messagename, arguments);
f.writeint32(intArray.slice(1));
}
The dereference happens by sending a separate message which only sets the filename (string) referenced by the individual fpic objects.
>can you use some jiter stuff to display it without need to store it to disk ?
If I only knew how....
I am away till tomorrow.
maybe you can find somehing about raw dato to jitter in the forum...
or post a request in new topic
FYI: Same problem with your patch as with my own. There is variability between runs.
Namely, I get cmd M15 in quick sequence with M16, but the data that gets printed out is from message with cmd M16.
I cant even concieve yet in my head how can this happen...
This is what print commands output:
aaaPhase: "Session page not active!" // wrong, because this message is from command "error" . The message command would be M42
aaaPhase: Dummy.png // wrong, because this part of command M15, should have label aaaAmplt
aaaPhase: Dummy.png // correct
aaaPhase: Report1AmpltChange.png //wrong, because Amplt is part of command M15, which has label aaaAmplt
aaaPhase: Report1PhaseChange.png //correct
And sometimes I get this:
aaaPhase: "Session page not active!"
aaaPhase: Dummy.png
aaaPhase: Dummy.png
aaaAmplt: Report1AmpltChange.png //note the change here
aaaPhase: Report1PhaseChange.png
You have probably some mistakes in sent messages or you send them too fast ?
best check is to test in max using a message
like in my example.
if all gets routed ok, then you need to check
UDP packet
Now we are getting closer to the true cause of the problem.
a.) The LiftUdpSender is sending two packets of the same kind with one bang. That is already wrong.
b.) The receiver actually works correctly, if I connect it directly to sadam.UdpReceiver.
But there is something funky with the stream handling. Something I dont understand about how this object is to be used.
get rid of path and send only file name
no need for that patch and strippath
I tested fpic - it does not prevent loaded file to be replaced.
all you need is to send it path again.
example:
fpic has loaded ~/Documents/test.png
you rewrite it and send :
pic ~/Documents/test.png
fpic replaces it's buffered contents with new png.
Thanks! That is another simplification. Obviously no need to expand the path either. Yesterday I managed to zero-in on the root cause of problems. Somehow the:
sadam.stream StreamName
object is not thread safe. I tried all kind of stuff. It just starts to shuffle garbage. When used for sending packets it can send two instead of one and when receiving it does weird stuff as well.
Currently I am reworking all the code to work without this object. (the patcher I posted before is only one of many). Hopefully I will have some results by the end of today.
I guess you don't need sadam.stream at all.
but where from are you receiving udp messages ?
some software on same computer or local network ?
Can be either, same computer or local network. Mostly local network.
you seem not to be willing to name any details about the source of messages ...
It is another computer on the same local network with software written in c-like language.
Dear "Source Audio",
I can confirm that all problems have been resolved just by removing references to "sadam.stream NamedStream" object. I am now passing sadam.udpreceiver and sadam.udpsender as a parameter to patches directly. Everything just works! But boy, it was nearly a week of work to fish out:
1.) zl.reg has a limit of 255 list items.
2.) sadam.udpReceiver does not implement message queue. It looses packets once total received unprocessed data exceeds 64kB. You can specify any buffer size, but once accumulated data reaches 64KB, any new packets are simply dropped.
3.) sadam.stream NamedStream has threading issues.
I cant thank you enough for providing additional insights on the last point. It helped me immensely. You must have spent several hours by fiddling with this...
Why MaxMSP:
a.) Does not allow binary file writing.
b.) Does not provide proper UDP protocol components.
remains a mistery, but I can understand that sometimes there are tradeoffs.
Kind Regards!
Atmapuri
I am glad you have you stuff finally working.
And - trust me I spent more time asking about details, then actually putting together that few examples.
zl.objects have default max size of 256 items, but one can increase that
as shown in zl objects help file.
then, there was fileout.js example since years arround, don't even remember who posted that, opposite of filein ...
could have worked for you to collect raw image bytes and save file to disk.
I asked abut that software that sends data to max, because there may be a different approach to send or download image files, without a need to stream them as UDP packet.
or to form UDP packets to be compatible with max UDP objects.
what I want to say - more precise you ask the questions and provide more details about the whole, it is easier to provide any help.
the only working link for the latest mxj is this https://cycling74.com/forums/sharing-mxj-fileout
if you need to read the data later only with max, you can also abuse a jitter matrix to create custom binary data file formats.