Question about send/receive implementation and efficiency
When you have patches loaded and there are multiple send and receive objects all over the place, does Max do "fixup" (like linker fixup, say) so that when something has to be send out, Max knows to which receivers the messages have to be sent?
Or does it look up the list of receivers every time a message arrives at a 'send' object?
I am using more and more send/receive objects in my patchers and am a little concerned about the impact of lookup if that has to happen for every message, particular when there are a lot of receivers.
Thanks,
D
seems like a direct patchcord connection will always be more efficient than send/receive pairs. https://cycling74.com/forums/which-method-is-faster-patch-cords-or-sendreceive-pair
Excellent. If you could tell me how to connect patchcords from an object in one patcher window to an object in another patcher window, I would be most grateful!
if you're talking about 2 separate main patches than you obviously can't use cords. but generally spoken: yes there is this lookup that makes s/r connections slower than direct connections. but not relevant in most cases
Yes, I was talking about separate patchers. I apologize for the vaguely sarcastic tone of my previous response but it always bothers me when I ask a question and get an answer that sidesteps what I'm trying to understand.
Anyhow, it depends how that lookup is done.
For example, (and I obviously don't know how the code is implemented), but if the names of [send] and [receive] objects are "registered when a patcher is loaded, it would be possible to essentially preconnect each [send] to all the currently valid [receives] as patchers are loaded (and disconnect when patchers are unloaded) so that no lookup is required when messages are sent. Just one level of indirection in the call (like a pointer deref).
On the other hand, if it is necessary to look for every appropriate [receive] object every time a [send] is triggered, then that would get quite slow if you start to have hundreds of such connections and a lot of messages.
more cords through inlets and outlets? gets messy but can be made direct. Also, no reason to ever have more than one [receive] with the same name in a given patch, just run the cords from a main one. And if you have a ton of differently-named [receives], you can just use [prepend] at the [send], give it an identifier/index, and then [route] at the [receive]. This potentially eliminates all receives in a patch except one.
You can send messages and values with [thispatcher] and [js] too, which might help, but s/r is simplest.
Please, nobody take this personally.
See this old blog posting of mine (http://deskew.com/blog/why-bother.html) or to summarize, it is really not helpful when responders, who have no idea what the original questioner wants to do and/or their underlying motivation, don't address the question at hand and just tell them not do do (whatever they THINK) is being done.
Having gotten that out of the way, I would really appreciate it if someone with knowledge of the implementation of [send]/[receive] can help me understand how it was implemented, so that (a) I don't have to go measure it for myself and (b) so I can determine whether I or not I need to implement my own externals to implement a more efficient version of [send]/[receive] to suit my needs.
By the way, there are LOTS of reasons (including a few good ones) why one might want to have more than one [receive] with the same name in the patch, but that's not even what I want to do.
If you have the chops to write your own externals for this sort of thing, certainly you have the chops to do do a couple of simple measurements to see how long such message passing takes.
Sigh. I have the chops but I also have sufficient experience to know that
A) while it's easy to measure, it's a lot of work to set up (you need to create and wire up a lot of objects and different amounts of them, chagr names of some of them, so you can see how the speed changes based on an increasing number of objects with the same or different names.
More importantly,
B) I can avoid that effort entirely if someone else already knows the answer. As Mr. Spock once put it, "solving the same problem twice is inefficient"
I guess I didn't really understand what you were asking, so my apologies. However, I don't see a need to ever have multiple [receive] objects with the same name in the same patch, except for cosmetics and shorter patch cords. Enlighten me on why this might be helpful?
Measuring the efficiency is a different issue than simply trying to eliminate the problem of too many receives, which is what I offered and thought you might be looking for. So yes, it's not what you were asking, but it was a potential solution, though I imagine you knew it already...
I for one don't have the chops to roll my own externals to make this more efficient, nor has it ever been an issue with processing efficiency in my experience---plenty of other efficiency areas to deal with that have much greater impact. But if it does bog down with many instances of the s/r objects, maybe it's something that can be looked into.
Referring to the blog mentioned above, I have to admit that using a huge number of s/r objects across different patches doesn't fit into my mental model of programming. So I would like to understand in which situations this may be useful or even necessary.
So many different reasons.
1) In the same patcher window, they're quite useful to avoid spagetti wiring when you have several logically separate blocks. Eventually, each of those blocks might live in a subpatcher but they still have to communicate
2) I use a large number of abstractions to encapsulate functionality and prevent there being too much "code" in a single patcher. I've seen patchers with many hundreds of objects in them and that's just too hard to manage. Proper abstractions and a good naming convention allows incredibly easy communication throughout the system without the need for lots of inlets and outlets and gazillions of wires
3) In my application that replaces Apple MainStage, I have a single top-level patcher that represents control surfaces for my hardware, and channel strips for mixing to my MOTU 828s. I can very easily create standalone separate patchers representing different "songs" (i.e, routing, required synths and so forth) and by use of send/receives, it becomes trivial to connect those patchers to the "master" for audio routing and control surface automation
4) My favorite is the ability to quickly create transformations on the data by simply creating a new object with some parameters and that's it. Take a look at this simple example. I've put everything in one patcher here but you'll appreciate that it become trivial to "insert"/"modify" behavior without spending time connecting wires up but it also becomes extremely convenient to temporarily load different patchers with predefined behavior. And depending on how you name things (for example, prefix with #0) you can have local or global messaging.
Now, if there's a lot of activity going on, MIDI notes, MIDI control messages and, more recently, some OSC stuff as well, my concern is making sure this stuff scales up without causing bottlenecks. In particular, if the receivers have to be looked up for EVERY event, then that's a potential bottleneck, particular if the lookups are linear. On the other hand, if the send/receive connections are made once whenever a patcher is loaded, then you could have as many of these things as you need and O(n) will be constant, which is what I would really like.
"but if the names of [send] and [receive] objects are "registered when a patcher is loaded, it would be possible to essentially preconnect each [send] to all the currently valid [receives] as patchers are loaded"
absolutely true, but that would be a another object. [receive] has a dynamic argument, and
there can be multiple send and receives, so that cant work this way.
if i am not wrong, the same is true for inlets/ootlets - a connection across levels also
differes from a "direct" connection. you can see that when you trace route a message
across levels - the step stops at the inlet object, then anpother step i required to send
the message from the inlet to the next object.
-110
WHAT would be another object? What does a dynamic argument have to do with anything? Initializing at patcher startup time just means that the send/receive objects do the work to connect themselves up as they are instantiated the first time (which is what happens when a patcher starts up). Nothing to prevent a receiver (or the forward object, by the way) from updating their connections if their argument gets changed.
I don't understand your point about multiple send and receives. Nor do I understand the comment about connections between inlets and outlets. What does that have to do with send/receive?
I'm still hoping someone can tell me how the current send/receive system is implemented!
----
absolutely true, but that would be a another object. [receive] has a dynamic argument, and
there can be multiple send and receives, so that cant work this way.
@djh
Thanks for the explanation. The benefits of flexible reconfiguration are quite convincing.
So I would also appreciate some info about the implementation of s/r.
I'm getting a better idea of it now, thanks for the additional explanation. I wasn't really talking about replacing multiple different s/r, there are plenty of reasons for having lots of them (though the prepend/route approach can eliminate some). I was more curious why one would have multiple instances of the same receive in the same patch, but as you said, it's very helpful for keeping things uncluttered. I suppose if one really wanted to reduce multiple identical receives, you could use [v] from the single [receive] and then have additional [v] objects around the patch, or have a [pattrstorage] for that patch if there are eventually subpatches made from the logic "blocks' of the interface. Of course, maybe this creates a similar bottleneck too.
Abstractions and modularity really do make things work beautifully, if they have sufficient time and planning up front. Usually I get spaghetti when I keep adding onto something without taking the time to step back and think "how can I make this new functionality a module, encapsulate it, etc.?" Instead, I just keep adding on "a little bit more" and then...well, I think we've probably all been there ;)
Yeah, hash tables are very efficient as long as the table size is chosen correctly and there aren't too many collisions. But if that lookup occurs upon arrival of every single message to a [send], then I wonder how much one would save if the connections were "cached".
What I'd give for a Max compiler :-)
Hi guys.
Here's my guess about how send/receive work - just guessing, because I never tried to reverse-engineer the system, I just stumbled onto some stack traces while debugging my objects. Anyway.
For every symbol associated with a send or a receive, a nobox object of the class "through" is created, and its address is stored in the s_thing field of the symbol. So if we have 10 [send foo] and 10 [receive foo], only one "through foo" exists (we can check this with the wonderful cnmat's [printit] object). When [send foo] receives a message, it hands it to its corresponding through object (I'd assume the through object's address is stored into the send object, but if for any reason it doesn't then it's just a lookup in the Max symbol hash table). The through object keeps a list (a linked list, or some sort of dynamic array) of every associated receive's outlet, and sends out the message from them one after another - it doesn't even have to explicitly call a method of receive, it just has to know the receive's object address in order to pass it to the outlet function.
I'll say it again - all this is just conjectural... but it makes sense to me, and it looks coherent with what I know. If all this is true, the whole system seems quite efficient to me...
hth
aa
Sigh --- that is precisely the question with which I started the thread. I know how it might best be done, but the question whether it was in fact done that way!
To reiterate ---- are the connections setup in advance, whenever a new SEND or RECEIVE object is created (or the name changed), or do they get looked up every time there's an event to be sent out?
-----
When [send foo] receives a message, it hands it to its corresponding through object (I'd assume the through object's address is stored into the send object, but if for any reason it doesn't then it's just a lookup in the Max symbol hash table).
I figured that C74 engineers monitor these forums, and had hoped they might respond --- I try not to bother their support directly unless I'm having a significant issue.
@ dhjdhjdhj
one of the reasons why the folks at c74 don't usually disclose too many inner details of their objects is they want to be free to change the implementation without advice. I might be wrong, but I don't expect them to tell you much about how send and receive work internally.
... but then, if you really want to be sure about how things go, why don't you code your own wireless communication system? I think I remember from some other post of yours that you are an experienced programmer - it would not be complicated making your own dhj.send/dhj.receive pair, with everything you need and nothing more inside...
good luck!
aa
Typically, apart from trade secret reasons, the main reason one doesn't disclose implementation details is so that people don't try to LEVERAGE those details, i.e, use non-approved function calls to get at something, because such things can stop working at any time.
Actually, it would be sufficient to know the Big O value.
I know it's not complicated but I'm damned if I want to spend my time reimplementing something that is already sufficiently optimized such that it doesn't warrent reimplementation. THAT IS WHY I ASKED THE ORIGINAL QUESTION! (Sorry for shouting but I'm getting tired of being told what I ought to do when in fact what I've been trying to do from the beginning is just get the answer to the question so that I can in fact determine what I ought/need to do)
You know what you definitely ought to do instead?
1 - being kinder with people spending their time trying to give answers to your questions. and
2 - understanding that this is a public forum, not your private email: therefore, the information you did not ask for might be interesting for someone else, or trigger discussions that go beyond your immediate need.
Anyway, sorry if the last paragraph in my reply has caused a significant loss of time, patience or whatever on your side. I won't make the mistake of answering you anymore.
Peace
aa
Other than responding with some frustration to unhelpful suggestions clearly aimed at me (and generally not containing information interesting to someone else), I do not see where I have been unkind. Indeed, (and specifically addressing your point (2) when people actually asked WHY, I responded with specific details (including taking the time to put together a Max patch to demonstrate) precisely because I deeply understand that there is value to others in going beyond just looking for my own answer. And I think that people found those responses very helpful.
It is never my intention to hurt anybody's feelings, but as Ricky Gervais has expressed so wonderfully:
"No one has the right to never be offended"
Regarding your point (1), I reiterate my earlier post
https://cycling74.com/forums/question-about-sendreceive-implementation-and-efficiency
By the way (and talking about discussions that go beyond immediate need), how did you implement the music notation in your bach project? Perhaps you can make one more mistake and answer this question (grin)
hey dhj.
I hold no grudge and I don't want to argue about why I think you've been unkind - although passages like "I'm getting tired of being told what I ought to do" don't sound exactly the most pleasant to me. But ok, wtf.
I really can't say much about how the layout algorithms work - try ask Daniele on the bach forum, he's the GUI guy, my job is the core of the system. What I can tell you is that everything is done from scratch, only relying upon the Max graphics API, it is font-based as much as possible, and everything else is vector graphics - nothing is bitmap.
If you have more specific questions we'll be glad to answer them - although it might happen that we add some unsolicited information ;)
Peace
aa
I would kill for the ability to see and edit music notation in Max but without using Java
Yes, you have clarified what you want to know, and I can appreciate your frustrations in not getting the answers (so far, anyway). But there are plenty of ways to ask for what you want and to express your frustration. For me, even writing "Sigh" in an answer sounds patronizing, like "Sigh...another amateurish, ignorant poster wasting my time". Even if that wasn't your intent, I think it can easily be read that way. When you are asking for information and other people's time, which they are giving freely, that kind of response isn't going to win you many friends...regardless of how unhelpful their answer might be to you.
All that said, I'm ready to move on, and I add a big +1 for the music notation in Max. But as we musicians know, it ain't like piano-roll, which so easily connects with numerical data...too many pesky rules going on in scoring to be at all straightforward.
I'm not sure I see your point about music notation in Max... in principle, this is exactly what bach is aimed at. What features do you think are lacking?
And there is Maxscore as well, for a more graphical and less data-oriented approach. (btw, Maxscore is based upon Java, whereas bach is completely written in C, not a single line of Java).
And let's not forget InScore, though it's less interactive and not exactly Max (but if you want awesome graphical quality, that's the way to go).
What's that you guys can't achieve with any of these tools? I'm very interested in this...
Well, first of all, I had not heard of bach until you started bitching about my posts and I went to see who you were (grin). It looks very interesting but I wasn't in a position to look at it in any depth the other day.
I've looked at Maxscore before but (a) found it buggy and (b) far too heavy for my needs. The fact that it was in Java was a downside as well.
I don't need awesome quality. I use the notation editors in Digital Performer, Logic Pro, and Sibelius (the last of which I dislike immensely from a usability perspective) for editing full songs when I need that kind of quality.
However, when I'm playing live, I sometimes have little sequences or single-key triggered chords that get used from time to time in some songs (think Pink Floyd AKS Synthi loops or Peter Gabriel Fairlight Page S stuff). Right now, I create those loops with DP or Logic, save them as MIDI files and import them into Max. I also use that really nice [ntom] object that I found somewhere (never could find the full library package that went with that BTW) to allow me to define single chords in a messagebox.....e.g. [play A3 C#4 E4]
I'd like to be able to do several things.
1) During performance, I'd love to have a really lightweight (so I could have lots of them) Max object that could quickly (and with no impact on anything going on in real-time) display those sequences, fed from either a loaded MIDI file, a 'coll' or table containing some notes, or from a messagebox. But it needs to be good enough to display rests and deal with MIDI data that is not perfectly quantized and still give a reasonable view
2) When creating these sequences (or fixing them quickly before a performance, say), I'd like to be able to insert notes, and click on existing notes and change their pitch, duration, or relative position in the score.
3) Lower priority but would be nice - as a MIDI file is playing, I'd like to see the currently playing note or chord highlighted.
There are other things related to displaying notes while actually playing but that's mostly useful for diagnostics.
i guess andrea should answer this, but try bach. I am nearly sure that you can do all of this with it. It's a great project!
Thanks Andrea!
It wasn't clear to me how "integrated" the scoring part was with the rest of the bach system and I haven't had a chance to look at it. It's on my list.
D
P.S. I still want to know how send/receive is implemented (grin)
With the current version of bach, you can't import MIDI files. This is one of the missing key features yet, but we're working on that. We hope we'll be able to issue a new version before long, with MIDI support and much more.
For the rest, you basically can do all you need - having your scores in message boxes, or dynamically load them from disk, or even have them stored into a preset object. And yes, you can edit them on the fly and have chords turning green as they're played. And rests are displayed. And much more.
About send/receive, what can I say? good luck... In my own patch, I'd take for good the mechanism I have described above (calling it "a guess" was a bit understated...) - in which case, O(n). But we're talking about your patch...
"With the current version of bach, you can't import MIDI files."
But it's not too hard to setup a simple system for quantization/conversion that just sends the right commands to a bach.score object within max, and there you go! definitely worth a try
Yes, clearly I would have to put some infrastructure around any such object...that's fine. If this bach.score object can be used independently of anything else, than it begins to sound very exciting, I will check it out.
Actually, bach includes a quantization object, bach.quantize, with a lot of parameters to tweak.
So yes, what you can do is building your own MIDIfile reader (probably built around detonate) import the sequence into bach.roll, then quantize it and give it to bach.score. It definitely can be done, and it doesn't even look too complicated...
I'm looking forward to it ---- downloading versions for Max 5 & 6 right now and will play with it in between my other activities today.