timing of jit.qt.grab when reading and writing matrixes

one more thing: it seems to be related to the jit.qt.grab object too: if i use a 50 fps camera, i end up with 25 fps. if i use a 25 fps camera, i end up with 12,5.
best
k

i just spent another day trying to understand this and actually, the mac os activity monitor (disk tab) does show the problem: it's not the data rates but the "reads in/sec":

- the "writes out/sec" are as expected and stay at the speed coming from the metro (50 fps = 50 writes/sec).

- the "reads in/sec" however, go bonkers: if i only read and not write my .jxf. files, they are between 0 (i think this is when automatic macos ram caching is at work) to a maximum of 500. to my understanding it should be 50 (since i read at 50 fps) but regardless the patch works fine at a solid 50 fps.

- but when i write to disk (and even if i only write and do not read!), "reads in/sec" shoots up to up to 8500 - which is when my patch slows down.

- all this is exclusively caused by sending "write" to a jit.matrix 50 times/sec. if i gate the write-command temporarily, the "reads in/sec" in the activity monitor instantly drop to 0 (or the 500 tops of the reading process when enabled).

- as well i tried to first cache ten frames to a jit.matrixset and then writing this to disk so i would get less write operations and bigger files but that makes things only worse: same problem as above but overall lower performance and strong stutter at regular intervals.

any ideas anyone????
k

come on guys, don't let me sit here alone in the dark, sobbing quietly.... someone has to know something about this!
is this a bug and i should file for support?!?

some more observations:
- as mentioned above, the problem only happens when overwriting .jxf-files but it does not always happen when doing so. sometimes the files get overwritten smoothly at 50fps and i do not have any "Reads in/sec", only the expected 50 "Writes out/sec". other times the reads shoot up to a couple of thousands as described above, slowing my patch down.

- i can't make it happen on purpose however: running the patch, it occurs for sure after a while but i can't track when overwriting happens smoothly and when not. it just starts randomly while running (me not touching anything): slows down for a couple of minutes, then speeds up again, then slows down again etc.

- i tried that on different mac systems and with max 5 and 6. the result is always the same.

- as well i tried to emulate my overwriting pattern with other programs but did not get anything close to this behaviour. it really seems to be max specific.

pleeeeeaaasssse...!

doesn't sound like there's much else you can do besides lower you FPS.
why 50 instead of 30?

if the problem only occurs when overwriting then why do you need to overwrite?

you're doing something that probably not a lot of people have tried before, and probably pushing the limits of your machine.

hello rob, so many thanks for answering!!

the machine is pretty powerful and easily capable of doing what i need - if there would be the 50 reads/sec i request and not the 8000+ max seems to generate randomnly when doing a simple overwrite. as well i can easily capture and play back several high def video streams simultaneously with other programs and my patch only uses 720p grayscale.
if i run my patch with a thirty frame camera, i get 15 in the end, it's just all the same - when overwriting starts, max goes bonkers after some time.

do you have any idea or educated guess about what might be causing this? where these thousands of reads/sec come from?

Hi, Karl

I've just read this and I see you mention having a SSD so a thought jumped to my mind.
I am not sure which ssd you have and how this one handles this exactly, but:

SSD's work in a way that you cannot simply overwrite single files. Instead all data in a certain block (the one including the file to be overwritten) has to be erased and rewritten with the new / now changed information. All depending on the situation.
see: http://www.anandtech.com/show/2738/5

Perhaps this is why you see a drop in FPS (and a spike in your disk I/O) the moment you start overwriting your frames.

thanks for the link - it's quite an interesting read but definitely not related to my problem.
again: it's not the hardware, it's max. i can run similar in type but way more demanding tasks on this machine with no sweat whatsoever.

it is some weird thing going on inside max, or more specifically the write message sent to a jit.matrix generating all these "reads" at seemingly random times (because as outlined above, sometimes it does work as expected at least for some time).

???

Hi karl,
I've got some experience with writing and reading files, for a perforamce called KKQQ ( http://2bcompany.ch/2bcompany/kkqqvideo.html ). I'had to record three different streams of video (WI-FI streams) and three streams of sound (one by actor) on SSD. The difficulty was each one played his partition at a different rate, the first one played in front of the camera at half the speed, the second one 0.66 the speed and the third at 0.75 the speed. When the performance began they were still playing in their boxes since 45min for the first, 30min for the second and 15min for the third. The performance began with the projection and they appear synchronized, technically the first plays at the rate of 2, the second at a rate of 1.66666 and the third at 1.33333. Voices are pitched even if they've tried to speak with lower voices (& slow motion gestures) at recording. So as a result at the beginnig of the performance I'm recording 3 video streams, 3 sound streams ( because they continue to record during the whole play until the projection catch the live action after 45min) and reading 3 video streams and 3 sound streams. I've recorded on little movie each 300 frames (something like 210 movie per actor) same for sound. Naming (001.mov -> 210.mov ), recording and writing was automatic after the 300 frames there was a switch to a second recorder ready to record. On the other hand for projection there were two players with selecting movies or sound, reading and playing in a similar automatic way with a switch betwen the players.

1. I've tried recording matrices but i've found that it was better to record little movies or chunks to avoid write and read operations (~ 2500 cumulated in a 90 minute perf between read and write are a lot i thought.)
2. I've forced the jit.qt.record to grab images to be synchronized with the sound with the little attached patch and these msp setting (48000khz with I/O vector size and signal vector size both set to 128 overdrive on). That give me one bang exactly every 40ms. In the other hand the projection was handled with a jit.qball after the little patch because forcing with the samee clock freeze Max and the little décalages or delays (glitch) were found funny by the audience.

Hope it can help you

Filippo

ok Msp at 48khz .-)