Forums > Java

Crash in JitterMatrix.copyMatrixToArray() ?

February 5, 2008 | 1:54 pm

Actually, an abort().

Here’s the most relevant part of the Crash Log, I’ve attached the whole thing for the curious. I expect lines 8-10 are from inside my mxj code (also attached, it’s presumably the jit_matrix method).

Any ideas? Again, the full gory details in the attachment.

Thread 0 Crashed:
0 libSystem.B.dylib 0x9588047a __kill + 10
1 libSystem.B.dylib 0x958f7782 raise + 26
2 libSystem.B.dylib 0x95906d3f abort + 73
3 libjvm.dylib 0x16c4bb65 JVM_RaiseSignal + 436741
4 libjvm.dylib 0x16b7af92 pss + 203970
5 libjvm.dylib 0x16b8fcdd JNI_CreateJavaVM_Impl + 71325
6 com.cycling74.JitterAPI 0x163105f3 jit_java_jitmatrix_copymatrixtoarray + 233
7 com.cycling74.JitterAPI 0x163108fb jit_java_jitmatrix_copymatrixtoarray_int + 49
8 ??? 0x1937189d 0 + 423041181
9 ??? 0x1936dc74 0 + 423025780
10 ??? 0x192b9227 0 + 422285863
11 libjvm.dylib 0x16aa163a 0×16988000 + 1152570
12 libjvm.dylib 0x16aa1356 0×16988000 + 1151830
13 libjvm.dylib 0x16a2b68c 0×16988000 + 669324
14 libjvm.dylib 0x16b841ab JNI_CreateJavaVM_Impl + 23403
15 com.cycling74.mxj 0x169543d4 call_method_with_coercion + 3624
16 com.cycling74.mxj 0×16954579 maxjava_anything + 344
17 …diaSetup_01b03.appRuntime46 0x0001f25c typedmess_fun + 512 (message.c:492)


February 5, 2008 | 6:16 pm

On Feb 5, 2008, at 5:54 AM, Peter Castine wrote:

> Actually, an abort().
>
> Here’s the most relevant part of the Crash Log, I’ve attached the
> whole thing for the curious. I expect lines 8-10 are from inside my
> mxj code (also attached, it’s presumably the jit_matrix method).
>
> Any ideas? Again, the full gory details in the attachment.

Can’t reproduce here. Can you give us real repro steps? Something
else corrupting memory?

Worst case, you might switch to the copyVectorToArray() methods,
since those functions have been more heavily tested and this
copyMatrixToArray() routine might have some issues (if discovered,
none of which would be fixed before Max 5).

-Joshua


February 5, 2008 | 7:55 pm

I’ll get you a copy of the app we’ve built around my mxj Object. I need to get a current version from John Dekron. It’ll probably be tomorrow.

Thanks for taking the time to look at what I posted so far.

FWIW, I was originally using getCell2d() calls, but they ate up CPU pretty seriously.

– P.


February 5, 2008 | 8:52 pm

On Feb 5, 2008, at 11:55 AM, Peter Castine wrote:

>
> I’ll get you a copy of the app we’ve built around my mxj Object. I
> need to get a current version from John Dekron. It’ll probably be
> tomorrow.
>
> Thanks for taking the time to look at what I posted so far.
>
> FWIW, I was originally using getCell2d() calls, but they ate up
> CPU pretty seriously.

As mentioned before, you might try copyVectorToArray(). This row by
row copy is way faster than getCell2d() approach, and shouldn’t be so
much slower than a copyMatrixToArray() approach. On first glance the
copyMatrixToArray() JNI code looks fine to me, but given your
crashlog, maybe there’s some problems for some cases in this
function. I’m pretty sure that the copyVectorToArray() method is
stable. That is unless this whole thing is the result of something
else trashing memory.

-Joshua


February 6, 2008 | 3:51 pm

OK, I’ll try the copyVector… route.

If you have a machine with some time on its hands, the following steps ought to reproduce the error.

0) Open the ‘jitSendPixm test2′ Patch I uploaded yesterday

1) (Optional) delete the patch cord to the left inlet of [mxj jitSendPixm]

(jit_matrix in inlet 0 triggers the TCP exchange with the PixelMaster hardware. The only PixelMasters are in Europe behind firewalls. If you leave the cord connected, you get zillions of •errors in the Max window but they’re cosmetic.)

2) Start the qmetro and let it run.

It can take a couple of hours to crash, but our app needs to run 24/7 unattended.

Thanks — Peter


February 6, 2008 | 4:53 pm

On Feb 6, 2008, at 7:51 AM, Peter Castine wrote:
>
> It can take a couple of hours to crash, but our app needs to run
> 24/7 unattended.

Based on this and looking at activity monitor, there’s a pretty big
memory leak going on here. Shuffle shuffle, using shark malloc trace,
it looks like something is up with copyMatrixToArray. I would suggest
making your own function which calls copyVectorToArray() with one row
size vector always and then copy to a whole matrix in Java. Let me
know if that approach still has similar memory leaks. Will fix the
copyMatrixToArray() issue in a future release.

-Joshua


February 6, 2008 | 8:31 pm

FWIW, upon further investigation, the memory leak only happens if the
array is not matched in size to the matrix size (product of all
dimensions). So the issue in your example is using one matrixbuf for
all inputs where the last input is not the same size as your array
and you leak memory. If you disconnect the last input which is a
different size than all the others, the memleak disappears (this
matrix doesn’t get copied to the buffer anyways either if the size
doesn’t match).

I’ve fixed this for the next version of Jitter (Max 5 only), so for
now, if you want to use these whole matrix at a time methods you’ll
want to make sure that the sizes match or else you’ll mem leak.
Otherwise use the recommended vector methods which should not be that
much slower if at all.

Thanks for finding this bug.

-Joshua


February 6, 2008 | 9:34 pm

We’re testing the copyVectorToArray() version now.

It took me a while to get the incantation right. For the benefit of anyone looking for a matrix-to-array conversion, here’s my code to replace copyMatrixToArray()

==========
JitterMatrix jm = new JitterMatrix(…);
int[] offset = new int[2]; // hard-wired to 2D matrices
int[] matrixBuf = new int[/*whatever it takes*/];
int[] dim = jm.getDim();

offset[0] = 0;
for (int j = 0; j < dim[1]; j++) {
offset[1] = j;
jm.copyVectorToArray(0, offset, matrixBuf, 4 * dim[0], 4 * dim[0] * j);
}
===========

Our app only deals with ARGB-plane matrices, a more general implementation would replace the constant 4 in the code with the
current plane count. The same applies to the hard-wired 2D thing.

The modification has worked for as long as I’ve spent writing this message. If it’s still running after supper I’ll be more confident.-

– P.


February 7, 2008 | 9:22 am

At my end the good news is that the copyVectorToArray() version ran happily overnight and still has a constant memory footprint. It’s running and performance is fine.

I had originally matched the matrixbuf to the size of the incoming matrix by creating a new int[dim[0]*dim[1]*planecount] on each call to jit_matrix. But I found that approach takes a significant performance hit because GC gets called far too often. I also think that had an issue with cumulative memory fragmentation, but I’m not sure.

Maintaining a single array large enough to hold the largest of the eight incoming matrices seemed to make a lot of sense. At least it works now. I hope.


February 7, 2008 | 8:37 pm

On 06 Feb 2008, at 21:31, Joshua Kit Clayton wrote:
>
> FWIW, upon further investigation, the memory leak only happens if the
> array is not matched in size to the matrix size (product of all
> dimensions).
>
> I’ve fixed this for the next version of Jitter (Max 5 only), so for
> now, if you want to use these whole matrix at a time methods you’ll
> want to make sure that the sizes match or else you’ll mem leak.
> Otherwise use the recommended vector methods which should not be that
> much slower if at all.

hmm, interesting. i didn’t know about copyArrayToMatrix() – must have slipped in unnoticed somewhere inbetween.
i was using copyArrayToVector() before and it works fine, as you said.
on a speed comparison however, i was happy to notice that copyArrayToMatrix() is much faster than the vector method – at least in my situation. cpu usage came down half way for a 100×100 matrix (measured with top).

BUT, i can’t escape the memory leak even if the array and the matrix match in size!

in the simplified example below, memory is still leaking. where am i going wrong?

any help is welcome, since the speed increase could be crucial to the project i am working on.
thanks,
volker.

import com.cycling74.max.*;
import com.cycling74.jitter.*;

public class LeakerTest extends MaxObject
{

int x = 100;
int y = 100;
int size = x * y;
private int[] intArr = new int[size];
JitterMatrix mat = new JitterMatrix("mm", 1, "char", x, y);

public LeakerTest (Atom[] args)
{
declareInlets(new int[]{DataTypes.ALL});
fill(100);
}

public void bang()
{
mat.copyArrayToMatrix( intArr );
}

public void fill(int k)
{
for(int i=0; i
intArr[i] = k;
}
}

GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail


February 7, 2008 | 10:07 pm

On Feb 7, 2008, at 12:37 PM, vboehm@gmx.ch wrote:
>
> hmm, interesting. i didn’t know about copyArrayToMatrix() – must
> have slipped in unnoticed somewhere inbetween.
> i was using copyArrayToVector() before and it works fine, as you said.
> on a speed comparison however, i was happy to notice that
> copyArrayToMatrix() is much faster than the vector method – at
> least in my situation. cpu usage came down half way for a 100×100
> matrix (measured with top).
>
> BUT, i can’t escape the memory leak even if the array and the
> matrix match in size!
> in the simplified example below, memory is still leaking. where am
> i going wrong?
> any help is welcome, since the speed increase could be crucial to
> the project i am working on.

Sorry, copyArrayToMatrix() *always* leaks in Jitter 1.6.x. It is
copyMatrixToArray() which has no memory leak if matched in size.
These methods were added later in the development cycle and were
obviously not carefully tested w/r/t memory leaks. Solution for both
in the next version for Max 5 only, but that doesn’t really help much
for now. Sorry about that.

The one thing you might be able to hack together now is the creative
use of one large array being sent into a single row matrix to
jit.scanwrap to wrap that long row into a 2d matrix.

-Joshua


February 8, 2008 | 12:57 pm

On 07 Feb 2008, at 23:07, Joshua Kit Clayton wrote:

>
> On Feb 7, 2008, at 12:37 PM, vboehm@gmx.ch wrote:
>>
>> hmm, interesting. i didn’t know about copyArrayToMatrix() – must
>> have slipped in unnoticed somewhere inbetween.
>> i was using copyArrayToVector() before and it works fine, as you
>> said.
>> on a speed comparison however, i was happy to notice that
>> copyArrayToMatrix() is much faster than the vector method – at
>> least in my situation. cpu usage came down half way for a 100×100
>> matrix (measured with top).
>>
>> BUT, i can’t escape the memory leak even if the array and the
>> matrix match in size!
>> in the simplified example below, memory is still leaking. where am
>> i going wrong?
>> any help is welcome, since the speed increase could be crucial to
>> the project i am working on.
>
> Sorry, copyArrayToMatrix() *always* leaks in Jitter 1.6.x. It is
> copyMatrixToArray() which has no memory leak if matched in size.

ah, my bad. didn’t get you were talking about the other direction only.

>
> The one thing you might be able to hack together now is the
> creative use of one large array being sent into a single row matrix
> to jit.scanwrap to wrap that long row into a 2d matrix.

that’s a good one!
a quick test shows not as good results as with copyArrayToMatrix(),
but it’s significantly faster than calling copyArrayToVector() for
each row in the output image.
thanks, would have never thought of that myself.
volker.


Viewing 12 posts - 1 through 12 (of 12 total)