Peter Castine

Here's the most relevant part of the Crash Log, I've attached the whole thing for the curious. I expect lines 8-10 are from inside my mxj code (also attached, it's presumably the jit_matrix method).

Any ideas? Again, the full gory details in the attachment.

0 libSystem.B.dylib     0x9588047a __kill + 10

1 libSystem.B.dylib     0x958f7782 raise + 26

2 libSystem.B.dylib     0x95906d3f abort + 73

3 libjvm.dylib     0x16c4bb65 JVM_RaiseSignal + 436741

4 libjvm.dylib     0x16b7af92 pss + 203970

5 libjvm.dylib     0x16b8fcdd JNI_CreateJavaVM_Impl + 71325

6 com.cycling74.JitterAPI     0x163105f3 jit_java_jitmatrix_copymatrixtoarray + 233

7 com.cycling74.JitterAPI     0x163108fb jit_java_jitmatrix_copymatrixtoarray_int + 49

11 libjvm.dylib     0x16aa163a 0x16988000 + 1152570

12 libjvm.dylib     0x16aa1356 0x16988000 + 1151830

13 libjvm.dylib     0x16a2b68c 0x16988000 + 669324

14 libjvm.dylib     0x16b841ab JNI_CreateJavaVM_Impl + 23403

15 com.cycling74.mxj     0x169543d4 call_method_with_coercion + 3624

16 com.cycling74.mxj     0x16954579 maxjava_anything + 344

17 ...diaSetup_01b03.appRuntime46    0x0001f25c typedmess_fun + 512 (message.c:492)

Here's the most relevant part of the Crash Log, I've attached the whole thing for the curious. I expect lines 8-10 are from inside my mxj code (also attached, it's presumably the jit_matrix method). 

Thread 0 Crashed:
0   libSystem.B.dylib             	0x9588047a __kill + 10
1   libSystem.B.dylib             	0x958f7782 raise + 26
2   libSystem.B.dylib             	0x95906d3f abort + 73
3   libjvm.dylib                  	0x16c4bb65 JVM_RaiseSignal + 436741
4   libjvm.dylib                  	0x16b7af92 pss + 203970
5   libjvm.dylib                  	0x16b8fcdd JNI_CreateJavaVM_Impl + 71325
6   com.cycling74.JitterAPI       	0x163105f3 jit_java_jitmatrix_copymatrixtoarray + 233
7   com.cycling74.JitterAPI       	0x163108fb jit_java_jitmatrix_copymatrixtoarray_int + 49
8   ???                           	0x1937189d 0 + 423041181
9   ???                           	0x1936dc74 0 + 423025780
10  ???                           	0x192b9227 0 + 422285863
11  libjvm.dylib                  	0x16aa163a 0x16988000 + 1152570
12  libjvm.dylib                  	0x16aa1356 0x16988000 + 1151830
13  libjvm.dylib                  	0x16a2b68c 0x16988000 + 669324
14  libjvm.dylib                  	0x16b841ab JNI_CreateJavaVM_Impl + 23403
15  com.cycling74.mxj             	0x169543d4 call_method_with_coercion + 3624
16  com.cycling74.mxj             	0x16954579 maxjava_anything + 344
17  ...diaSetup_01b03.appRuntime46	0x0001f25c typedmess_fun + 512 (message.c:492)


crash-in-jittermatrix-copymatrixtoarray

On Feb 5, 2008, at 5:54 AM, Peter Castine wrote:

> Here's the most relevant part of the Crash Log, I've attached the 

> whole thing for the curious. I expect lines 8-10 are from inside my 

> mxj code (also attached, it's presumably the jit_matrix method).

> Any ideas? Again, the full gory details in the attachment.

Can't reproduce here. Can you give us real repro steps? Something 

Worst case, you might switch to the copyVectorToArray() methods, 

since those functions have been more heavily tested and this 

copyMatrixToArray() routine might have some issues (if discovered, 

none of which would be fixed before Max 5).

> Actually, an abort().
>
> Here's the most relevant part of the Crash Log, I've attached the  
> whole thing for the curious. I expect lines 8-10 are from inside my  
> mxj code (also attached, it's presumably the jit_matrix method).
>
> Any ideas? Again, the full gory details in the attachment.

Can't reproduce here. Can you give us real repro steps? Something  
else corrupting memory?

Worst case, you might switch to the copyVectorToArray() methods,  
since those functions have been more heavily tested and this  
copyMatrixToArray() routine might have some issues (if discovered,  
none of which would be fixed before Max 5).

I'll get you a copy of the app we've built around my mxj Object. I need to get a current version from John Dekron. It'll probably be tomorrow.

Thanks for taking the time to look at what I posted so far.

FWIW, I was originally using getCell2d() calls, but they ate up CPU pretty seriously.

FWIW, I was originally using getCell2d() calls, but they ate up  CPU pretty seriously.   

On Feb 5, 2008, at 11:55 AM, Peter Castine wrote:

> I'll get you a copy of the app we've built around my mxj Object. I 

> need to get a current version from John Dekron. It'll probably be 

> Thanks for taking the time to look at what I posted so far.

> FWIW, I was originally using getCell2d() calls, but they ate up 

As mentioned before, you might try copyVectorToArray(). This row by 

row copy is way faster than getCell2d() approach, and shouldn't be so 

much slower than a copyMatrixToArray() approach. On first glance the 

copyMatrixToArray() JNI code looks fine to me, but given your 

crashlog, maybe there's some problems for some cases in this 

function. I'm pretty sure that the copyVectorToArray() method is 

stable. That is unless this whole thing is the result of something 

>
> I'll get you a copy of the app we've built around my mxj Object. I  
> need to get a current version from John Dekron. It'll probably be  
> tomorrow.
>
> Thanks for taking the time to look at what I posted so far.
>
> FWIW, I was originally using getCell2d() calls, but they ate up   
> CPU pretty seriously.

As mentioned before, you might try copyVectorToArray(). This row by  
row copy is way faster than getCell2d() approach, and shouldn't be so  
much slower than a copyMatrixToArray() approach. On first glance the  
copyMatrixToArray() JNI code looks fine to me, but given your  
crashlog, maybe there's some problems for some cases in this  
function. I'm pretty sure that the copyVectorToArray() method is  
stable. That is unless this whole thing is the result of something  
else trashing memory.

If you have a machine with some time on its hands, the following steps ought to reproduce the error.

0) Open the 'jitSendPixm test2' Patch I uploaded yesterday

1) (Optional) delete the patch cord to the left inlet of [mxj jitSendPixm]

(jit_matrix in inlet 0 triggers the TCP exchange with the PixelMaster hardware. The only PixelMasters are in Europe behind firewalls. If you leave the cord connected, you get zillions of •errors in the Max window but they're cosmetic.)

It can take a couple of hours to crash, but our app needs to run 24/7 unattended.

On Feb 6, 2008, at 7:51 AM, Peter Castine wrote:

> It can take a couple of hours to crash, but our app needs to run 

Based on this and looking at activity monitor, there's a pretty big 

memory leak going on here. Shuffle shuffle, using shark malloc trace, 

it looks like something is up with copyMatrixToArray. I would suggest 

making your own function which calls copyVectorToArray() with one row 

size vector always and then copy to a whole matrix in Java. Let me 

know if that approach still has similar memory leaks. Will fix the 

copyMatrixToArray() issue in a future release.

On Feb 6, 2008, at 7:51 AM, Peter Castine wrote:
>
> It can take a couple of hours to crash, but our app needs to run  
> 24/7 unattended.

Based on this and looking at activity monitor, there's a pretty big  
memory leak going on here. Shuffle shuffle, using shark malloc trace,  
it looks like something is up with copyMatrixToArray. I would suggest  
making your own function which calls copyVectorToArray() with one row  
size vector always and then copy to a whole matrix in Java. Let me  
know if that approach still has similar memory leaks. Will fix the  
copyMatrixToArray() issue in a future release.

FWIW, upon further investigation, the memory leak only happens if the 

array is not matched in size to the matrix size (product of all 

dimensions). So the issue in your example is using one matrixbuf for 

all inputs where the last input is not the same size as your array 

and you leak memory. If you disconnect the last input which is a 

different size than all the others, the memleak disappears (this 

matrix doesn't get copied to the buffer anyways either if the size 

I've fixed this for the next version of Jitter (Max 5 only), so for 

now, if you want to use these whole matrix at a time methods you'll 

want to make sure that the sizes match or else you'll mem leak. 

Otherwise use the recommended vector methods which should not be that 

FWIW, upon further investigation, the memory leak only happens if the  
array is not matched in size to the matrix size (product of all  
dimensions). So the issue in your example is using one matrixbuf for  
all inputs where the last input is not the same size as your array  
and you leak memory. If you disconnect the last input which is a  
different size than all the others, the memleak disappears (this  
matrix doesn't get copied to the buffer anyways either if the size  
doesn't match).

I've fixed this for the next version of Jitter (Max 5 only), so for  
now, if you want to use these whole matrix at a time methods you'll  
want to make sure that the sizes match or else you'll mem leak.  
Otherwise use the recommended vector methods which should not be that  
much slower if at all.

We're testing the copyVectorToArray() version now.

It took me a while to get the incantation right. For the benefit of anyone looking for a matrix-to-array conversion, here's my code to replace copyMatrixToArray()

int[] offset = new int[2]; // hard-wired to 2D matrices

int[] matrixBuf = new int[/*whatever it takes*/];

jm.copyVectorToArray(0, offset, matrixBuf, 4 * dim[0], 4 * dim[0] * j);

Our app only deals with ARGB-plane matrices, a more general implementation would replace the constant 4 in the code with the 

current plane count. The same applies to the hard-wired 2D thing.

The modification has worked for as long as I've spent writing this message. If it's still running after supper I'll be more confident.-

==========
JitterMatrix jm = new JitterMatrix(...);
int[] offset = new int[2]; // hard-wired to 2D matrices
int[] matrixBuf = new int[/*whatever it takes*/];
int[] dim = jm.getDim();

offset[0] = 0;
for (int j = 0; j < dim[1]; j++) {
  offset[1] = j;
  jm.copyVectorToArray(0, offset, matrixBuf, 4 * dim[0], 4 * dim[0] * j);
  }
===========

Our app only deals with ARGB-plane matrices, a more general implementation would replace the constant 4 in the code with the 
current plane count. The same applies to the hard-wired 2D thing.

At my end the good news is that the copyVectorToArray() version ran happily overnight and still has a constant memory footprint. It's running and performance is fine.

I had originally matched the matrixbuf to the size of the incoming matrix by creating a new int[dim[0]*dim[1]*planecount] on each call to jit_matrix. But I found that approach takes a significant performance hit because GC gets called far too often. I also think that had an issue with cumulative memory fragmentation, but I'm not sure.

Maintaining a single array large enough to hold the largest of the eight incoming matrices seemed to make a lot of sense. At least it works now. I hope.

I had originally matched the matrixbuf to the size of the incoming matrix by creating a new int[dim[0]*dim[1]*planecount] on each call to jit_matrix. But I found that approach takes a significant performance hit because GC gets called far too often. I also think that had an issue with cumulative memory fragmentation, but I'm not sure. 

Maintaining a single array large enough to hold the largest of the eight incoming matrices seemed to make a lot of sense. At least it works now. I hope.


On 06 Feb 2008, at 21:31, Joshua Kit Clayton wrote:

> FWIW, upon further investigation, the memory leak only happens if the 

> array is not matched in size to the matrix size (product of all 

> I've fixed this for the next version of Jitter (Max 5 only), so for 

> now, if you want to use these whole matrix at a time methods you'll 

> want to make sure that the sizes match or else you'll mem leak. 

> Otherwise use the recommended vector methods which should not be that 

hmm, interesting. i didn't know about copyArrayToMatrix() - must have slipped in unnoticed somewhere inbetween.

i was using copyArrayToVector() before and it works fine, as you said. 

on a speed comparison however, i was happy to notice that copyArrayToMatrix() is much faster than the vector method - at least in my situation. cpu usage came down half way for a 100x100 matrix (measured with top).

BUT, i can't escape the memory leak even if the array and the matrix match in size!

in the simplified example below, memory is still leaking. where am i going wrong?

any help is welcome, since the speed increase could be crucial to the project i am working on.

public class LeakerTest extends MaxObject 

    private int[] intArr = new int[size];    

    JitterMatrix mat = new JitterMatrix("mm", 1, "char", x, y);

        declareInlets(new int[]{DataTypes.ALL});

        mat.copyArrayToMatrix( intArr );

GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.

On 06 Feb 2008, at 21:31, Joshua Kit Clayton wrote:
>
> FWIW, upon further investigation, the memory leak only happens if the  
> array is not matched in size to the matrix size (product of all  
> dimensions). 
>
> I've fixed this for the next version of Jitter (Max 5 only), so for  
> now, if you want to use these whole matrix at a time methods you'll  
> want to make sure that the sizes match or else you'll mem leak.  
> Otherwise use the recommended vector methods which should not be that  
> much slower if at all.

hmm, interesting. i didn't know about copyArrayToMatrix() - must have slipped in unnoticed somewhere inbetween.
i was using copyArrayToVector() before and it works fine, as you said. 
on a speed comparison however, i was happy to notice that copyArrayToMatrix() is much faster than the vector method - at least in my situation. cpu usage came down half way for a 100x100 matrix (measured with top).

any help is welcome, since the speed increase could be crucial to the project i am working on.
thanks,
volker.

import com.cycling74.max.*;
import com.cycling74.jitter.*;

public class LeakerTest extends MaxObject 
{

	int x = 100;
	int y = 100;
	int size = x * y;
	private int[] intArr = new int[size];	
	JitterMatrix mat = new JitterMatrix("mm", 1, "char", x, y);

	public LeakerTest (Atom[] args)
	 {
		declareInlets(new int[]{DataTypes.ALL});
		fill(100);
	 }

	public void bang() 
	{			
		mat.copyArrayToMatrix( intArr );
	}

	public void fill(int k) 
	{
		for(int i=0; i
			intArr[i] = k;
	}
}
-- 
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail


On Feb 7, 2008, at 12:37 PM, vboehm@gmx.ch wrote:

> hmm, interesting. i didn't know about copyArrayToMatrix() - must 

> have slipped in unnoticed somewhere inbetween.

> i was using copyArrayToVector() before and it works fine, as you said.

> on a speed comparison however, i was happy to notice that 

> copyArrayToMatrix() is much faster than the vector method - at 

> least in my situation. cpu usage came down half way for a 100x100 

> BUT, i can't escape the memory leak even if the array and the 

> in the simplified example below, memory is still leaking. where am 

> any help is welcome, since the speed increase could be crucial to 

Sorry, copyArrayToMatrix() *always* leaks in Jitter 1.6.x. It is 

copyMatrixToArray() which has no memory leak if matched in size. 

These methods were added later in the development cycle and were 

obviously not carefully tested w/r/t memory leaks. Solution for both 

in the next version for Max 5 only, but that doesn't really help much 

The one thing you might be able to hack together now is the creative 

use of one large array being sent into a single row matrix to 

jit.scanwrap to wrap that long row into a 2d matrix.

On Feb 7, 2008, at 12:37 PM, vboehm@gmx.ch wrote:
>
> hmm, interesting. i didn't know about copyArrayToMatrix() - must  
> have slipped in unnoticed somewhere inbetween.
> i was using copyArrayToVector() before and it works fine, as you said.
> on a speed comparison however, i was happy to notice that  
> copyArrayToMatrix() is much faster than the vector method - at  
> least in my situation. cpu usage came down half way for a 100x100  
> matrix (measured with top).
>
> BUT, i can't escape the memory leak even if the array and the  
> matrix match in size!
> in the simplified example below, memory is still leaking. where am  
> i going wrong?
> any help is welcome, since the speed increase could be crucial to  
> the project i am working on.

Sorry, copyArrayToMatrix() *always* leaks in Jitter 1.6.x. It is  
copyMatrixToArray() which has no memory leak if matched in size.  
These methods were added later in the development cycle and were  
obviously not carefully tested w/r/t memory leaks. Solution for both  
in the next version for Max 5 only, but that doesn't really help much  
for now. Sorry about that.

The one thing you might be able to hack together now is the creative  
use of one large array being sent into a single row matrix to  
jit.scanwrap to wrap that long row into a 2d matrix.

On 07 Feb 2008, at 23:07, Joshua Kit Clayton wrote:

> On Feb 7, 2008, at 12:37 PM, vboehm@gmx.ch wrote:

>> hmm, interesting. i didn't know about copyArrayToMatrix() - must 

>> have slipped in unnoticed somewhere inbetween.

>> i was using copyArrayToVector() before and it works fine, as you 

>> on a speed comparison however, i was happy to notice that 

>> copyArrayToMatrix() is much faster than the vector method - at 

>> least in my situation. cpu usage came down half way for a 100x100 

>> BUT, i can't escape the memory leak even if the array and the 

>> in the simplified example below, memory is still leaking. where am 

>> any help is welcome, since the speed increase could be crucial to 

> Sorry, copyArrayToMatrix() *always* leaks in Jitter 1.6.x. It is 

> copyMatrixToArray() which has no memory leak if matched in size.

ah, my bad. didn't get you were talking about the other direction only.

> The one thing you might be able to hack together now is the 

> creative use of one large array being sent into a single row matrix 

> to jit.scanwrap to wrap that long row into a 2d matrix.

a quick test shows not as good results as with copyArrayToMatrix(), 

but it's significantly faster than calling copyArrayToVector() for 

thanks, would have never thought of that myself.

>
> On Feb 7, 2008, at 12:37 PM, vboehm@gmx.ch wrote:
>>
>> hmm, interesting. i didn't know about copyArrayToMatrix() - must  
>> have slipped in unnoticed somewhere inbetween.
>> i was using copyArrayToVector() before and it works fine, as you  
>> said.
>> on a speed comparison however, i was happy to notice that  
>> copyArrayToMatrix() is much faster than the vector method - at  
>> least in my situation. cpu usage came down half way for a 100x100  
>> matrix (measured with top).
>>
>> BUT, i can't escape the memory leak even if the array and the  
>> matrix match in size!
>> in the simplified example below, memory is still leaking. where am  
>> i going wrong?
>> any help is welcome, since the speed increase could be crucial to  
>> the project i am working on.
>
> Sorry, copyArrayToMatrix() *always* leaks in Jitter 1.6.x. It is  
> copyMatrixToArray() which has no memory leak if matched in size.

>
> The one thing you might be able to hack together now is the  
> creative use of one large array being sent into a single row matrix  
> to jit.scanwrap to wrap that long row into a 2d matrix.

that's a good one!
a quick test shows not as good results as with copyArrayToMatrix(),  
but it's significantly faster than calling copyArrayToVector() for  
each row in the output image.
thanks, would have never thought of that myself.
volker.

just wanted to ping this old post, since it seems to me that, almost ten years after the original post, on max 7.3.4, the crash still happens with the copyArrayToMatrix() method.

Is this a known fact? Will there ever be a moment when this method will work smoothly or has it been somehow silently deprecated?

Just as a side-note anyway, if anyone would be interested, I have done some simple speed tests between the stable copyMatrixToArray() and copyVectorToArray() and the first one is significantly faster only if you pass multidimensional matrixes to both methods (using the for loop as suggested above to surf the various lines). If you first convert the n*m matrix into a n*1 using jit.scanwrap and then use copyVectorToArray() on the only resulting line the speed is almost the same. So the same could theoretically be true with (a stable version of) copyArrayToMatrix() and copyVectorToMatrix().

Hi all,
just wanted to ping this old post, since it seems to me that, almost ten years after the original post, on max 7.3.4, the crash still happens with the copyArrayToMatrix() method.
Is this a known fact? Will there ever be a moment when this method will work smoothly or has it been somehow silently deprecated?
 
Just as a side-note anyway, if anyone would be interested, I have done some simple speed tests between the stable copyMatrixToArray() and copyVectorToArray() and the first one is significantly faster only if you pass multidimensional matrixes to both methods (using the for loop as suggested above to surf the various lines). If you first convert the n*m matrix into a n*1 using jit.scanwrap and then use copyVectorToArray() on the only resulting line the speed is almost the same. So the same could theoretically be true with (a stable version of) copyArrayToMatrix() and copyVectorToMatrix().

Crash in JitterMatrix.copyMatrixToArray() ?