Bpatcher doesn't free the memory after several replacement

Rajan Craveri's icon

Hi all,
during an installation my patch crashed after 5 hours of work,
back in studio I discovered a strange thing:

In my patch I use a bpatcher and I load into it different effects (patches),
each effect loads its textures from hd,
when I replace the patch inside bpatcher I expect the bpatcher free the memory
and load a new patch but it seems that in some way some texture remain in memory
and after 5 hours the RAM is full and the patch crash.

I isolated the behavior in a demo patch
can someone reproduce the bug?
you can track the ram memory increasing in activity monitor

thanks for your help

Rajan Craveri's icon

some screenshot

Schermata-2013-07-10-alle-12.28.46.png
png
Schermata-2013-07-10-alle-12.29.31.png
png
Rajan Craveri's icon
Lee's icon

i'll see if I can test this tonight... I do alot of work with bpatchers and every few hours I tend to restart MAX as things start slowing down a bit - never checked memory tho so will have a look

Rajan Craveri's icon

thanks Lee let me know

Rajan Craveri's icon

here is the first extract of the crash log of my patch
it crashes after 5 hours more or less

Process: IT_SYSTEM [171]
Path: /Users/USER/Documents/IT_SYSTEM.app/Contents/MacOS/IT_SYSTEM
Identifier: com.cycling74.MaxRuntime
Version: 6.0.8 [a0c1b20] (6.0.8)
Code Type: X86 (Native)
Parent Process: launchd [123]
User ID: 501

Date/Time: 2013-07-10 16:28:17.992 +0200
OS Version: Mac OS X 10.8.4 (12E55)
Report Version: 10

Crashed Thread: 0 Java: AWT-AppKit Dispatch queue: com.apple.main-thread

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x00000000b7cc3010

VM Regions Near 0xb7cc3010:
IOKit 00000000b7c83000-00000000b7cc3000 [ 256K] rw-/rw- SM=SHM
-->
IOKit 00000000b7cd6000-00000000ba496000 [ 39.8M] rw-/rw- SM=SHM

Application Specific Information:
Java information:
Exception type: Bus Error (0xa) at pc=0000000094057a40

Java VM: Java HotSpot(TM) Client VM (20.51-b01-457 mixed mode macosx-x86)

Current thread (0000000018279000): JavaThread "AWT-AppKit" [_thread_in_native, id=-1405838808, stack(00000000bf800000,00000000c0000000)]
Stack: [00000000bf800000,00000000c0000000]

Java Threads: ( => current thread )
00000000302f7000 JavaThread "AWT-Shutdown" [_thread_blocked, id=-1321005056, stack(00000000b1331000,00000000b1431000)]
0000000017b86c00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=-1323651072, stack(00000000b10ab000,00000000b11ab000)]
0000000017b85c00 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=-1324707840, stack(00000000b0fa9000,00000000b10a9000)]
0000000017b84c00 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=-1325764608, stack(00000000b0ea7000,00000000b0fa7000)]
0000000017b83c00 JavaThread "Surrogate Locker Thread (Concurrent GC)" daemon [_thread_blocked, id=-1326821376, stack(00000000b0da5000,00000000b0ea5000)]
0000000030287400 JavaThread "Finalizer" daemon [_thread_blocked, id=-1327878144, stack(00000000b0ca3000,00000000b0da3000)]
0000000030286400 JavaThread "Reference Handler" daemon [_thread_blocked, id=-1328934912, stack(00000000b0ba1000,00000000b0ca1000)]
=>0000000018279000 JavaThread "AWT-AppKit" [_thread_in_native, id=-1405838808, stack(00000000bf800000,00000000c0000000)]
Other Threads:
0000000030284800 VMThread [stack: 00000000b0a9f000,00000000b0b9f000] [id=-1329991680]
0000000017b88400 WatcherThread [stack: 00000000b11ad000,00000000b12ad000] [id=-1322594304]

VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None

Heap
par new generation total 14784K, used 8308K [000000001a810000, 000000001b810000, 000000001c810000)
eden space 13184K, 62% used [000000001a810000, 000000001b013680, 000000001b4f0000)
from space 1600K, 6% used [000000001b4f0000, 000000001b509bf0, 000000001b680000)
to space 1600K, 0% used [000000001b680000, 000000001b680000, 000000001b810000)
concurrent mark-sweep generation total 49152K, used 1118K [000000001c810000, 000000001f810000, 000000002a810000)
concurrent-mark-sweep perm gen total 12288K, used 5100K [000000002a810000, 000000002b410000, 000000002e810000)

Code Cache [0000000018801000, 00000000188c2000, 000000001a801000)
total_blobs=356 nmethods=219 adapters=81 free_code_cache=32765440 largest_free_block=0

Virtual Machine Arguments:
JVM Args: -Xincgc -Xms64m -Xmx256m
Java Command:
Launcher Type: generic
Physical Memory: Page Size = 4k, Total = 4096M, Free = 204M

Rajan Craveri's icon

No! in the same test patch today the bpatcher doesn't keeps the memory but discards it correctly when i replace a patcher inside...
not good, not good...

Rajan Craveri's icon

tested on
MacBook pro and Mac mini i5
MAX 6.0.8
java 7 and java 6
OSX 10.8.4

Emmanuel Jourdan's icon

It's crashing here on 6.0.8 but working fine with 6.1.3, you might want to give it a try using the latest version.

Rajan Craveri's icon

Emmanuel Jourdan thank you for your help,
I follow your suggestion but I encountered the same problem
I isolated again the problem in this example patch
It seems that jit.gl.pix is the cause but I'm not sure
can you try with this version
thank you

bpatcher_jitpix_bug.zip
zip
Schermata-2013-07-12-alle-12.59.41.png
png
Schermata-2013-07-10-alle-12.28.461.png
png
Schermata-2013-07-10-alle-12.29.311.png
png
Rajan Craveri's icon

sorry old screenshots

Schermata-2013-07-12-alle-12.55.25.png
png
Schermata-2013-07-12-alle-12.53.04.png
png
Rajan Craveri's icon

Can someone reproduce this problem with jit.gl.pix inside a bpatcher?

Rajan Craveri's icon

I tried to export glsl from hit.gl.pix and load the shader into a jit.gl.slab but I got the same problem after several replacement of the patch inside a bpatcher you can noticed the batch doesn't free the RAM.

here is the test patch with jit.gl.slab and the exported shader

ji_gl_slab_bug_bpatcher.zip
zip
Rajan Craveri's icon

I tried to send a "purge" message by shell object to free the memory but It doesn't work
my patch always crash after some hours

now i added an applescript to control if my app is running and if it doesn't the applescript restart my mac mini but sure this is not a solution

... this time I really don't know how I can proceed

Andrew Pask's icon

Yes, I can see an increase in memory use here - we'll investigate

Cheers

-A

Rajan Craveri's icon

Thanks for the feedback, I wait for your help
let me know if I can help you with particular tests.

evsc's icon

I am experiencing similar problems:

I use MXJ objects producing opengl jitter graphics about 2 bpatchers deep, and Max constantly crashes, within the first seconds. I discovered that the the patch lasts longer if i get rid of the bpatcher hierarchy. If my MXJ objects are in the main patch, the crash happens way later. So i know there is a general memory issue with my patch, but the bpatchers definitely amplify the problem.

My system:
Windows 7
Max 5.1.4
JVM 1.7

Curiously the same patch (with the bpatcher hierarchy) runs fine on a Mac (OS 10.6.8, JVM 1.6, Max Runtime 5.1.9).

I then made some changes to my max.java.config.txt, to force Max to use JVM 1.6 (which i also have on my Windows machine), and to enlarge the normal (120m) and maximum Memory (to 512m).

max.java.jvm.version 1.6
max.jvm.option -Xincgc
max.jvm.option -Xms120m
max.jvm.option -Xmx512m

This seems to have improved the situation a little bit. I can actually run my patch for a couple of minutes, but i still see the Memory of Max growing in the Task Manager, and when it hits ~250MB, Max crashes.

Now i am trying to get proper debug messages from the JVM, that'll hopefully provide more insight.

Rajan Craveri's icon

If this test can help I tried instead of "replace"
- delete the bpatcher
- create a new one
- load a new patcher inside it
but I noticed the same memory increasing

Max Patch
Copy patch and select New From Clipboard in Max.

here is the example

Andrew Pask's icon

There could be lots of different things going on in this thread.

The OP's problem is unrelated to bpatcher, it's a memory leak in jit.gl.pix
I found this out by creating a similar patcher loading system using scripting and abstractions.

We're happy to take a look in support at these issues if you can chase them down a bit

Cheers

Rajan Craveri's icon

Dear Andrew,
I followed your suggestion and I stop to investigate bpatcher,

so I tried to load an abstraction with jit.gl.pix
and another with jit.gl.slab

I noticed that both of them have memory problems.
If I load and delete the abstraction my memory increase and increase.

Here are the two example patches

test_abstract_jit_gl_pix.zip
zip
test_abstraction_jit_slab.zip
zip
Rajan Craveri's icon
Max Patch
Copy patch and select New From Clipboard in Max.

I tested this problem in a simple patch without scripting
simple cut and paste several times the jit.gl.pix objects
I noticed that there is the memory increasing
here the code

Rajan Craveri's icon

How can I help the support team to find a solution?
Are there some tests I can do that could be useful?

Andrew Pask's icon

We've got these issues ticketed - thanks for your reports. They are being looked at in engineering.

Sorry for the inconvenience in the meantime. I do not have an estimate for when the fixes will be available

Cheers

Andrew

Rajan Craveri's icon

Dear Andrew, my teamwork ask to me when I'll fix this problem and I don't know what can I say.
Can your engineers give to me an estimate for when the fixes will be available?
It's quite important for me
thanks

Joshua Kit Clayton's icon

@micron Thanks for your detailed report and examples. This pointed out a few issues and will be fixed in the forthcoming 6.1.4 release. We don't provide release dates in advance, but this should be available in the relatively near future.

Rajan Craveri's icon

Thanks joshua for your attention into my problems,
I'm waiting the new version of max.

Rajan Craveri's icon

Hi joshua, i saw in the new version the "jit.gl.pix memory improvement" thanks
but I tried immidiatly if that fix my problems and Max crashed after few hours like before
to better understand i opened a new topic with an example patch to reproduce the bug
https://cycling74.com/forums/textures-fill-the-memory-and-crash/

Joshua Kit Clayton's icon

As mentioned on the other thread, but including here for posterity's sake:

Many thanks for following up on this. We believe we’ve solved the most serious of memory leaks here (thanks again for your great reporting on that which helped us find numerous issues).

However, I am able to reproduce your reported behavior in OpenGL Driver Monitor, despite the fact that we have matched pairs of glGenTextures and glDeleteTextures, and we consistently reuse the same texture ids. The OpenGL driver monitor reports them growing however, which is curious, and suggests that it marks them for “garbage collection” or similar at a later point.

On a lark, I tried deleting the context and rebuilding it, and it would seem seem from using OpenGL monitor that in such a case, it reclaims these textures. I’m not sure of a better way to trigger this reclamation at this point and I didn’t let run long enough to confirm whether or not it solves the crashes, but please try sending the jit.window object the message “depthbuffer 0, depthbuffer 1″ to reclaim these textures, and let us know what you find.

Thanks,
Joshua