dynamically loading java classes in a standalone
Hey gang --
I'm building some standalones (Max 4.6.3, OSX 10.4.11, 2 GHz Intel
MacBook), and I can get things to work by putting the max.jar file in
"my.app/Contents/support/java/lib/". Now I have a large jar file,
and I put it in "my.app/Contents/support/java/classes/". The problem
is that the standalone is treating the jar file as a system jar and
loading the entire thing into memory at start-up. I want to add the
jar to the dynamic classpath so that it will only load the classes I
need from the jar.
I do this when running MaxMSP by setting the "max.dynamic.class.dir"
in the "max.java.config.txt" to point to the large jar file. How can
I set the dynamic classpath in a standalone?
thanks!
brad
I don't really understand what you mean by saying that your entire
jar is being loaded at startup. If you have any mxj instances in your
standalone patch they should only load the classes that they have
dependencies on. I am not aware of a way to get the JVM to force load
a bunch of arbitrary classes.It loads any class when it needs it with
the exception of the bootstrap classes.
Regardless. I am not totally sure but if you include
max.java.config.txt it is possible that mxj will find it. You might
want to mess around with this.
In the case that it does find it you will need to mess around with
paths for the max.dynamic.jar.dir, max.dynamic.class.dir etc since
they will need to be
relative and I am not sure what they would be relative to.
Sorry I can't be more helpful.
t
On Feb 21, 2008, at 10:51 AM, Brad Garton wrote:
> Hey gang --
>
> I'm building some standalones (Max 4.6.3, OSX 10.4.11, 2 GHz Intel
> MacBook), and I can get things to work by putting the max.jar file
> in "my.app/Contents/support/java/lib/". Now I have a large jar
> file, and I put it in "my.app/Contents/support/java/classes/". The
> problem is that the standalone is treating the jar file as a system
> jar and loading the entire thing into memory at start-up. I want
> to add the jar to the dynamic classpath so that it will only load
> the classes I need from the jar.
>
> I do this when running MaxMSP by setting the
> "max.dynamic.class.dir" in the "max.java.config.txt" to point to
> the large jar file. How can I set the dynamic classpath in a
> standalone?
>
> thanks!
>
> brad
>
I am guessing at behavior I don't understand. What is the difference
between the 'immutable classpath" and the "dynamic classpath". I
have a jar file that's about 6 Mbtyes, and inflates to about 30
Mbytes. If I put it on the system (immutable) classpath or just put
it into one of the java classes/ or lib/ subdirs, it takes about a
minute to instantiate the object that references a class in that
jar. I can watch the memory usage slowly creep up to about 30
Mbytes from the starting point as apparently all of the classes are
being loaded and inflated in memory
If, however, I explicitly put the jar itself on the "dynamic
classpath" then the object that draws on that jar load immediately
with no huge memory allocation. I don't know what is causing the
different behaviors. I assumed it had to do with how the JVM was
loading the classes in the jar.
Also, if I unpack the jar into all the separate classes in the
classes/ dir, it acts like when I add the jar to the "dynamic
classpath"; i.e. it loads quickly.
brad
On Feb 21, 2008, at 3:44 PM, topher lafata wrote:
> I don't really understand what you mean by saying that your entire
> jar is being loaded at startup. If you have any mxj instances in
> your standalone patch they should only load the classes that they
> have dependencies on. I am not aware of a way to get the JVM to
> force load a bunch of arbitrary classes.It loads any class when it
> needs it with the exception of the bootstrap classes.
>
> Regardless. I am not totally sure but if you include
> max.java.config.txt it is possible that mxj will find it. You might
> want to mess around with this.
> In the case that it does find it you will need to mess around with
> paths for the max.dynamic.jar.dir, max.dynamic.class.dir etc since
> they will need to be
> relative and I am not sure what they would be relative to.
>
> Sorry I can't be more helpful.
>
> t
>
> On Feb 21, 2008, at 10:51 AM, Brad Garton wrote:
>
>> Hey gang --
>>
>> I'm building some standalones (Max 4.6.3, OSX 10.4.11, 2 GHz Intel
>> MacBook), and I can get things to work by putting the max.jar file
>> in "my.app/Contents/support/java/lib/". Now I have a large jar
>> file, and I put it in "my.app/Contents/support/java/classes/".
>> The problem is that the standalone is treating the jar file as a
>> system jar and loading the entire thing into memory at start-up.
>> I want to add the jar to the dynamic classpath so that it will
>> only load the classes I need from the jar.
>>
>> I do this when running MaxMSP by setting the
>> "max.dynamic.class.dir" in the "max.java.config.txt" to point to
>> the large jar file. How can I set the dynamic classpath in a
>> standalone?
>>
>> thanks!
>>
>> brad
>>
>
On Feb 21, 2008, at 14:01 PM, Brad Garton wrote:
> I am guessing at behavior I don't understand. What is the
> difference between the 'immutable classpath" and the "dynamic
> classpath". I have a jar file that's about 6 Mbtyes, and inflates
> to about 30 Mbytes. If I put it on the system (immutable)
> classpath or just put it into one of the java classes/ or lib/
> subdirs, it takes about a minute to instantiate the object that
> references a class in that jar. I can watch the memory usage
> slowly creep up to about 30 Mbytes from the starting point as
> apparently all of the classes are being loaded and inflated in memory
The system classpath is the normal java classpath which is specified
to the vm at startup. java -classpath C:/blah/blah etc.Once the vm is
started it can't be changed. This relates to the security of the vm.
The only way you can add a new class to the vm at runtime is by using
a custom classloader.
The dynamic classpath is a construct we came up with so that classes
could be reloaded by the mxj classloader (our custom classloader) so
you don't have to quit and start max everytime you change a class
definition. This is useful during the development phase of an mxj class.
Thus, any class that you want to reload dynamically after its
definition changes needs to be in the dynamic classpath.For a
standalone the classes won't really be changing so you should put
your jar in:
my.app/Contents/support/java/lib/
alongside max.jar and see if that helps.
The only reason that your code is working at all in the standalone is
because as a last resort max will search the whole max search path
for classes and jars it cannot find in the classpath. This would
explain why it takes such a long time for your classes in the jar to
load. The fact that the memory is increasing suggests that there may
be a memory leak when max resorts to scanning the whole max search
path for a class.
It is not finding your jar in the classes directory since that
directory is only for classes not jar files. However it is finding
the classes in your jar file when it resorts to the full max searchpath.
The c74/classes folder is by default part of the dynamic search path.
We do not have a default dynamic jar path but one can be specified in
max.java.config.txt.
I am not sure how this plays out in a standalone since I am not sure
how to specify a relative path to a dynamic jar directory in the
context of a standalone. It would probably have to be relative to the
max executable itself.
In theory you could specify in max.java.config.txt these relative
paths for any dynamic class and/or dynamic jar directories and
included that max.java.config.txt in your standalone.It is just a
question of finding out what they should be specified relative to
which as i said above is probably the max executable itself.
So to make a long story short.
try moving your jar to the same directory as max.jar and see if your
standalone behaves as expected.
t
> If, however, I explicitly put the jar itself on the "dynamic
> classpath" then the object that draws on that jar load immediately
> with no huge memory allocation. I don't know what is causing the
> different behaviors. I assumed it had to do with how the JVM was
> loading the classes in the jar.
>
> Also, if I unpack the jar into all the separate classes in the
> classes/ dir, it acts like when I add the jar to the "dynamic
> classpath"; i.e. it loads quickly.
>
> brad
>
>
> On Feb 21, 2008, at 3:44 PM, topher lafata wrote:
>
>> I don't really understand what you mean by saying that your entire
>> jar is being loaded at startup. If you have any mxj instances in
>> your standalone patch they should only load the classes that they
>> have dependencies on. I am not aware of a way to get the JVM to
>> force load a bunch of arbitrary classes.It loads any class when it
>> needs it with the exception of the bootstrap classes.
>>
>> Regardless. I am not totally sure but if you include
>> max.java.config.txt it is possible that mxj will find it. You
>> might want to mess around with this.
>> In the case that it does find it you will need to mess around with
>> paths for the max.dynamic.jar.dir, max.dynamic.class.dir etc since
>> they will need to be
>> relative and I am not sure what they would be relative to.
>>
>> Sorry I can't be more helpful.
>>
>> t
>>
>> On Feb 21, 2008, at 10:51 AM, Brad Garton wrote:
>>
>>> Hey gang --
>>>
>>> I'm building some standalones (Max 4.6.3, OSX 10.4.11, 2 GHz
>>> Intel MacBook), and I can get things to work by putting the
>>> max.jar file in "my.app/Contents/support/java/lib/". Now I have
>>> a large jar file, and I put it in "my.app/Contents/support/java/
>>> classes/". The problem is that the standalone is treating the
>>> jar file as a system jar and loading the entire thing into memory
>>> at start-up. I want to add the jar to the dynamic classpath so
>>> that it will only load the classes I need from the jar.
>>>
>>> I do this when running MaxMSP by setting the
>>> "max.dynamic.class.dir" in the "max.java.config.txt" to point to
>>> the large jar file. How can I set the dynamic classpath in a
>>> standalone?
>>>
>>> thanks!
>>>
>>> brad
>>>
>>
>
one more question.
how large are all of those classes before you jar them up?
it is very possible that the jar is being unzipped when the mxj
classloader falls back on trying to locate the class in the max
search path. as a matter of fact it is entirely possible that all
jars in the max search path are being unzipped temporarily into
memory ,including max.jar, since the classloader needs to crack open
any jar file it finds in the search path to check if it contains the
class it is looking for.
This previous point would explain the memory blowup you are seeing.
As mentioned before, the long time it takes for instantiation would
be due to the fact that the whole max search path is being scanned
for each class you are loading since your jar is not in the system or
dynamic classpath and the classloader is falling back to its last
resort scheme for classloading.
t
On Feb 21, 2008, at 14:01 PM, Brad Garton wrote:
> I am guessing at behavior I don't understand. What is the
> difference between the 'immutable classpath" and the "dynamic
> classpath". I have a jar file that's about 6 Mbtyes, and inflates
> to about 30 Mbytes. If I put it on the system (immutable)
> classpath or just put it into one of the java classes/ or lib/
> subdirs, it takes about a minute to instantiate the object that
> references a class in that jar. I can watch the memory usage
> slowly creep up to about 30 Mbytes from the starting point as
> apparently all of the classes are being loaded and inflated in memory
>
> If, however, I explicitly put the jar itself on the "dynamic
> classpath" then the object that draws on that jar load immediately
> with no huge memory allocation. I don't know what is causing the
> different behaviors. I assumed it had to do with how the JVM was
> loading the classes in the jar.
>
> Also, if I unpack the jar into all the separate classes in the
> classes/ dir, it acts like when I add the jar to the "dynamic
> classpath"; i.e. it loads quickly.
>
> brad
>
>
> On Feb 21, 2008, at 3:44 PM, topher lafata wrote:
>
>> I don't really understand what you mean by saying that your entire
>> jar is being loaded at startup. If you have any mxj instances in
>> your standalone patch they should only load the classes that they
>> have dependencies on. I am not aware of a way to get the JVM to
>> force load a bunch of arbitrary classes.It loads any class when it
>> needs it with the exception of the bootstrap classes.
>>
>> Regardless. I am not totally sure but if you include
>> max.java.config.txt it is possible that mxj will find it. You
>> might want to mess around with this.
>> In the case that it does find it you will need to mess around with
>> paths for the max.dynamic.jar.dir, max.dynamic.class.dir etc since
>> they will need to be
>> relative and I am not sure what they would be relative to.
>>
>> Sorry I can't be more helpful.
>>
>> t
>>
>> On Feb 21, 2008, at 10:51 AM, Brad Garton wrote:
>>
>>> Hey gang --
>>>
>>> I'm building some standalones (Max 4.6.3, OSX 10.4.11, 2 GHz
>>> Intel MacBook), and I can get things to work by putting the
>>> max.jar file in "my.app/Contents/support/java/lib/". Now I have
>>> a large jar file, and I put it in "my.app/Contents/support/java/
>>> classes/". The problem is that the standalone is treating the
>>> jar file as a system jar and loading the entire thing into memory
>>> at start-up. I want to add the jar to the dynamic classpath so
>>> that it will only load the classes I need from the jar.
>>>
>>> I do this when running MaxMSP by setting the
>>> "max.dynamic.class.dir" in the "max.java.config.txt" to point to
>>> the large jar file. How can I set the dynamic classpath in a
>>> standalone?
>>>
>>> thanks!
>>>
>>> brad
>>>
>>
>
Ok, I see I'm pretty confused about this. But the behavior I'm
seeing is also confusing, and I can't think of why. It's my maxlispj
object, and the BGabcl.jar file is 6.0 Mbytes. Un-jarred and
inflated, it's about 30 Mbytes of classes (which is the amount that
gets allocated when it does the "slow load" thing). Here's the
behavior:
In a standalone, if I put the BGabcl.jar file in either of these
directories:
my.app/Contents/support/java/lib/
my.app/Contents/support/
it loads quickly without the additional 30 Mbyte memory allocation.
If I put it here:
my.app/Contents/support/java/classes/
it takes a long time to load and allocates about 30 Mbytes of memory.
In the MaxMSP application, if I put it here:
/Applications/MaxMSP 4.6/Cycling '74/java/classes/
it takes a long time to load and allocates about 30 Mbytes of
memory. But if I add this line to the max.java.config.txt file:
max.dynamic.class.dir "/Applications/MaxMSP 4.6/Cycling '74/java/
classes/BGabcl.jar"
it loads quickly. Hence the source of my confusion about the dynamic
classpath.
One final weirdness, if I put the BGabcl.jar file here:
/Applications/MaxMSP 4.6/Cycling '74/java/lib/
with or without an entry in the max.java.config.txt file, it loads
quickly (and works ok!), but I get these errors in the Max window:
error: java.util.zip.ZipException: No such file or directory
error: at java.util.zip.ZipFile.open(Native Method)
error: at java.util.zip.ZipFile.(ZipFile.java:203)
error: at java.util.zip.ZipFile.(ZipFile.java:84)
followed by some Lisp loader errors, but the object gets instantiated
ok!
I've been using the object a lot, and it seems to work well even in
the slow-load case. I really don't know what's going on with the
loading differences, though.
I know you've got a lot going on right now, so don't delve into this
too deeply unless something seems obvious to you from the above
description. If I put it in the right dirs, it works fine, and I'm
happy.
brad
This is what I think is happening, because the memory increase I see
is about how large the un-jarred classes are.
I think my confusion came from the dynamic classpath entry in the
max.java.config.txt file making it work ok in the java/classes/ dir
in the MaxMSP application hierarchy.
brad
On Feb 21, 2008, at 5:50 PM, topher lafata wrote:
> one more question.
>
> how large are all of those classes before you jar them up?
>
> it is very possible that the jar is being unzipped when the mxj
> classloader falls back on trying to locate the class in the max
> search path. as a matter of fact it is entirely possible that all
> jars in the max search path are being unzipped temporarily into
> memory ,including max.jar, since the classloader needs to crack
> open any jar file it finds in the search path to check if it
> contains the class it is looking for.
>
> This previous point would explain the memory blowup you are seeing.
> As mentioned before, the long time it takes for instantiation would
> be due to the fact that the whole max search path is being scanned
> for each class you are loading since your jar is not in the system
> or dynamic classpath and the classloader is falling back to its
> last resort scheme for classloading.
> it takes a long time to load and allocates about 30 Mbytes of
> memory. But if I add this line to the max.java.config.txt file:
>
> max.dynamic.class.dir "/Applications/MaxMSP 4.6/Cycling '74/java/
> classes/BGabcl.jar"
>
> it loads quickly. Hence the source of my confusion about the
> dynamic classpath.
>
yeah. dont do this. the class directory is for classes not jars. you
could do:
max.dynamic.jar.dir "/Applications/MaxMSP 4.6/Cycling '74/java/myjars"
and put BGabcl.jar in it.
I cant say what the behavior of mxj is when you set a dynamic class
dir to a jar. suffice it to say that it is undefined.
> One final weirdness, if I put the BGabcl.jar file here:
>
> /Applications/MaxMSP 4.6/Cycling '74/java/lib/
>
> with or without an entry in the max.java.config.txt file, it loads
> quickly (and works ok!), but I get these errors in the Max window:
>
> error: java.util.zip.ZipException: No such file or directory
> error: at java.util.zip.ZipFile.open(Native Method)
> error: at java.util.zip.ZipFile.(ZipFile.java:203)
> error: at java.util.zip.ZipFile.(ZipFile.java:84)
>
there might be something weird in the way your jar is constructed?
manifest out of sync? confusion about having that weird dynamic
classpath entry point to a jar? i cant say.
> I know you've got a lot going on right now, so don't delve into
> this too deeply unless something seems obvious to you from the
> above description. If I put it in the right dirs, it works fine,
> and I'm happy.
awesome.
ciao
t