I seem to have created a memory leak of some kind.

I have an object, say "Link", that makes associations between two instances of another type of objects, say "Thing". Basically, I'm testing for similarity between Things, and using Links to record the similarities found.

The Link has a constructor which takes two Things, and some analysis data indicating the ways in which they are similar. What I'm curious about is whether such a constructor would make deep copies of the two Thing objects, or just store references? The way my memory usage is spinning out of control, I'm guessing that actual copies are being made. Is there any way to check this? Is there any way to avoid this?

I have an object, say "Link", that makes associations between two instances of another type of objects, say "Thing". Basically, I'm testing for similarity between Things, and using Links to record the similarities found. 

memory-running-wild

As an example, I've just been testing a run of the code in which listA has 292 members, and listB has 203. A complete run of the test for similarity will create 59276 "Link" objects. The total memory used by Max before the run (shown in "top") is about 64MB, and after the run it's about 100MB - so the creation of Link objects presumably used up about 36MB. (If I comment out the line that instantiates the Link objects, the memory stays at 64MB.)

The Link object contains two double arrays (not particularly long ones) and the 2 "Thing" object references mentioned above. But these numbers would suggest about 600 bytes per object. Isn't that a bit much? If that seems like a reasonable size, then I'll just have to figure out a more efficient way of mapping out the similarities I'm after...

Can you post the code for the Link class?

> As an example, I've just been testing a run of the code in which

> listA has 292 members, and listB has 203. A complete run of the test

> for similarity will create 59276 "Link" objects. The total memory

> used by Max before the run (shown in "top") is about 64MB, and after

> the run it's about 100MB - so the creation of Link objects presumably

> used up about 34MB. (If I comment out the line that instantiates the

> Link objects, the memory stays at 64MB.)

> The Link object contains two double arrays (not particularly long

> ones) and the 2 "Thing" object references mentioned above. But these

> numbers would suggest about 600 bytes per object. Isn't that a bit

> much? If that seems like a reasonable size, then I'll just have to

> figure out a more efficient way of mapping out the similarities I'm

> J. _______________________________________________ java-dev mailing

jbmaxwell wrote:
> As an example, I've just been testing a run of the code in which
> listA has 292 members, and listB has 203. A complete run of the test
> for similarity will create 59276 "Link" objects. The total memory
> used by Max before the run (shown in "top") is about 64MB, and after
> the run it's about 100MB - so the creation of Link objects presumably
> used up about 34MB. (If I comment out the line that instantiates the
> Link objects, the memory stays at 64MB.)
>
> Doesn't that seem high?
>
> The Link object contains two double arrays (not particularly long
> ones) and the 2 "Thing" object references mentioned above. But these
> numbers would suggest about 600 bytes per object. Isn't that a bit
> much? If that seems like a reasonable size, then I'll just have to
> figure out a more efficient way of mapping out the similarities I'm
> after...
>
> J. _______________________________________________ java-dev mailing
>


Okay, I've attached the source for the "Link" object... which is actually an "Allusion" object.

I did notice that there is another ArrayList of object references I had forgotten about (gulp), and a HashMap... okay, that's a bit embarrassing... (oh, and I load some short home movies, a couple of 8-channel aif files, and... just kidding.)

Anyway, I commented out that code and cut about 9MB off the run - not that I can do that and still have the program work, but I though I'd at least test it. Taking this into consideration, do you think the memory usage is about what would be expected? That is, around 27MB for the creation of the 59276 objects? (That should be about 477 bytes per instance.) Still seems high, to me.

Okay, I've attached the source for the "Link" object... which is actually an "Allusion" object. 

I did notice that there is another ArrayList of object references I had forgotten about (gulp), and a HashMap... okay, that's a bit embarrassing... (oh, and I load some short home movies, a couple of 8-channel aif files, and... just kidding.) 

Anyway, I commented out that code and cut about 9MB off the run - not that I can do that and still have the program work, but I though I'd at least test it. Taking this into consideration, do you think the memory usage is about what would be expected? That is, around 27MB for the creation of the 59276 objects? (That should be about 477 bytes per instance.) Still seems high, to me. 

Yeah, I didn't see how it could be copying the objects, but that was the only way I could imagine the memory getting so far out of control. On certain analysis jobs, the memory runs out to 500+ MB! I have a sinking feeling that I haven't done anything obviously wrong, except implement a really bloated design. There are ways around it, but they won't be as clean to work with at higher levels of the program, which is lame.

Maybe Owen will see something else, but I think I'm going to be rewriting a bunch of old code tonight...

...okay, found a way around it by rewriting some code...

Do tell - I'd been about to start picking the original apart to see what 

> ...okay, found a way around it by rewriting some code...

Do tell - I'd been about to start picking the original apart to see what 
was going on.

jbmaxwell wrote:
> ...okay, found a way around it by rewriting some code...
>
>
>
>


Quote: owen wrote on Wed, 18 July 2007 21:19

----------------------------------------------------

> Do tell - I'd been about to start picking the original apart to see what 

> > ...okay, found a way around it by rewriting some code...

Oh, thanks! Well, I actually realized that with only a small redesign I could get rid of this class entirely, without losing any real functionality. The overall model is every-so-slightly less OOP, but not enough to miss.

I messed around for several hours this morning trying to figure out how that other class could be eating so much memory, but I never figured it out. The new version I have basically pulls the functionality of the "Allusion" into the objects that the Allusion class was associating. I had to move the statistical data into these objects as well, but it's more convenient to have it there anyway, so no loss.

Anyway, what I thought was a clever design was just bloated, I guess! If you do look at it, purely out of theoretical interest, and see anything really obvious, let me know. But please don't waste any of your time.

Quote: owen wrote on Wed, 18 July 2007 21:19
----------------------------------------------------
> Do tell - I'd been about to start picking the original apart to see what 
> was going on.
>
> jbmaxwell wrote:
> > ...okay, found a way around it by rewriting some code...
> >
> >
> >
> >
>
>
----------------------------------------------------

Oh, thanks! Well, I actually realized that with only a small redesign I could get rid of this class entirely, without losing any real functionality. The overall model is every-so-slightly less OOP, but not enough to miss. 

memory running wild