memory usage on linux

[Murat Tasan]

hi,

i have a java application that can be fairly memory intensive. so, when i
run the program, i use the -Xmx option to prevent out of memory errors.
in particular, i have been using -Xmx512m, which should be enough for my
needs... but the out of memory errors continue.

so, i tried to observe this in action.

i ran top while my program was executing, and sorted by memory usage. as
expected on linux, all the java threads show up as processes. now, top
reports the rss on each thread to be around 75MB when the application hits
the error... and about 7-8 threads when this occurs. now i know that the
-Xmx option is for java objects only, and doesn't pertain to the virtual
machine's own memory requirements... but this observation seems to
indicate that each thread actually is consuming its own 75MB of memory.
this is also supported with each java thread only consuming about 4K of
shared memory, as reported by "top".

is what i think i'm observing actually happening? is each java thread
actually consuming that much extra memory (i.e. without sharing it all
through the single java virtual machine heap)? somehow, i doubt it... but
the out of memory errors seem to be happening well before 512MB of java
object information is created.

note, too, that i am not actually creating these seperate threads... my
app is single-threaded... the 7-8 are those threads executed intially by
the java virtual machine.

thanks for any insight on this matter... or even a link to a good source
that describes this.

thanks much again,

murat

--
Murat Tasan
mx**@po.cwru.edu
ta***@eecs.cwru.edu
mu*********@cwru.edu
http://genomics.cwru.edu

[Murat Tasan]

well, i found the problem. the program does indeed use over 512MB of heap
space.

the interesting part though, is for linux users. the "top" program seemed
to give faulty output when run without being modified interactively. that
is... in situation 1, i ran top, hit "M" to sort by memory, and watched
(using the default refresh rate, which is something like 3-5 seconds on my
box). it started to stall and not update correctly... i only caught it by
running situation 2, where i ran top, hit "M" to sort by memory, then "s"
and gave a refresh of 1 second. suddenly the values changed drastically
and the memory usage climbed quite high.

this must be a bug in top, because the java virtual machine doesn't grow
that quickly in a second. i've verified this again by running top again
and setting the refresh rate to 1 second before executing the java app.

so, as a note... when examining memory or cpu usage through top, make sure
you manually set the rate through the "s" key.

of course, this may only be a bug on a few systems.

sorry about that,

murat

On Tue, 16 Mar 2004, Murat Tasan wrote:

hi,

i have a java application that can be fairly memory intensive. so, when i
run the program, i use the -Xmx option to prevent out of memory errors.
in particular, i have been using -Xmx512m, which should be enough for my
needs... but the out of memory errors continue.

so, i tried to observe this in action.

i ran top while my program was executing, and sorted by memory usage. as
expected on linux, all the java threads show up as processes. now, top
reports the rss on each thread to be around 75MB when the application hits
the error... and about 7-8 threads when this occurs. now i know that the
-Xmx option is for java objects only, and doesn't pertain to the virtual
machine's own memory requirements... but this observation seems to
indicate that each thread actually is consuming its own 75MB of memory.
this is also supported with each java thread only consuming about 4K of
shared memory, as reported by "top".

is what i think i'm observing actually happening? is each java thread
actually consuming that much extra memory (i.e. without sharing it all
through the single java virtual machine heap)? somehow, i doubt it... but
the out of memory errors seem to be happening well before 512MB of java
object information is created.

note, too, that i am not actually creating these seperate threads... my
app is single-threaded... the 7-8 are those threads executed intially by
the java virtual machine.

thanks for any insight on this matter... or even a link to a good source
that describes this.

thanks much again,

murat

--
Murat Tasan
mx**@po.cwru.edu
ta***@eecs.cwru.edu
mu*********@cwru.edu
http://genomics.cwru.edu