Object creation overhead

[Richard]

Hi all,

I am looking for some help on understanding the overhead associated with
object creation in Java.

I am writing an application where I have written a class to encapsulate some
text data.

The class is contains these private variables:

// private variables
private String id = null;
private String subject = null;
private String from = null;
private String date = null;
private String bytes = null;
private String lines = null ;
private boolean isread;
private String server = null;
private int port;
private String group = null;
Now I create each object with no parameters passed into the constructor, and
manually populate 6 of the String variables indicated above. Also the
server, port and group variables are populated for each object instance.

Of course there are accessor methods to deal with accessing the private
variables, in total about 29 methods for this class. The Class implements
Serializable and Comparable.

Now I am trying to figure out when one of these puppies is created, how much
memory does an instance take up. I am not sure how to correctly do this.

I determine the size of all the text contained by the object by using the
following method:

myObject.toString().getBytes()).length

where toString has been overridden in the class to create a StringBuffer.
The above method reports an average size of 233 bytes.

I create about 82,000 of these objects and place them into an ArrayList.
When I cacluate the following:

Runtime.getRuntime().totalMemory()

before and after the ArrayList creation and divide the difference by
ArrayList.size(), I come to the conclusion that each object in the
ArrayList is about 2300 bytes.

I want to reduce the size of this, since this ArrayList is using roughly
172MB. I use ArrayList.trimToSize() but it doesn't seem to do much....

How can I accurately determine each object in the ArrayLists size? That is
how much memory the class consumes and not just some of its variables?

Any suggestions on how to reduce the memory footprint of this class?

Thanks in advance,
Rich

p.s. in a similar app written in c++, the same List consumes about 18MB, and
I am trying to get a similar memory footprint as this....

[Anthony Borla]

"Richard" <ri*****@praxistech.com> wrote in message
news:PW5xb.174421$jy.168107@clgrps13...

Hi all,

I am looking for some help on understanding the overhead
associated with object creation in Java.

I am writing an application where I have written a class to
encapsulate some text data.

The class is contains these private variables:

// private variables
private String id = null;
private String subject = null;
private String from = null;
private String date = null;
private String bytes = null;
private String lines = null ;
private boolean isread;
private String server = null;
private int port;
private String group = null;
Now I create each object with no parameters passed into
the constructor, and manually populate 6 of the String variables
indicated above. Also the server, port and group variables are
populated for each object instance.

Of course there are accessor methods to deal with accessing
the private variables, in total about 29 methods for this class.
The Class implements Serializable and Comparable.
<SNIP>
Any suggestions on how to reduce the memory footprint of
this class?

General approaches:

* Reduce the size of each object

- Use 'proxy' objects to represent each 'real' object.
When the proxy is selected, the 'real' object is
instantiated from data stored elsewhere [e.g. in
a database, file, or cache]

- Minimise the number of fields in each object. For
example, instead of using separate 'String' objects
to store data such as 'subject', 'from', etc, just use a
single 'StringBuffer' to hold *all* this data, and
extract it from the 'StringBuffer' via accessors.
The 'StringBufferReader/Writer' classes could help
here

- Use alternate means to represent object data. For
example, instead of pbjects like 'String', encode data
in 'int' primitives and decode using bitwise operations
via accessors

* Reduce the number of objects in memory at the same
time, something that will, of course, require a redesign.
Not knowing your requirements prevents a detailed
response, but here are a few ideas:

- Adopt a database approach i.e. objects stored
externally and only retrieved on demand [probably
not feasable for you, but I thought I'd mention it :)]

- Use a some sort of caching system e.g. keep only
frequently accessed items in memory, etc

- If needing to process the whole chunk of objects
alter your algorithms to work on subsets ata a time

In general, I'd question the need to keep that many objects in memory [but
of course I don't know your requirements]. Hopefully, though, you now have a
few ideas to consider.

p.s. in a similar app written in c++, the same List consumes
about 18MB, and I am trying to get a similar memory footprint
as this....

Not surprising since there is no JVM and library routines to load into
memory :) !

I hope this helps.

Anthony Borla

[Richard]

Anthony Borla wrote:

General approaches:

* Reduce the size of each object

- Use 'proxy' objects to represent each 'real' object.
When the proxy is selected, the 'real' object is
instantiated from data stored elsewhere [e.g. in
a database, file, or cache]
Probably will have to do this, but I have not finalized an approach...


- Minimise the number of fields in each object. For
example, instead of using separate 'String' objects
to store data such as 'subject', 'from', etc, just use a
single 'StringBuffer' to hold *all* this data, and
extract it from the 'StringBuffer' via accessors.
The 'StringBufferReader/Writer' classes could help
here

- Use alternate means to represent object data. For
example, instead of pbjects like 'String', encode data
in 'int' primitives and decode using bitwise operations
via accessors

How about using a primitive like char or char[]?

My main concern was the overhead of the String class, not much for a couple
of thousand Strings, but now I am dealing with upwards of 100,000...

The end result is that this ArrayList is used in a TableModel. Looks like I
have a lot of re-writing to do that I didn't want. Oh well, at least it
will perform better...

Thank you for your help!
Rich

[nos]

please, what is a "proxy" object? how to make one?
"Anthony Borla" <aj*****@bigpond.com> wrote in message
news:jU******************@news-server.bigpond.net.au...

"Richard" <ri*****@praxistech.com> wrote in message
news:PW5xb.174421$jy.168107@clgrps13...

Hi all,

I am looking for some help on understanding the overhead
associated with object creation in Java.

I am writing an application where I have written a class to
encapsulate some text data.

The class is contains these private variables:

// private variables
private String id = null;
private String subject = null;
private String from = null;
private String date = null;
private String bytes = null;
private String lines = null ;
private boolean isread;
private String server = null;
private int port;
private String group = null;
Now I create each object with no parameters passed into
the constructor, and manually populate 6 of the String variables
indicated above. Also the server, port and group variables are
populated for each object instance.

Of course there are accessor methods to deal with accessing
the private variables, in total about 29 methods for this class.
The Class implements Serializable and Comparable.
<SNIP>

Any suggestions on how to reduce the memory footprint of
this class?

General approaches:

* Reduce the size of each object

- Use 'proxy' objects to represent each 'real' object.
When the proxy is selected, the 'real' object is
instantiated from data stored elsewhere [e.g. in
a database, file, or cache]

- Minimise the number of fields in each object. For
example, instead of using separate 'String' objects
to store data such as 'subject', 'from', etc, just use a
single 'StringBuffer' to hold *all* this data, and
extract it from the 'StringBuffer' via accessors.
The 'StringBufferReader/Writer' classes could help
here

- Use alternate means to represent object data. For
example, instead of pbjects like 'String', encode data
in 'int' primitives and decode using bitwise operations
via accessors

* Reduce the number of objects in memory at the same
time, something that will, of course, require a redesign.
Not knowing your requirements prevents a detailed
response, but here are a few ideas:

- Adopt a database approach i.e. objects stored
externally and only retrieved on demand [probably
not feasable for you, but I thought I'd mention it :)]

- Use a some sort of caching system e.g. keep only
frequently accessed items in memory, etc

- If needing to process the whole chunk of objects
alter your algorithms to work on subsets ata a time

In general, I'd question the need to keep that many objects in memory [but
of course I don't know your requirements]. Hopefully, though, you now have

a few ideas to consider.

p.s. in a similar app written in c++, the same List consumes
about 18MB, and I am trying to get a similar memory footprint
as this....

Not surprising since there is no JVM and library routines to load into
memory :) !

I hope this helps.

Anthony Borla

[Anthony Borla]

"nos" <no*@nospam.com> wrote in message
news:shbxb.316149$Tr4.986273@attbi_s03...

please, what is a "proxy" object? how to make one?

It's just a plain object that is used to represent or take the place of
another object. While useful in many situations [i.e. there can be many
reasons for using proxies including security reasons, and minimising
resources loadtimes etc], one particular use is to reduce the memory
footprint of a Collection-type of data structure.

For example, you need to have an 'ArrayList' object manage 1000 objects of
'MyClass'; this class looks like:

class MyClass
{
...
// Lots of fields ...
private long id;
private String name;
private String address;
...
}

If 'MyClass' has twenty 'String' fields then that means that memory for
20,000 'String' objects is needed, both to manage them, and for the 'String'
data. This is, of course, over and above the memory required for the
'MyClass' objects themselves.

The 'proxy' class approach won't see a reduction in the number of 'MyClass'
objects, merely a reduction in the amount of memory each may require for its
field data. So, the original class is now split into two; there is the proxy
class:

class MyClassProxy
{
...
// Only a couple of fields needed...
private long id;
...
// Method to retrieve the actual object proxied by this one
public MyClass getMyClassObject() { ... }
...
}

and a modified, probably pared-down, version of the original class:

class MyClass
{
...
// Still lots of fields, but probably less methods ...
private long id;
private String name;
private String address;
...
}

With the 'ArrayList' object loaded with 'MyClassProxy' objects, less memory
is being used. Only when a 'MyClassProxy' object is selected is a 'MyClass'
object instantiated, using data stored somewhere else *outside memory* [e.g.
file, database, URL, etc].

Note:

* This approach has many variants - I have provided a very simple
example, and ignored / glossed over the implementation issues.
However the general approach is:

- Use proxy object in place of real object

- When proxy object is selected have it retrieve / create
the real object

* Proxy cannot [usually] be used where access / retrieval time is
crucial

More substantial examples, and relevant in-depth discussions may be found by
Google searching for 'Proxy Design Pattern'.

I hope this helps.

Anthony Borla

[Anthony Borla]

"Richard" <ri*****@praxistech.com> wrote in message
news:obbxb.31522$oN2.7335@edtnps84...

Anthony Borla wrote:

How about using a primitive like char or char[]?

Sure :). However you can use a single 'int' to store data for multiple
fields - how's *that* for space-saving ;) !!!

My main concern was the overhead of the String class, not
much for a couple of thousand Strings, but now I am
dealing with upwards of 100,000...

The end result is that this ArrayList is used in a TableModel.
Looks like I have a lot of re-writing to do that I didn't want.
Oh well, at least it will perform better...

A redesign is often the best cure for ailing performance.

Thank you for your help!

A pleasure :) !

Cheers,

Anthony Borla

[nos]

ya, very helpful, thanks for taking the time to explain
"Anthony Borla" <aj*****@bigpond.com> wrote in message
news:x0*****************@news-server.bigpond.net.au...

"nos" <no*@nospam.com> wrote in message
news:shbxb.316149$Tr4.986273@attbi_s03...

please, what is a "proxy" object? how to make one?

It's just a plain object that is used to represent or take the place of
another object. While useful in many situations [i.e. there can be many
reasons for using proxies including security reasons, and minimising
resources loadtimes etc], one particular use is to reduce the memory
footprint of a Collection-type of data structure.

For example, you need to have an 'ArrayList' object manage 1000 objects of
'MyClass'; this class looks like:

class MyClass
{
...
// Lots of fields ...
private long id;
private String name;
private String address;
...
}

If 'MyClass' has twenty 'String' fields then that means that memory for
20,000 'String' objects is needed, both to manage them, and for the

'String' data. This is, of course, over and above the memory required for the
'MyClass' objects themselves.

The 'proxy' class approach won't see a reduction in the number of 'MyClass' objects, merely a reduction in the amount of memory each may require for its field data. So, the original class is now split into two; there is the proxy class:

class MyClassProxy
{
...
// Only a couple of fields needed...
private long id;
...
// Method to retrieve the actual object proxied by this one
public MyClass getMyClassObject() { ... }
...
}

and a modified, probably pared-down, version of the original class:

class MyClass
{
...
// Still lots of fields, but probably less methods ...
private long id;
private String name;
private String address;
...
}

With the 'ArrayList' object loaded with 'MyClassProxy' objects, less memory is being used. Only when a 'MyClassProxy' object is selected is a 'MyClass' object instantiated, using data stored somewhere else *outside memory* [e.g. file, database, URL, etc].

Note:

* This approach has many variants - I have provided a very simple
example, and ignored / glossed over the implementation issues.
However the general approach is:

- Use proxy object in place of real object

- When proxy object is selected have it retrieve / create
the real object

* Proxy cannot [usually] be used where access / retrieval time is
crucial

More substantial examples, and relevant in-depth discussions may be found by Google searching for 'Proxy Design Pattern'.

I hope this helps.

Anthony Borla

[Richard]

Anthony Borla wrote:

How about using a primitive like char or char[]?

Sure :). However you can use a single 'int' to store data for multiple
fields - how's *that* for space-saving ;) !!!

Actually I would be interested in your approach using this technique...
This could come in handy....

The end result is that this ArrayList is used in a TableModel.
Looks like I have a lot of re-writing to do that I didn't want.
Oh well, at least it will perform better...

A redesign is often the best cure for ailing performance.

Indeed, and since there was no original design, its been continuously
redesigned all the time :-)

I did find a partial solution to my problem, I am dealing with NNTP article
headers, specifically mulitpart messages. I realized after you pointed out
the obvious "reduce the number of fields" advice that I only need to store
subsequent article IDs for any multipart segment.

Further more, incomplete multiparts are discarded. I also took your
suggestion of using StringBuffer for the fields. Memory consumption is now
better, but there are still issues.

For instance, I am using the old way of storing a List, by writing it to a
file as an ObjectStream. I haven't looked into the beans package of
converting this List into an XML representation yet, but there is an issue
of concurrent access of the file by multiple threads, and representing
state of individual headers when they are in use in a TableModel.

I was thinking of using a Proxy like you suggested but I only think that
would solve one issue, the memory consumption.

So a big redesign in in the works, I'll probably come up with a solution
around the christmas holidays when I have the time to work on it...

By the way, does anybody know the default max heap size of the java 1.4.2
plugin? I don't want to have to tell my mother how to modify this stuff in
her control panel ;-)

Rich