Which uses less memory?

[bret.wortman]

I'm not sure if this is as easy a question as I'd like it to be, but
here goes....

I'm working on an application that is very memory intensive, so we're
trying to reduce the memory footprint of classes wherever possible. I
have a need for a class which is able to have a type identifier which
can be examined at run-time to determine whether that class (called
Domain) contains data I care about or not.

I've thought of two ways to implement this:

1. Add a type attribute and set it to a descriptive string.
2. Create marker classes and use multiple inheritance to "attach"
these markers to specific Domains.

Here's the kicker: I need to serialize these Domains and send them to/
from Java code as well as work on them using Python. We're looking to
use Hessian and pyactivemq to handle the Java/Python interfaces.
Which, I guess, leads to the following group of questions:

1. Which method has the smaller footprint within the Python engine?
2. Do these protocols (Hessian and Stomp) preserve the class
information when the class is serialized?

Any input would be most welcome. Thanks!
Bret Wortman

[Nick Craig-Wood]

br**********@gmail.com <br**********@gmail.comwrote:

I'm not sure if this is as easy a question as I'd like it to be, but
here goes....

I'm working on an application that is very memory intensive, so we're
trying to reduce the memory footprint of classes wherever possible. I
have a need for a class which is able to have a type identifier which
can be examined at run-time to determine whether that class (called
Domain) contains data I care about or not.

I've thought of two ways to implement this:

1. Add a type attribute and set it to a descriptive string.
2. Create marker classes and use multiple inheritance to "attach"
these markers to specific Domains.

Here's the kicker: I need to serialize these Domains and send them to/
from Java code as well as work on them using Python. We're looking to
use Hessian and pyactivemq to handle the Java/Python interfaces.
Which, I guess, leads to the following group of questions:

1. Which method has the smaller footprint within the Python engine?
2. Do these protocols (Hessian and Stomp) preserve the class
information when the class is serialized?

Any input would be most welcome. Thanks!

I don't know the answers to your specific questions but we managed to
1/3 the memory requirement of our app by identifying the class having
the most instances (about 1,000,000 in our case) and adding __slots__
to it!

I'd guess that if you __slot__-ed the Domain class then you'll find
the overhead of a type attribute is minimal (4 bytes per instance I
think).

No idea about Hessian or Stomp (never heard of them!) but classes with
__slot__s are normal classes which would pickle or unpickle.

--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick

[Gabriel Genellina]

En Sat, 17 Nov 2007 19:30:04 -0300, Nick Craig-Wood <ni**@craig-wood.com>
escribi�:

> I'm working on an application that is very memory intensive, so we're
trying to reduce the memory footprint of classes wherever possible. I

I'd guess that if you __slot__-ed the Domain class then you'll find
the overhead of a type attribute is minimal (4 bytes per instance I
think).

No idea about Hessian or Stomp (never heard of them!) but classes with
__slot__s are normal classes which would pickle or unpickle.

Actually classes with __slots__ require that you write your own
__getstate__ and __setstate__. The following implementation may be enough
in simple cases:

pyclass X(object):
.... __slots__ = ('foo','bar')
....
pyx = X()
pyx.foo = 123
pydumps(x)
Traceback (most recent call last):
....
TypeError: a class that defines __slots__ without defining __getstate__
cannot b
e pickled

class X(object):
__slots__ = ('foo','bar')
def __getstate__(self):
return dict((name, getattr(self, name))
for name in self.__slots__
if hasattr(self, name))
def __setstate__(self, state):
for name,value in state.iteritems():
setattr(self, name, value)

pyx = X()
pyx.foo = 123
pyp = dumps(x)
pyx2 = loads(p)
pytype(x2)
<class '__main__.X'>
pyx2.foo
123
pyx2.bar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: bar

--
Gabriel Genellina