473,700 Members | 2,429 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

__slots__ vs __dict__


Classes using __slots__ seem to be quite a bit smaller and faster
to instantiate than regular Python classes using __dict__.

Below are the results for the __slots__ and __dict__ version of a
specific class with 16 attributes. Each line in the tables shows the
number of instances created so far, the total memory usage in Bytes,
the CPU time in secs, the average size per instance in Bytes and the
average CPU time per instance in micseconds.

Instances of this particular class with __slots__ are almost 6x
smaller and nearly 3x faster to create than intances of the __dict__
version. Results for other classes will vary, obviously.

Comments?

/Jean Brouwers
ProphICy Semiconductor, Inc.
PS) The tests were run on a dual 2.4 GHz Xeon system with RedHat
8.0 and Python 2.3.2. The test script is attached but keep in mind
that it only has been tested on Linux. It will not work elsewhere
due to the implementation of the memory() function.
testing __slots__ version ...
4096 insts so far: 3.0e+05 B 0.030 sec 73.0 B/i 7.3 usec/i
8192 insts so far: 8.8e+05 B 0.070 sec 107.5 B/i 8.5 usec/i
16384 insts so far: 1.5e+06 B 0.150 sec 92.2 B/i 9.2 usec/i
32768 insts so far: 3.3e+06 B 0.280 sec 101.0 B/i 8.5 usec/i
65536 insts so far: 6.6e+06 B 0.560 sec 101.2 B/i 8.5 usec/i
131072 insts so far: 1.4e+07 B 1.200 sec 103.4 B/i 9.2 usec/i
262144 insts so far: 2.7e+07 B 2.480 sec 103.4 B/i 9.5 usec/i
524288 insts so far: 5.5e+07 B 5.630 sec 104.0 B/i 10.7 usec/i
1048576 insts so far: 1.1e+08 B 13.980 sec 104.0 B/i 13.3 usec/i
1050000 insts total: 1.1e+08 B 14.000 sec 103.9 B/i 13.3 usec/i
testing __dict__ version ...
4096 insts so far: 2.4e+06 B 0.050 sec 595.0 B/i 12.2 usec/i
8192 insts so far: 4.6e+06 B 0.090 sec 564.5 B/i 11.0 usec/i
16384 insts so far: 9.5e+06 B 0.180 sec 581.8 B/i 11.0 usec/i
32768 insts so far: 1.9e+07 B 0.370 sec 582.2 B/i 11.3 usec/i
65536 insts so far: 3.8e+07 B 0.830 sec 582.6 B/i 12.7 usec/i
131072 insts so far: 7.6e+07 B 1.760 sec 582.7 B/i 13.4 usec/i
262144 insts so far: 1.5e+08 B 4.510 sec 582.8 B/i 17.2 usec/i
524288 insts so far: 3.1e+08 B 12.820 sec 582.8 B/i 24.5 usec/i
1048576 insts so far: 6.1e+08 B 38.370 sec 583.1 B/i 36.6 usec/i
1050000 insts total: 6.1e+08 B 38.380 sec 583.1 B/i 36.6 usec/i
-------------------------------slots.py-------------------------------
<pre>

from time import clock as time_clock
def cputime(since=0 .0):
'''Return CPU in secs.
'''
return time_clock() - since
import os
_proc_status = '/proc/%d/status' % os.getpid() # Linux only
_scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
'KB': 1024.0, 'MB': 1024.0*1024.0}

def _VmB(VmKey):
global _scale
try: # get the /proc/<pid>/status pseudo file
t = open(_proc_stat us)
v = [v for v in t.readlines() if v.startswith(Vm Key)]
t.close()
# convert Vm value to bytes
if len(v) == 1:
t = v[0].split() # e.g. 'VmRSS: 9999 kB'
if len(t) == 3: ## and t[0] == VmKey:
return float(t[1]) * _scale.get(t[2], 0.0)
except:
pass
return 0.0

def memory(since=0. 0):
'''Return process memory usage in bytes.
'''
return _VmB('VmSize:') - since

def stacksize(since =0.0):
'''Return process stack size in bytes.
'''
return _VmB('VmStk:') - since

def slots(**kwds):
'''Return the slots names as sequence.
'''
return tuple(kwds.keys ())

# __slots__ version
class SlotsClass(obje ct):
__slots__ = slots(_attr1= False,
_attr2= None,
_attr3= None,
_attr4= None,
_attr5= None,
_attr6= None,
_attr7= 0,
_attr8= None,
_attr9= None,
_attr10=None,
_attr11=None,
_attr12=None,
_attr13=None,
_attr14=None,
_attr15=None,
_attr16=None)

def __init__(self, tuple4, parent):
self._attr1 = False
self._attr2 = None
self._attr3 = None
self._attr4 = None
self._attr5 = None
self._attr6 = None
if parent:
self._attr7 = parent._attr7 + 1
self._attr8 = parent._attr8
self._attr9 = parent._attr9
self._attr10 = parent
self._attr11 = parent._attr11
self._attr12 = parent._attr12
else:
self._attr7 = 0
self._attr8 = None
self._attr9 = None
self._attr10 = None
self._attr11 = self
self._attr12 = None
self._attr13, self._attr14, self._attr15, self._attr16 = tuple4
# __dict__ version
class DictClass(objec t):
_attr1 = None
_attr2 = None
_attr3 = None
_attr4 = None
_attr5 = None
_attr6 = None
_attr7 = 0
_attr8 = None
_attr9 = None
_attr10 = None
_attr11 = None
_attr12 = None
_attr13 = None
_attr14 = None
_attr15 = None
_attr16 = None

def __init__(self, tuple4, parent):
if parent:
self._attr7 = parent._attr7 + 1
self._attr8 = parent._attr8
self._attr9 = parent._attr9
self._attr10 = parent
self._attr11 = parent._attr11
self._attr12 = parent._attr12
else:
self._attr11 = self
self._attr13, self._attr14, self._attr15, self._attr16 = tuple4
if __name__ == '__main__':

import sys

def report(txt, n, b0, c0):
c = cputime(c0);
b = memory(b0)
print "%8d insts %s: %8.1e B %7.3f sec %6.1f B/i %6.1f usec/i" \
% (n, txt, b, c, b/n, 1.0e6*c/n)

if not sys.platform.st artswith('linux '):
raise NotImplementedE rror, "%r not supported" % sys.platform

if 'dict' in sys.argv[1:]:
print 'testing __dict__ version ...'
testClass = DictClass
else:
print 'testing __slots__ version ...'
testClass = SlotsClass

t4 = ('', 0, 0, [])
b0 = memory()
c0 = cputime()
p = testClass(t4, None)
n, m = 1, 4096
# generate 1+ M instances
while n < 1050000: # 1048576:
p = testClass(t4, p)
n += 1
if n >= m: # occasionally print stats
m += m
report('so far', n, b0, c0)
report(' total', n, b0, c0)

</pre>
Jul 18 '05 #1
5 2929
Jean Brouwers <JB*******@Prop hICy.com> writes:
Classes using __slots__ seem to be quite a bit smaller and faster
to instantiate than regular Python classes using __dict__.

[snip]

Yes, but instances usually will not have a Class.__dict__ attribute anymore.
Thus the following code throws an exception on binding new attributes on a
per-instance basis:

# start
class Foo (object):
__slots__ = "_test"

def __init__ (self):
self._test = None

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end

Rebinding __slots__ in __setattr__ fails with a bus error on my system (it
should not be possible anyways, because __slots__ is a tuple):

# start
class Foo (object):
__slots__ = "_test", "_test2"

def __init__ (self):
self._test = 1
self._test2 = 2

def __setattr__ (self, name, value):
# just test a simple rebinding
self.__slots__ = self.__slots__
return

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end

--- gdb backtrace

(gdb) r foo.py
....
Program received signal SIGBUS, Bus error.
0x08070050 in PyDict_GetItem ()
(gdb) bt
#0 0x08070050 in PyDict_GetItem ()
#1 0x08080ab7 in _PyType_Lookup ()
.....
Regards
Marcus

--
We don't understand the software, and sometimes we don't understand the
hardware, but we can *see* the blinking lights!
Jul 18 '05 #2
On Thu, May 13, 2004 at 12:07:54AM +0200, Marcus von Appen wrote:
Jean Brouwers <JB*******@Prop hICy.com> writes:
Classes using __slots__ seem to be quite a bit smaller and faster
to instantiate than regular Python classes using __dict__. [snip]

Yes, but instances usually will not have a Class.__dict__ attribute anymore.
Thus the following code throws an exception on binding new attributes on a
per-instance basis:

# start
class Foo (object):
__slots__ = "_test"

def __init__ (self):
self._test = None

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end


This code is a syntax error for me -- the final return isn't in a function.
Rebinding __slots__ in __setattr__ fails with a bus error on my system (it
should not be possible anyways, because __slots__ is a tuple):

# start
class Foo (object):
__slots__ = "_test", "_test2"

def __init__ (self):
self._test = 1
self._test2 = 2

def __setattr__ (self, name, value):
# just test a simple rebinding
self.__slots__ = self.__slots__
return

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end


Also a syntax error. I also don't get a bus error, just infinite recursion
(because assigning to self.__slots__ calls __setattr__, but __slots__ isn't
an attribute of the instance). What version of Python, and what platform?
I'm guessing you're on something like FreeBSD where Python has had trouble
coping gracefully with infinite recursion.

-Andrew.
Jul 18 '05 #3
Andrew Bennetts <an************ ***@puzzling.or g> writes:

[...]
# start
class Foo (object):
__slots__ = "_test"

def __init__ (self):
self._test = None

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end
This code is a syntax error for me -- the final return isn't in a function.


Sorry for that messy code - the return is absolutely wrong (no idea, why I put
it there after typing the example in).
Rebinding __slots__ in __setattr__ fails with a bus error on my system (it
should not be possible anyways, because __slots__ is a tuple):

# start
class Foo (object):
__slots__ = "_test", "_test2"

def __init__ (self):
self._test = 1
self._test2 = 2

def __setattr__ (self, name, value):
# just test a simple rebinding
self.__slots__ = self.__slots__
return

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end


Also a syntax error. I also don't get a bus error, just infinite recursion
(because assigning to self.__slots__ calls __setattr__, but __slots__ isn't
an attribute of the instance). What version of Python, and what platform?


Typed in a wrong line again here. Change it to something appropriate like

def __setattr__ (self, name, value):
# just test a simple rebinding
self.__slots__. __add__ (tuple (name))
return
I'm guessing you're on something like FreeBSD where Python has had trouble
coping gracefully with infinite recursion.


Right, but this was caused by typing in wrong code. Sorry for that.

Regards
Marcus

--
We don't understand the software, and sometimes we don't understand the
hardware, but we can *see* the blinking lights!
Jul 18 '05 #4
On Thu, May 13, 2004 at 06:55:39AM +0200, Marcus von Appen wrote:
Andrew Bennetts <an************ ***@puzzling.or g> writes:

[...]

# start
class Foo (object):
__slots__ = "_test", "_test2"

def __init__ (self):
self._test = 1
self._test2 = 2

def __setattr__ (self, name, value):
# just test a simple rebinding
self.__slots__ = self.__slots__
return

if __name__ == "__main__":
f = Foo ()
f.testvar = "test"
return
# end


Also a syntax error. I also don't get a bus error, just infinite recursion
(because assigning to self.__slots__ calls __setattr__, but __slots__ isn't
an attribute of the instance). What version of Python, and what platform?


Typed in a wrong line again here. Change it to something appropriate like

def __setattr__ (self, name, value):
# just test a simple rebinding
self.__slots__. __add__ (tuple (name))
return


Now your example runs just fine (i.e. the script terminates normally, and
nothing happens). I still don't see any error. (And why do you keep
putting a redundant return at the end of your functions?)

Are you sure this is the same code that you get a crash with? What version
of Python are you using? (I've tested with 2.2.3 and 2.3.3).

-Andrew.
Jul 18 '05 #5
Andrew Bennetts <an************ ***@puzzling.or g> writes:

[...]
Now your example runs just fine (i.e. the script terminates normally, and
nothing happens). I still don't see any error.
I thought you would inspect the objects yourself. Let me show you another (now
working) example, which will explain it:

------
class DictClass (object):
test = "DictTest"

def __init__ (self):
pass

class SlotClass (object):
__slots__ = "test"

def __init__ (self):
self.test = "SlotTest"

def print_obj (obj):
# inspect .test and show the object attributes
print obj.test
print dir (obj)

# bind new attribute and inspect the object attributes
obj.new_test = "Test"
print obj.new_test
print dir (obj)
print "------"
return

if __name__ == "__main__":
dict_obj = DictClass ()
slot_obj = SlotClass ()

print_obj (dict_obj)
print_obj (slot_obj)

---

You will get something like the following output:

----------------
DictTest
['__class__', '__delattr__', '__dict__', ....., '__weakref__', 'test']

Test
['__class__', '__delattr__', '__dict__', ...., '__weakref__', 'new_test',
'test']

------

SlotTest
['__class__', '__delattr__', ...., '__slots__', '__str__', 'test']

Traceback (most recent call last):
File "foo.py", line 31, in ?
print_obj (slot_obj)
File "foo.py", line 19, in print_obj
obj.new_test = "Test"
AttributeError: 'SlotClass' object has no attribute 'new_test'
----------------

As you will notice, a __slot__ed object/class has no __weakref__ nor __dict__
attribute.
Thus binding new object attributes will fail with an AttributeError.

So let's try it again with using __setattr__ in SlotClass:

class SlotClass (object):
test = "SlotTest"
test2 = "SlotTest2" # just for making a tuple creation easier
__slots__ = test, test2

def __init__ (self):
pass

def __setattr__ (self, name, value):
# create a dict here to add its key to the __slot__ tuple
self.__slots__. __add__ (tuple ((name)))
return
def print_obj (obj):
# inspect .test and show the object attributes
print obj.test
print dir (obj)

# bind new attribute and inspect the object attributes
obj.new_test = "Test"
print obj.new_test
print dir (obj)
print "------"
return

if __name__ == "__main__":

#dict_obj = DictClass ()
slot_obj = SlotClass ()

#print_obj (dict_obj)
print_obj (slot_obj)

----
As you will see, you get an AttributeError again.tuple ((name))was not
concatenated to __slots__.
I think that should explain enough about the advantage and disadvantage of
__slots__ here.

(And why do you keep putting a redundant return at the end of your functions?)
I'm used to it :-).
Are you sure this is the same code that you get a crash with? What version
of Python are you using? (I've tested with 2.2.3 and 2.3.3).


No, I put messy code without _really_ thinking about it in my first post.
I did not realize that self.__slots__ = self._slots in __setattr__ will end
up in a recursion...

Regards
Marcus

--
We don't understand the software, and sometimes we don't understand the
hardware, but we can *see* the blinking lights!
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
2193
by: simon place | last post by:
Removing __slots__ ~~~~~~~~~~~~~~~~~~~ To do this nicely requires the renaming of __dict__ to, say, __attribs__ , ( since dict is unnecessarily unspecific, this seem like a small improvement in itself. ) then using the setting of __attribs__ to a non-mutable type (ie tuple of attribute names) to indicate the behaviour of __slots__, rendering it unnecessary, this I think is a good simplification.
1
1990
by: anabell | last post by:
I have a code like this: sqlString = 'INSERT INTO ' + self.TableName + ' VALUES (' + self.TableFields + ')' self.cursor.execute(sqlString, self.__dict__) This works correctly. However, I'm applying __slots__ in my script. And doing so would need the above statement modified. How will __slots__ perform the same task defined above? Is there a container that holds the values of attributes contained in __slots__? In __dict__, you have...
3
1635
by: Nick Jacobson | last post by:
The __slots__ attribute of new-style classes can reduce memory usage when there are millions of instantiations of a class. So would a __slots__ attribute for functions/methods have a similar benefit? i.e. could a function using __slots__ use significantly less memory, and therefore run faster, if called millions of times? If so, it will hopefully be in a future version of Python.
7
1663
by: Porky Pig Jr | last post by:
Hello, I"m still learning Python, but going through the Ch 5 OOP of Nutshell book. There is discussion on __slots__, and my understanding from reading this section is that if I have a class Rectangle (as defined in some prior sections), and then I provide definition class OptimizedRectangle(Rectangle): __slots__ = 'width', 'heigth' I can use the instance of OptimizedRectangle, say, x, with 'width' and 'heigth', but (quoting the book)...
1
1563
by: Alex | last post by:
I would greatly appreciate an advice on the following matter that was very much discussed circa 2002 but in all this discussion I could not find the final answer with a code example. Neither I can find it in Python documentation. I have a series of new classes with child-parent relationship and each has unique __slots__. They don't have __dict__ . I need to be able to pickle and unpickle them. As far as I could understand, I need to...
4
1493
by: dan.j.weber | last post by:
I see some programs declaring the names of class variables in "__slots__". I've looked this up and the docs say something about old and new style classes, whatever that means. Can someone give me a simple, brief explanation of what __slots__ are and when I should use them? Thanks.
3
1566
by: Schüle Daniel | last post by:
Hello, consider this code >>> class A(object): .... def __init__(self): .... self.a = 1 .... self.b = 2 .... >>> class B(A):
15
531
by: David Isaac | last post by:
1. "Without a __dict__ variable, instances cannot be assigned new variables not listed in the __slots__ definition." So this seemed an interesting restriction to impose in some instances, but I've noticed that this behavior is being called by some a side effect the reliance on which is considered unPythonic. Why? 2. What is a simple example where use of slots has caused "subtle" problems, as some claim it will?
27
1733
by: Licheng Fang | last post by:
Python is supposed to be readable, but after programming in Python for a while I find my Python programs can be more obfuscated than their C/C ++ counterparts sometimes. Part of the reason is that with heterogeneous lists/tuples at hand, I tend to stuff many things into the list and *assume* a structure of the list or tuple, instead of declaring them explicitly as one will do with C structs. So, what used to be struct nameval { char *...
0
9217
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9076
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8974
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8926
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6563
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4404
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4659
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3092
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2392
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.