473,545 Members | 4,914 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

New+old-style multiple inheritance

We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance. In so doing we have
uncovered some unexpected behaviour:

<quote>
class Foo:
pass

class Bar(object):
pass

class Baz(Foo,Bar):
pass

# Monkey-patch Foo to add a special method
def my_nonzero(self ):
print "my_nonzero called"
return False
Foo.__nonzero__ = my_nonzero

b = Baz()

print "doing the test on Baz(Foo,Bar). Should return false"
if b:
print "true"
else:
print "false"
</quote>

Produces this output:

doing the test on Baz(Foo,Bar). Should return false
true

With some experimentation it is clear that this behaviour only occurs
when you combine new+old-style multiple inheritance, monkey-patching
and special methods. If Foo and Bar are either old or new-style it
works. calling b.__nonzero__() directly works. Defining __nonzero__
within Foo works.

I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.

Any ideas?

Cheers,
Stephen.
Dec 18 '07 #1
13 2475
On Dec 18, 7:08 am, "stephen...@goo glemail.com"
<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance. In so doing we have
uncovered some unexpected behaviour:
<snip>
>
I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.
You've already discovered why--you're mixing old and new style
classes.

Monkey patching is definitely unpythonic. You must be a Ruby guy. Why
don't you try doing something else to get the behavior you want,
something more explicit?

Dec 18 '07 #2
On 18 dic, 12:08, "stephen...@goo glemail.com"
<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance. In so doing we have
uncovered some unexpected behaviour:

<quote>
class Foo:
pass

class Bar(object):
pass

class Baz(Foo,Bar):
pass

# Monkey-patch Foo to add a special method
def my_nonzero(self ):
print "my_nonzero called"
return False
Foo.__nonzero__ = my_nonzero

b = Baz()

print "doing the test on Baz(Foo,Bar). Should return false"
if b:
print "true"
else:
print "false"
</quote>

Produces this output:

doing the test on Baz(Foo,Bar). Should return false
true

With some experimentation it is clear that this behaviour only occurs
when you combine new+old-style multiple inheritance, monkey-patching
and special methods. If Foo and Bar are either old or new-style it
works. calling b.__nonzero__() directly works. Defining __nonzero__
within Foo works.

I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.
I think I can barely explain what happens here (but I may be
absolutely wrong! Please someone with greater knowledge of Python
innards correct whatever is wrong on my description!)

type objects contain "slots" (function pointers) corresponding to the
"magic" methods like __nonzero__ (this one is stored into the
nb_nonzero slot). new-style classes are types; old-style classes are
not (they're all instances of classobj).
The slots are populated when a new type is created (e.g., when
creating a new-style class) and are updated when a magic attribute is
set onto the type. By example, setting the __nonzero__ attribute on a
new-style class updates the nb_nonzero slot. old-style classes just
store the magic attribute in its __dict__. Note that if you patch Foo
*before* defining Baz, it works fine, because Baz sees the magic
attribute and can populate its nb_nonzero slot when the new type is
created.

When you define Baz, neither Foo nor Bar have a __nonzero__ at this
time, so the nb_nonzero slot on Baz is empty.
Later, when you alter Foo, Baz cannot notice it (Foo has no way to
notify Baz that something changed).

If Foo were a new-style class, things are different: new-style classes
maintain a list of subclasses, and the subclasses can then be notified
of changes. In particular, setting a magic attribute on a base class
notifies all its subclasses, and the corresponding slots are updated.

The problem seems to be exactly that: old-style base classes can't
notify its derived new-style classes when magic methods are added, so
the corresponding slots aren't updated. Always asking the base class
whether it has a magic method or not would slow down all method
lookups.

To force a slot update:

pyBaz.__nonzero __ = "xxx"
pydel Baz.__nonzero__
pybool(b)
my_nonzero called
False

(setting or deleting __nonzero__ triggers the slot update)

This is *why* it happens. How to avoid this... well, don't do that in
the first place :) Or try to patch the base class *before* the new-
style derived class is defined. Or replace the derived class with a
new version of itself (once the base was patched). Or, if you know all
the derived classes, force a slot update on them as above.

--
Gabriel Genellina
Dec 18 '07 #3
On Dec 18, 10:08 am, "stephen...@goo glemail.com"
<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance.
New library? Geez, if people are dumb enough to do this, are you sure
you want your application to depend on their library?

Sometimes you have to work with code that's not up to your standards,
but come on.
Carl Banks
Dec 18 '07 #4
On Dec 18, 2:09 pm, Jonathan Gardner
<jgardner.jonat hangardner....@ gmail.comwrote:
On Dec 18, 7:08 am, "stephen...@goo glemail.com"

<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance. In so doing we have
uncovered some unexpected behaviour:

<snip>
I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.

You've already discovered why--you're mixing old and new style
classes.

Monkey patching is definitely unpythonic. You must be a Ruby guy. Why
don't you try doing something else to get the behavior you want,
something more explicit?
Time and place.

A well-considered, timely monkey-patch can sometimes save all kinds of
workarounds and other headaches.
Carl Banks
Dec 18 '07 #5
On Dec 18, 8:25 pm, Carl Banks <pavlovevide... @gmail.comwrote :
On Dec 18, 2:09 pm, Jonathan Gardner

<jgardner.jonat hangardner....@ gmail.comwrote:
On Dec 18, 7:08 am, "stephen...@goo glemail.com"
<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance. In so doing we have
uncovered some unexpected behaviour:
<snip>
I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.
You've already discovered why--you're mixing old and new style
classes.
Monkey patching is definitely unpythonic. You must be a Ruby guy. Why
don't you try doing something else to get the behavior you want,
something more explicit?

Time and place.

A well-considered, timely monkey-patch can sometimes save all kinds of
workarounds and other headaches.

Carl Banks
Indeed, I chuckled at the idea I was a Ruby programmer. Of course
it's a bit of a hack but monkey-patching has been common place in the
Zope world for years (maybe Zope3 has rid itself of the problem).

Agreed, mixing new and old-style classes in multiple inheritance is
asking for trouble (although the Guido's essay on the new-style
classes suggests you can: http://www.python.org/download/relea.../#subclassing).
I won't name and shame the library involved but unfortunately we can't
always pick and choose the tools we work with unless we want to write
(or rewrite) everything ourselves.

Cheers,
Stephen.
Dec 18 '07 #6
Thank Gabriel,

That sounds exactly right and your work-around provides confirmation.
This code is an expedient solution for us so I expect we'll go with
forcing a slot update. Hopefully the library concerned is going to
fix the problem in a future release.

Cheers,
Stephen.

On Dec 18, 7:40 pm, Gabriel Genellina <gagsl-...@yahoo.com.a rwrote:
On 18 dic, 12:08, "stephen...@goo glemail.com"

<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance. In so doing we have
uncovered some unexpected behaviour:
<quote>
class Foo:
pass
class Bar(object):
pass
class Baz(Foo,Bar):
pass
# Monkey-patch Foo to add a special method
def my_nonzero(self ):
print "my_nonzero called"
return False
Foo.__nonzero__ = my_nonzero
b = Baz()
print "doing the test on Baz(Foo,Bar). Should return false"
if b:
print "true"
else:
print "false"
</quote>
Produces this output:
doing the test on Baz(Foo,Bar). Should return false
true
With some experimentation it is clear that this behaviour only occurs
when you combine new+old-style multiple inheritance, monkey-patching
and special methods. If Foo and Bar are either old or new-style it
works. calling b.__nonzero__() directly works. Defining __nonzero__
within Foo works.
I know this level of messing with python internals is a bit risky but
I'm wondering why the above code doesn't work.

I think I can barely explain what happens here (but I may be
absolutely wrong! Please someone with greater knowledge of Python
innards correct whatever is wrong on my description!)

type objects contain "slots" (function pointers) corresponding to the
"magic" methods like __nonzero__ (this one is stored into the
nb_nonzero slot). new-style classes are types; old-style classes are
not (they're all instances of classobj).
The slots are populated when a new type is created (e.g., when
creating a new-style class) and are updated when a magic attribute is
set onto the type. By example, setting the __nonzero__ attribute on a
new-style class updates the nb_nonzero slot. old-style classes just
store the magic attribute in its __dict__. Note that if you patch Foo
*before* defining Baz, it works fine, because Baz sees the magic
attribute and can populate its nb_nonzero slot when the new type is
created.

When you define Baz, neither Foo nor Bar have a __nonzero__ at this
time, so the nb_nonzero slot on Baz is empty.
Later, when you alter Foo, Baz cannot notice it (Foo has no way to
notify Baz that something changed).

If Foo were a new-style class, things are different: new-style classes
maintain a list of subclasses, and the subclasses can then be notified
of changes. In particular, setting a magic attribute on a base class
notifies all its subclasses, and the corresponding slots are updated.

The problem seems to be exactly that: old-style base classes can't
notify its derived new-style classes when magic methods are added, so
the corresponding slots aren't updated. Always asking the base class
whether it has a magic method or not would slow down all method
lookups.

To force a slot update:

pyBaz.__nonzero __ = "xxx"
pydel Baz.__nonzero__
pybool(b)
my_nonzero called
False

(setting or deleting __nonzero__ triggers the slot update)

This is *why* it happens. How to avoid this... well, don't do that in
the first place :) Or try to patch the base class *before* the new-
style derived class is defined. Or replace the derived class with a
new version of itself (once the base was patched). Or, if you know all
the derived classes, force a slot update on them as above.

--
Gabriel Genellina
Dec 18 '07 #7

<st********@goo glemail.comwrot e in message
news:a9******** *************** ***********@t1g 2000pra.googleg roups.com...
| Thank Gabriel,
|
| That sounds exactly right and your work-around provides confirmation.
| This code is an expedient solution for us so I expect we'll go with
| forcing a slot update. Hopefully the library concerned is going to
| fix the problem in a future release.

To work with 3.0, they will have to, or rather it will be done for them,
since there will only be new style classes.

Dec 19 '07 #8
On Dec 18, 9:25 pm, Carl Banks <pavlovevide... @gmail.comwrote :
A well-considered, timely monkey-patch can sometimes save all kinds of
workarounds and other headaches.

Carl Banks
+1

Michele Simionato
Dec 19 '07 #9
On Dec 18, 3:16 pm, Carl Banks <pavlovevide... @gmail.comwrote :
On Dec 18, 10:08 am, "stephen...@goo glemail.com"

<stephen...@goo glemail.comwrot e:
We are trying to monkey-patch a third-party library that mixes new and
old-style classes with multiple inheritance.

New library? Geez, if people are dumb enough to do this, are you sure
you want your application to depend on their library?

Sometimes you have to work with code that's not up to your standards,
but come on.
Doing the armchair code reviewer without context is easy but my guess
would be that the old style library classes were written long before
the new style ones (perhaps even before the latter were introduced)
and/or written independently by different developers/groups, without
any plan to mix the two in the future. Also remember that up to 2.4
even the standard exception classes were old-style so it's not safe to
assume that you only have new style classes to worry about when the
very standard library includes lots of legacy code.

George
Dec 19 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
3176
by: Hai-Chu Hsu | last post by:
Hi, I want to restore my databases from the old SQL Server installation into a new installation of SQL Server. My new installation of SQL Server has different data path from the old installation. In addition, the data owners of some databases in the old SQL Server installation are not dbo. I have backups for all the databases in the old...
0
1163
by: Alienz | last post by:
Hey, I was reading the google groups for info on this but the explanation left out exactly how to do the query.. Lets say I have table1 with 2 combo boxes combo1 has a value list of: "new", "old", "documentation" - if I select "new", Id like combo2 to requery with the options "hardware", "software" - if I select "old", Id like combo2 to...
6
2504
by: Robert Schuldenfrei | last post by:
Dear NG, As expected, when I went to implement TIMESTAMP, I failed. With the help of Kevin Yu, I got the 2 code segments at the bottom working using a WHERE clause that checks all columns. SQL does not like my use of TIMESTAMP. First, notice that I have used "string" type data to hold the TIMESTAMP column in C#. Second, the Primary...
1
1969
by: Dean R. Henderson | last post by:
I setup FormsAuthentication on a couple websites where it has been working as expected for a long time. I used this code to setup the same type of authentication on a new website I am working on and the Cookie Name is not getting setup the same way. In my Web.config file, I use the following basic settings on both the old and new websites:...
2
983
by: S. Justin Gengo | last post by:
I just found out about and tried a free Devloper Toolbar from Microsoft that is an add on for IE. The things it does are fantastic. I only needed to install and play with it for a couple of minutes to realize that it will something I'l use from now on. I wanted to let everyone here know about it in case you hadn't heard of it before....
0
1500
by: Colin Fox | last post by:
I've got a trigger function that gets executed after an insert or an update. It basically just creates a copy of the row in a history table. However, there are cases when a row will be updated to be the same as it was - in other words, no change. In these cases, the history is useless and taking up space. So I want the procedure to start...
9
5234
by: John | last post by:
Tried this on microsoft.public.access.gettingstarted - no response - perhaps more appropriate here. I'm not a database user, simply helping someone get started with a new computer. The old computer (win98se) runs a database in MS Access 97 pro, with all the attendant permissions etc. I can work on the database without problems, once...
2
3126
by: crferguson | last post by:
I'm having a really odd issue. Recently my company has upgraded our data server. For a couple of months I'm having to host two versions of the same website on our webserver until the new data server is placed in production. Basically, on the same webserver, I have "Website_Old" which pulls its data from our old data server. I now also have...
26
2960
by: Nospam | last post by:
I am trying to open a link in a new template window : <a onclick="windowopen('example.html','example');return false;" href="http://www.example.com" target="_blank"example link</a> such that clicking on the example link will open it in a new window in an already saved template/site/ called example.html
0
7464
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7396
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
1
7413
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7751
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
5968
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
4943
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3440
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1012
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
700
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.