473,324 Members | 2,246 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

subclassing extension type and assignment to __class__

Hello,

I am currently writing python bindings to an existing C++ library, and I
just encountered a problem that I hope has been solved by more
experienced python programmers:

A C++ class (let's call it CClass) is binded using classical Python
extension API to _PClass, which is accesible through python without any
problem. The problem is that I want this class to be extended/extensible
in python, and expose the python-extended version (PClass) to library
users (_PClass should never be used directly nor be retruned by any
function).
The aim is to leave only performance critical methods in C++ so that the
binding work is minimal, and develop the other methods in python so that
they are easier to maintain/extend.

We thus have something like this

class PClass(_PClass):
def overide_method(self,...):
...
def new_method(self,...):
...

and I can define
a=PClass()
and use my new or overiden method
a.overide_method() a.new_method() as intended...

So far, so good, trouble begin when I have a method from another class
PClass2 derived from _PClass2 which bind the C++ class CClass2, that
should return objects of type PClass:

Let call this method troublesome_method:

b=_PClass2()
c=b.troublesome_method()
type(c) gives _PClass.

now I want to define a python class PClass2 for extending methods of
_PClass2 like I have done for _PClass, in particular I want that PClass2
troublesome_method return objects of type PClass instead of _PClass...

To this end I try something like this

class PClass2(_PClass2):
...
def troubelsome_method(self):
base=_PClass2.troublesome_method(self)
base.__class__=PClass

and I have python complaining about TypeError: __class__ assignment:
only for heap types...

It seems thus that my approach is not possible, but problem is that I
can not see another approach that would not involve far more efforts
that this one (I have the impression I have to forget about making my
PClass derived from _PClass, but instead embed a _PClass instance in
PClass, far from ideal cause I would have to reimplement all method, not
only those that I want to change :( )
This is particularly frustrating cause I also have the impression that
want I want to do was at one time possible in python, let say in
2002-2003, when __class__ was already assignable but before assignment
was forbidden for non-heap types

Any hint on this problem?

Thanks,

Greg.
Jul 18 '05 #1
6 3038
gregory lielens <gr*************@fft.be> writes:
Hello,

I am currently writing python bindings to an existing C++ library, and
I just encountered a problem that I hope has been solved by more
experienced python programmers:

A C++ class (let's call it CClass) is binded using classical Python
extension API to _PClass, which is accesible through python without
any problem. The problem is that I want this class to be
extended/extensible in python, and expose the python-extended version
(PClass) to library users (_PClass should never be used directly nor
be retruned by any function).
The aim is to leave only performance critical methods in C++ so that
the binding work is minimal, and develop the other methods in python
so that they are easier to maintain/extend.

We thus have something like this

class PClass(_PClass):
def overide_method(self,...):
...
def new_method(self,...):
...

and I can define
a=PClass()
and use my new or overiden method
a.overide_method() a.new_method() as intended...

So far, so good, trouble begin when I have a method from another class
PClass2 derived from _PClass2 which bind the C++ class CClass2, that
should return objects of type PClass:

Let call this method troublesome_method:

b=_PClass2()
c=b.troublesome_method()
type(c) gives _PClass.

now I want to define a python class PClass2 for extending methods of
_PClass2 like I have done for _PClass, in particular I want that
PClass2 troublesome_method return objects of type PClass instead of
_PClass...

To this end I try something like this

class PClass2(_PClass2):
...
def troubelsome_method(self):
base=_PClass2.troublesome_method(self)
base.__class__=PClass

and I have python complaining about TypeError: __class__ assignment:
only for heap types...

It seems thus that my approach is not possible, but problem is that I
can not see another approach that would not involve far more efforts
that this one (I have the impression I have to forget about making my
PClass derived from _PClass, but instead embed a _PClass instance in
PClass, far from ideal cause I would have to reimplement all method,
not only those that I want to change :( )
This is particularly frustrating cause I also have the impression that
want I want to do was at one time possible in python, let say in
2002-2003, when __class__ was already assignable but before assignment
was forbidden for non-heap types

Any hint on this problem?

I have just run into the same thing with the builtin list type. The trick
was not to declare my subclass directly from list, but rather a subclass
of list. So try this:

class PClassBase(_PClass):
pass

class PClass(PClassBase):
...

class PClass2(PClassBase):
...

Why? I cannot say other than I have noted that Python new-style classes
and extension types are not quite the same thing. That is, a new-style
class is a particular kind of extension type.

Lenard Lindstrom
<le***@telus.net>
Jul 18 '05 #2
Thanks for your answer, it means I am not the only one having this kind
of problem...
I have just run into the same thing with the builtin list type. The trick
was not to declare my subclass directly from list, but rather a subclass
of list. So try this:

class PClassBase(_PClass):
pass

class PClass(PClassBase):
...

class PClass2(PClassBase):
...
I think here you mean PClass2(PClassBase2), with a
PClassBase2(_PClass2): In my example the 2 class do not inherit fro the
same base class... Why? I cannot say other than I have noted that Python new-style classes
and extension types are not quite the same thing. That is, a new-style
class is a particular kind of extension type.


I see how it can help me making PClassBase a PClass (or any type derived
from PClassBase), but what I fail to see is how this will allow me to
change a _PClass into one of it's derived types: this is necessary for
updating methods of _PClass2 that return object of type _PClass (and on
this I do not have much control, both _PClass and _PClass2 and all of
their methods are implemented in C++). To updates theses methods for the
pure python derived class PClass2, I have to make them return the python
class PClass instead of the original _PClass, which means I still have
to transform a _PClass object into its derived type PClassBase or
PClass, and at this time things go pear-shaped :-(....or is there
something I miss?

Greg.
Jul 18 '05 #3
gregory lielens <gr*************@fft.be> writes:
Thanks for your answer, it means I am not the only one having this
kind of problem...
I have just run into the same thing with the builtin list type. The trick
was not to declare my subclass directly from list, but rather a subclass
of list. So try this:
class PClassBase(_PClass):
pass
class PClass(PClassBase):
...
class PClass2(PClassBase):
...

I think here you mean PClass2(PClassBase2), with a
PClassBase2(_PClass2): In my example the 2 class do not inherit fro
the same base class...
Why? I cannot say other than I have noted that Python new-style classes
and extension types are not quite the same thing. That is, a new-style
class is a particular kind of extension type.


I see how it can help me making PClassBase a PClass (or any type
derived from PClassBase), but what I fail to see is how this will
allow me to change a _PClass into one of it's derived types: this is
necessary for updating methods of _PClass2 that return object of type
_PClass (and on this I do not have much control, both _PClass and
_PClass2 and all of their methods are implemented in C++). To updates
theses methods for the pure python derived class PClass2, I have to
make them return the python class PClass instead of the original
_PClass, which means I still have to transform a _PClass object into
its derived type PClassBase or PClass, and at this time things go
pear-shaped :-(....or is there something I miss?

Sorry for the confusion. This looks like a situation best handled with
embedding rather than inheritance. The PClass and PClass2 wrapper classes
can share whatever relationship _PClass and _PClass2 share.

class PClass:
def __init__(inst=None):
if inst is None:
inst = _PClass()
self._inst = inst
def override_method(self, ...):
self._inst.override_method(...)
def new_method(self, ...):
...

class PClass2:
self._inst = _PClass2()
def troublesome_method(self):
base = PClass(self._inst.troublesome_method())
...
return base

Alternatively you could provide a way to pass PClass back to the extension module
for _PClass2.troublesome_method to use instead of _PClass when creating a return
object. But the embedding approach seems more appropriate when _PClass is not to
be used directly.

Lenard Lindstrom
<le***@telus.net>
Jul 18 '05 #4
Sorry for the confusion. This looks like a situation best handled with
embedding rather than inheritance. The PClass and PClass2 wrapper classes
can share whatever relationship _PClass and _PClass2 share.

class PClass:
def __init__(inst=None):
if inst is None:
inst = _PClass()
self._inst = inst
def override_method(self, ...):
self._inst.override_method(...)
def new_method(self, ...):
...

class PClass2:
self._inst = _PClass2()
def troublesome_method(self):
base = PClass(self._inst.troublesome_method())
...
return base
Yes I was affraid this would be the conclusion: embedding instead of
inheritance...But this means that not only the method that need to be
modified need to be rewritten in python, all the other ones also just to
delegate to the embedded instance...
This is not too practical, even if such writing can be more or less
automatized...
For my purpose it seems that there was a big step backward between
python 2.2 and 2.3, and I am surprised this has been mostly
unnoticed...This problem should be common for many people trying to
write mixed C/C++ / python classes. I found some reference on this in
the list archive, a few people beeing anoyed by it, but the proposition
to make the test about assignment to class less strict was not retained
it seems...I plan to experiment a little with the python C
implementation to see what happen if I bypass this test...
Alternatively you could provide a way to pass PClass back to the extension module
for _PClass2.troublesome_method to use instead of _PClass when creating a return
object. But the embedding approach seems more appropriate when _PClass is not to
be used directly.


Yes but again this is not so practical, the purpose of the _PClass /
PClass implementation was to make the bindings as light, and as easy to
implement / extend as possible, and both the embedding and the creation
of PClass within _PClass2 methods defeat this purpose...

I think allowing assignment to class for native types should be done if
possible, for example if the new class is a derived type (possibly using
__slots__)... I saw a post on this by Guido, but the reason why it was
not possible is not clear to me and the thread died without further
explanations...Maybe my experiments will show me what happen, or someone
more knowledgeable than me can explain why this is not possible?

Greg.
Jul 18 '05 #5
gregory lielens <gr*************@fft.be> writes:

(...)
Yes I was affraid this would be the conclusion: embedding instead of
inheritance...But this means that not only the method that need to be
modified need to be rewritten in python, all the other ones also just
to delegate to the embedded instance...
This is not too practical, even if such writing can be more or less
automatized...


Unless I'm misunderstanding, couldn't one of __getattr__ or
__getattribute__ make mapping other methods you don't override very
simple and practical, not to mention fully automated with 2-3 lines of
code? In particular, __getattr__ would seem good for your use since
it is only called for attributes that couldn't already be located.

I've had code that wrapped underlying objects in a number of cases, and
always found that to be a pretty robust mechanism.

-- David
Jul 18 '05 #6
Unless I'm misunderstanding, couldn't one of __getattr__ or
__getattribute__ make mapping other methods you don't override very
simple and practical, not to mention fully automated with 2-3 lines of
code? In particular, __getattr__ would seem good for your use since
it is only called for attributes that couldn't already be located.

I've had code that wrapped underlying objects in a number of cases, and
always found that to be a pretty robust mechanism.


Thanks for mentioning this, after more research I came up with something
usable, using delegating technique from python cookbook:
#wrapper

def wrap_class(base):
class wrapper:
__base__=base
def __init__(self,*args,**kwargs):
if len(args)==1 and type(args[0])==self.__base__ and kwargs=={}:
self._base = args[0]
else:
self._base=self.__class__.__base__.__new__(self.__ class__.__base__,*args,**kwargs)
def __getattr__(self,s):
return self._base.__getattribute__(s)
return wrapper

#wrap class
complex2=wrap_class(complex)
#extend wrapper class
def supaprint(self):
print "printed with supaprint(tm):"
print self
complex2.supaprint=supaprint

#define wrapper class from base class
c1=1+1j
c2=complex2(c1)
#test old functionalities
print c1==c2
print "c1=",c1
print "c2=",c1
#test extension
c2.supaprint()
c1.supaprint()

So this basically fit the bill, it even delegates special methods it
seems, although I am not sure why...It is like writting heritage ourself,
in a way :-)

Remaning problem is that if we use the class generating
wrapper function (simple), we loose the classic class definition syntax
and rely on explicitely adding methods.
The alternative is to include the wrapper
machinery in every "home-derived" class, but you are right, it is not as bad as I
though :-)

The biggest point I am not sure now is performance: Isn't a big penalty
associated to this embedding, compared to derivation? Python performance
is not so critical in our application, but I would be uncomfortable having
a factor 10 penalty in methd calling associated to this approach...For now, this will be used for
testing new implementations/extensions, that will be added to the C++
written binding afterwards.

I'd like to thanks the list for the hints, I will post the results of my
experimatations relaxing the assigment to __class__ test if they are
interesting.

Greg.

Jul 18 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
by: Chris Cioffi | last post by:
Hello all! I'm trying to subclass int such that once it reaches a certain value, it flips back to the starting count. This is for use as an X12 control number. Subclassing int and getting the...
3
by: Mizrandir | last post by:
I'd like to subclass numarray's array. I'd like to add some methods and override others like __init__. I don't know how to do this and haven't found help searching the manual or the web, can...
4
by: harold fellermann | last post by:
Hi all, I have a problem pickling an extension class. As written in the Extending/Embedding Manual, I provided a function __reduce__ that returns the appropreate tuple. This seams to work fine,...
166
by: Graham | last post by:
This has to do with class variables and instances variables. Given the following: <code> class _class: var = 0 #rest of the class
5
by: Barry Kelly | last post by:
I'm running this version of Python: Python 2.4.3 (#1, May 18 2006, 07:40:45) on cygwin I read in the documentation that these two expressions are interchangeable: ...
4
by: KraftDiner | last post by:
I have the following code... import array len32 = array.array('L') len16 = array.array('H') len32.append(0) len16.append(0) y = len32
3
by: dgdev | last post by:
I would like to pickle an extension type (written in pyrex). I have it working thus far by defining three methods: class C: # for pickling __getstate__(self): ... # make 'state_obj' return...
1
by: gregory.lielens | last post by:
Hello, We are currently writing python bindings to an existing C++ library, and we encountered a problem that some of you may have solved (or that has found unsolvable :( ): A C++ class...
16
by: manatlan | last post by:
I've got an instance of a class, ex : b=gtk.Button() I'd like to add methods and attributes to my instance "b". I know it's possible by hacking "b" with setattr() methods. But i'd like to do...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.