Is there a consensus on how to check a polymorphic instance?

[Mike Meng]

hi all,
I'm a newbie Python programmer with a C++ brain inside. I have a
lightweight framework in which I design a base class and expect user to
extend. In other part of the framework, I heavily use the instance of
this base class (or its children class). How can I ensure the instance
IS-A base class instance, since Python is a fully dynamic typing
language?

I searched and found several different ways to do this:

1. In `Text Processing in Python', David Mertz suggests using `hasattr'
to check abilities instead of type checking.

2. In ActiveState's Python Cookbook site, Alex Martelli suggested a
safe and accurate `look before you leap' technique on
http://aspn.activestate.com/ASPN/Coo...n/Recipe/52291

3. In this group, some others suggest using isinstance() builtin
function to check instance.

Now my questions:
1. Is my design speaks native Pythonish? Such design is native in C++
and Java OO style, but I'm not sure about this in Python.

2. As I state in title, is there a consensus on how to check a
polymorphic instance? I find all the three ways above is a little bit
tedious compared with C++ and Java, in which compile-time type checking
ensures the interface protocol, any better ideas in Python?
Any comments will be appreciated.

[Dan Perl]

As you say yourself, many people in this group will probably recommend
solution #3. I would agree with them.

You should keep in mind that Alex's recipe (solution #2) was published in
2001 (before version 2.2 which introduced big changes in terms of OO) so I'm
not sure he would recommend that solution anymore. I'll leave it to him to
give his final judgement.

I haven't read Mertz's book and I don't know in what context he recommends
solution #1. Personally I see it useful in situations where you want to
check whether the class has a method like next() for example, so you want to
see whether the object is iterable. But if you implement a base class with
an attribute x1 it may be risky to just check objects for having that
attribute. The object may have an attribute with that name but its role may
be completely different than what you expect (e.g., your class may use x1
for a list, but your object is an instance of a class that uses x1 for a
string). Same thing with methods.

Go with isinstance(). That's what it's for.

Dan

"Mike Meng" <me******@gmail .com> wrote in message
news:11******** **************@ z14g2000cwz.goo glegroups.com.. .

hi all,
I'm a newbie Python programmer with a C++ brain inside. I have a
lightweight framework in which I design a base class and expect user to
extend. In other part of the framework, I heavily use the instance of
this base class (or its children class). How can I ensure the instance
IS-A base class instance, since Python is a fully dynamic typing
language?

I searched and found several different ways to do this:

1. In `Text Processing in Python', David Mertz suggests using `hasattr'
to check abilities instead of type checking.

2. In ActiveState's Python Cookbook site, Alex Martelli suggested a
safe and accurate `look before you leap' technique on
http://aspn.activestate.com/ASPN/Coo...n/Recipe/52291

3. In this group, some others suggest using isinstance() builtin
function to check instance.

Now my questions:
1. Is my design speaks native Pythonish? Such design is native in C++
and Java OO style, but I'm not sure about this in Python.

2. As I state in title, is there a consensus on how to check a
polymorphic instance? I find all the three ways above is a little bit
tedious compared with C++ and Java, in which compile-time type checking
ensures the interface protocol, any better ideas in Python?
Any comments will be appreciated.

[Steven Bethard]

Mike Meng wrote:

I'm a newbie Python programmer with a C++ brain inside. I have a
lightweight framework in which I design a base class and expect user to
extend. In other part of the framework, I heavily use the instance of
this base class (or its children class). How can I ensure the instance
IS-A base class instance, since Python is a fully dynamic typing
language?

The short answer is that if you want to ensure an instance is-a subclass
of the base class, you should use isinstance:

class HasF(object): .... def f(self):
.... raise NotImplementedE rror
.... class SubHasF(HasF): .... def f(self):
.... return self.__class__
.... class NonsubHasF(obje ct): .... def f(self):
.... return self.__class__
.... isinstance(SubH asF(), HasF) True isinstance(Nons ubHasF(), HasF) False

However it's often not necessary to go this route. Consider:
class HasNoF(object): .... pass
.... def use_c(c): .... try:
.... f = c.f
.... except AttributeError:
.... raise TypeError('argu ment to use_c must have an f method')
.... return f()
.... use_c(SubHasF() ) <class '__main__.SubHa sF'> use_c(NonsubHas F()) <class '__main__.Nonsu bHasF'> use_c(HasNoF())

Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<interacti ve input>", line 5, in use_c
TypeError: argument to use_c must have an f method

Is it really necessary that the classes passed to your equivalent of
'use_c' are actually subclasses of your equivalent to 'HasF'? Or is it
just necessary that they support the appropriate methods? If it's the
second, I would just test for the appropriate methods and catch the
AttributeErrors (and possibly the TypeError that's thrown if 'f' isn't
callable).

If you can give a little more detail on your particular example, I (and
others) can probably give you more helpful suggestions...

Steve

[Mike Meng]

Thank you Dan,
Solution #1 is concluded from 'Text Processing in Python', section
1.1.3, `Pythonic Polymorphisim', around there, here is an example he
provides:

def toDOM(xml_src = None):
from xml.dom import minidom
if hasattr(xml_src , 'documentElemen t'):
return xml_src # it's already a DOM object
elif hasattr(xml_src , 'read'):
# it is something that knows how to read data
return minidom.parseSt ring(xml_src.re ad())
elif type(xml_src) in (StringType, UnicodeType):
# it is a filename of an XML document
xml = open(xml_src).r ead()
return minidom.parseSt ring(xml)
else:
raise ValueError, "Must be initialized with " + \
"filename, file-like object, or DOM object"
What's your opinion?

[Steven Bethard]

Mike Meng wrote:

1. In `Text Processing in Python', David Mertz suggests using `hasattr'
to check abilities instead of type checking.

hasattr can give you problems if the attribute you're checking for is
calculated on the fly since hasattr basically does the same thing
getattr does:

class C(object): .... def time():
.... def fget(self):
.... time.sleep(5)
.... return time.time()
.... return dict(fget=fget)
.... time = property(**time ())
.... c = C()
hasattr(c, 'time') True c.time

1101191279.3729 999

If you run the code above, you'll get the 5 second pause both during the
hasattr call and the c.time access. If any object you use calculates
attribute values on the fly, calling hasattr before directly accessing
the attribute will cause the value to be calculated twice.

Steve

[Mike Meng]

hi Steve,
Your example is helpful. Thank you.

Here is my problem. My project needs to parse some user input
string according to specific rules. The rules can be defined with a
string, a CSV file, an XML file, and other forms. Followed the standard
OO style, I designed a dictionary-like base class to abstract those
different sources and provides other parts a uniform interface to query
those rules. I put some hard code into the base class and expect user
to inherit from it.

Any good suggestion?

Mike

[Steven Bethard]

Mike Meng wrote:

Here is my problem. My project needs to parse some user input
string according to specific rules. The rules can be defined with a
string, a CSV file, an XML file, and other forms. Followed the standard
OO style, I designed a dictionary-like base class to abstract those
different sources and provides other parts a uniform interface to query
those rules. I put some hard code into the base class and expect user
to inherit from it.

The big question, I guess, is what do you want to happen if a user did
not inherit from your base class, but still provides all the appropriate
functionality? In a dynamically-typed language like Python the answer
is usually that the user's class should still be considered valid,
though of course there are exceptions to this rule.

For example, say I write the following function:

def process_rules(r ules): .... for rule in rules:
.... print rule.name
....

Do I really care if the 'rules' object inherits from, say, the Rules
class? Probably not. I care that it is iterable, and that the items
that it contains have a name attribute. Here's a few different ways I
could write an object that conforms to this interface:
class Rule(object): .... def __init__(self, name):
.... self.name = name
.... process_rules([Rule(s) for s in 'abc']) a
b
c class Rules(object): .... def __init__(self, names):
.... self.names = names
.... def __iter__(self):
.... for name in self.names:
.... yield Rule(name)
.... process_rules(R ules('abc')) a
b
c class NastyRules(obje ct): .... def __init__(self, names):
.... self.names = names
.... self.index = -1
.... def __iter__(self):
.... return iter([self]*len(self.names ))
.... def __getattr__(sel f, attr):
.... if attr == 'name':
.... self.index += 1
.... return self.names[self.index]
.... raise AttributeError
.... process_rules(N astyRules('abc' )) a
b
c

Of course, if you write your code like NastyRules, you should be drug
out into the street and shot ;) but the point is that there are a
variety of ways that a class could support the given interface and still
do everything you've asked it to. If you'd like to give your users the
option to implement the interface in whatever way seems most
appropriate, you shouldn't be testing isinstance:
isinstance([Rule(s) for s in 'abc'], list) True isinstance(Rule s('abc'), list) False isinstance(Nast yRules('abc'), list) False

In the example above, if you test isinstance, you disallow your user
from writing either of the other two implementations .

Note that you can still give useful error messages if you catch the
appropriate exceptions:
def process_rules(r ules): .... try:
.... rules = iter(rules)
.... except TypeError:
.... raise TypeError('proc ess_rules argument must '
.... 'support iterator protocol')
.... for rule in rules:
.... try:
.... print rule.name
.... except AttributeError:
.... raise TypeError('proc ess_rules argument must '
.... 'produce objects with name attribute')
.... process_rules([Rule(s) for s in 'abc']) a
b
c process_rules(R ules('abc')) a
b
c process_rules(N astyRules('abc' )) a
b
c process_rules(1 ) Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<interacti ve input>", line 5, in process_rules
TypeError: process_rules argument must support iterator protocol process_rules(r ange(10))

Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<interacti ve input>", line 11, in process_rules
TypeError: process_rules argument must produce objects with name attribute

So again, the real question here is: If a user creates a class that
appropriately implements the interface, but doesn't inherit from your
base class, should their code fail?

Steve

[Dan Perl]

I was referring to a different kind of context in my posting. Not exactly
the code implementation but rather how is the code being used. Are you the
only one using this code and can you be confident that the function toDOM
will be called only with objects that it is designed to handle? Or are you
implementing this function as part of a module that will be used by
hundreds/thousands of users over the next ten years?

In other words, how much do you trust the test that if hasattr(xml_src ,
'documentElemen t') is true then "it's already a DOM object"? What if a user
who downloads your code off the internet 2 years from now calls toDOM with
an object that he implemented and that has an attribute 'documentElemen t'
but that object behaves in a way totally different from what you expect for
a DOM object? Even you may make such a mistake if you use this code again
only one year from now, after forgetting how you implemented it.

OTOH, you may not have much of a choice but use the hasattr test in this
case if documentElement is an attribute that is commonly used by all the DOM
parser implementations and you want to be able to switch parsers without
changing your code. I am using the assumption that not all the parsers
inherit from a common base class or else you could use isinstance with that
class. I am not familiar enough with XML parsers so I don't know if
documentElement is such an attribute.

The other hasattr test in the example, for a 'read' attribute, is such a
case. 'read' is a commonly used attribute name for methods that read data.
It doesn't matter whether it is a file, or a stream, or a queue, or
whatever, as long as it is readable and read() returns a string. Mind you,
this is still an incomplete test because it does not guarantee the string to
be XML. But it is probably the best choice compared to testing for all the
possible sources of XML text that you might use.

I see actually two issues here. One is of checking inputs and that comes
down to safety vs. flexibility. You make a test very strict or very
flexible depending on many criteria. You have to consider your users,
future changes, etc.

The other issue is one of design. How do you design a set of classes that
need to follow a common interface but implement that interface in different
ways? You can create a base class and subclass all the other classes from
this one. But thanks to python's dynamic typing, if the base class is
abstract and all its attributes are overridden in the subclasses, you may as
well just forget about inheritance and just implement all the subclasses to
follow an interface protocol. That's what iterators and generators are.

So, getting back to our topic, if you use inheritance to implement your
classes or you have only one class, then use isinstance. If you have a set
of classes that do not have a common base class and instead just implement
an interface protocol, then use hasattr. Your question was about
polymorphic instances. That implies inheritance, but you may want to
consider also protocol implementation and that is what other posters are
suggesting.

I think that an interesting question here is whether inheritance or protocol
implementation is the preferred way in python. I'm not sure what the answer
is to that question. One advantage I see in inheritance is to allow
stronger checks like isinstance vs. hasattr. I come from a very
conservative background in software development and strong checks are deeply
rooted in my mind. But it looks to me that python is not about that or else
it wouldn't have dynamic typing.

Dan

"Mike Meng" <me******@gmail .com> wrote in message
news:11******** **************@ f14g2000cwb.goo glegroups.com.. .

Thank you Dan,
Solution #1 is concluded from 'Text Processing in Python', section
1.1.3, `Pythonic Polymorphisim', around there, here is an example he
provides:

def toDOM(xml_src = None):
from xml.dom import minidom
if hasattr(xml_src , 'documentElemen t'):
return xml_src # it's already a DOM object
elif hasattr(xml_src , 'read'):
# it is something that knows how to read data
return minidom.parseSt ring(xml_src.re ad())
elif type(xml_src) in (StringType, UnicodeType):
# it is a filename of an XML document
xml = open(xml_src).r ead()
return minidom.parseSt ring(xml)
else:
raise ValueError, "Must be initialized with " + \
"filename, file-like object, or DOM object"
What's your opinion?

[Dan Perl]

"Steven Bethard" <st************ @gmail.com> wrote in message
news:lkCod.8865 3$5K2.13406@att bi_s03...

Mike Meng wrote:

Here is my problem. My project needs to parse some user input
string according to specific rules. The rules can be defined with a
string, a CSV file, an XML file, and other forms. Followed the standard
OO style, I designed a dictionary-like base class to abstract those
different sources and provides other parts a uniform interface to query
those rules. I put some hard code into the base class and expect user
to inherit from it.

The big question, I guess, is what do you want to happen if a user did not
inherit from your base class, but still provides all the appropriate
functionality? In a dynamically-typed language like Python the answer is
usually that the user's class should still be considered valid, though of
course there are exceptions to this rule.

I have a question here, as we are discussing now protocol interfaces vs.
inheritance. Is using a class that implements a protocol without inheriting
from a base class still "polymorphi sm"? There are probably many definitions
for polymorphism and probably all those definitions can be interpreted in
such a way that they accept also protocols. But what I would like to hear
is what is the general opinion of people who use python. I am biased
because I come from a C++ and Java background and I am still used to a
certain practical meaning for "polymorphi sm". But it's beginning to dawn on
me that it is only a bias and polymorphism does apply also to python
protocol interfaces. Is that generally accepted?

Dan

[Steven Bethard]

Dan Perl wrote:

I have a question here, as we are discussing now protocol interfaces vs.
inheritance. Is using a class that implements a protocol without inheriting
from a base class still "polymorphi sm"?

Some good definitions:

http://en.wikipedia.org/wiki/Polymor...ter_science%29)

There are probably two types of polymorphism relevant to the current
discussion: parametric polymorphism and subtyping polymorphism. In
parametric polymorphism, a function is written so as to work with
objects of different types. Subtyping polymorphism is a subtype of
parametric polymorphism that restricts the polymorphism to only objects
that inherit from a specified base class.

In its most basic use, Python is inherently parametrically polymorphic
-- all functions are written so as to work with different types.
Consider a few simple functions:

def f(x): .... return 'f(%s)' % x
.... def g(x): .... return 2**x
....

Both of these functions work perfectly well with a wide range of objects:
f(1), f(2.0), f('abc'), f([1,2,3]) ('f(1)', 'f(2.0)', 'f(abc)', 'f([1, 2, 3])') g(4), g(5.0), g(decimal.Decim al(6)) (16, 32.0, Decimal("64"))

I didn't have to do anything special to make them work this way. Any
object that supports __str__ can be passed to f, e.g.:

class S(object): .... def __str__(self):
.... return 'str'
.... f(S()) 'f(str)'

and any object that supports __rpow__ (or __coerce__, or some other way
to be raised to a power) can be passed to g, e.g.:
class P(object): .... def __rpow__(self, other):
.... return other**other
.... g(P()) 4

So both functions f and g are parametrically polymorphic. If you wanted
to artifically restrict f or g to support only subtype polymorphism, you
could do this with isinstance. In most cases, I would probably advise
against this -- if you're using isinstance to restrict your functions to
only subtype polymorphism, you're reducing the usefulness of your code.

This is not to say that there are never any times when you want to use
isinstance. Here's a good example from my code for a Bunch class[1]:

class Bunch(object):
...
def update(self, *args, **kwds):
if len(args) == 1:
other, = args
if isinstance(othe r, self.__class__) :
other = other.__dict__
try:
self.__dict__.u pdate(other)
except TypeError:
raise TypeError('cann ot update Bunch with %s' %
type(other).__n ame__)
elif len(args) != 0:
raise TypeError('expe cted 1 argument, got %i' % len(args))
self.__dict__.u pdate(kwds)

In the code here, we use isinstance to check on whether or not to use
the argument's __dict__. Now I could have written the isinstance
section as:

try:
other = other.__dict__
except AttributeError:
pass

and then the function would 'work' with any object that had a __dict__
attribute. But since other classes are not guaranteed to use __dict__
in the same way that Bunch does (e.g. they're not guaranteed to have all
attributes in their __dict__s), this would give some potentially
confusing results:
class C(object): .... def x():
.... def get(self):
.... return 5
.... return dict(fget=get)
.... x = property(**x())
.... c = C()
b = Bunch()
b.update(c)
b.x

Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
AttributeError: 'Bunch' object has no attribute 'x'

The point here is that I'm using isinstance here because I *know* there
are objects out there that, unlike Bunch objects, may have 'attributes'
not stored in their __dict__, and I *know* they might get passed to
Bunch.update. Since I can't appropriately support their use, I want the
user to be notified of this (with the TypeError).

Of course, I'm still making heavy use of the more general parametric
polymorphism because even if the object doesn't inherit from Bunch, I
still pass it to dict.update, which works with any type that supports
the mapping protocol (and a few other types besides). So if, in the
future dict.update gains support for some new type, my code doesn't
artifically restrict the type of parameter that can be passed to my
function.

Hope this was helpful (or at least vaguely intelligible) ;)

STeve
[1] Thanks to Peter Otten for the implementation suggestion