Why keep identity-based equality comparison?

[spam.noam]

Hello,

Guido has decided, in python-dev, that in Py3K the id-based order
comparisons will be dropped. This means that, for example, "{} < []"
will raise a TypeError instead of the current behaviour, which is
returning a value which is, really, id({}) < id([]).

He also said that default equality comparison will continue to be
identity-based. This means that x == y will never raise an exception,
as is the situation is now. Here's his reason:

Let me construct a hypothetical example: suppose we represent a car
and its parts as objects. Let's say each wheel is an object. Each
wheel is unique and we don't have equivalency classes for them.
However, it would be useful to construct sets of wheels (e.g. the set
of wheels currently on my car that have never had a flat tire). Python
sets use hashing just like dicts. The original hash() and __eq__
implementation would work exactly right for this purpose, and it seems
silly to have to add it to every object type that could possibly be
used as a set member (especially since this means that if a third
party library creates objects for you that don't implement __hash__
you'd have a hard time of adding it).

Now, I don't think it should be so. My reason is basically "explicit is
better than implicit" - I think that the == operator should be reserved
for value-based comparison, and raise an exception if the two objects
can't be meaningfully compared by value. If you want to check if two
objects are the same, you can always do "x is y". If you want to create
a set of objects based on their identity (that is, two different
objects with the same value are considered different elements), you
have two options:
1. Create another set type, which is identity-based - it doesn't care
about the hash value of objects, it just collects references to
objects. Instead of using set(), you would be able to use, say,
idset(), and everything would work as wanted.
2. Write a class like this:

class Ref(object):
def __init__(self, obj):
self._obj = obj
def __call__(self):
return self._obj
def __eq__(self, other):
return isinstance(othe r, Ref) and self._obj is other._obj
def __hash__(self):
return id(self._obj) ^ 0xBEEF

and use it like this:

st = set()
st.add(Ref(whee l1))
st.add(Ref(whee l2))
if Ref(wheel1) in st:
....
Those solutions allow the one who writes the class to define a
value-based comparison operator, and allow the user of the class to
explicitly state if he wants value-based behaviour or identity-based
behaviour.

A few more examples of why this explicit behaviour is good:

* Things like "Decimal(3. 0) == 3.0" will make more sense (raise an
exception which explains that decimals should not be compared to
floats, instead of returning False).
* You won't be able to use objects as keys, expecting them to be
compared by value, and causing a bug when they don't. I recently wrote
a sort-of OCR program, which contains a mapping from a numarray array
of bits to a character (the array is the pixel-image of the char).
Everything seemed to work, but the program didn't recognize any
characters. I discovered that the reason was that arrays are hashed
according to their identity, which is a thing I had to guess. If
default == operator were not defined, I would simply get a TypeError
immediately.
* It is more forward compatible - when it is discovered that two types
can sensibly be compared, the comparison can be defined, without
changing an existing behaviour which doesn't raise an exception.

My question is, what reasons are left for leaving the current default
equality operator for Py3K, not counting backwards-compatibility?
(assume that you have idset and iddict, so explicitness' cost is only
two characters, in Guido's example)

Thanks,
Noam

[Christopher Subich]

Antoon Pardon wrote:

Op 2006-01-10, Peter Decker schreef <py******@gmail .com>:

I don't see the two comparisons as equivalent at all. If two things
are different, it does not follow that they can be ranked.

That a < b returns false doesn't imply that a and b can be ranked.
take sets. set([1,2]) and set([1,3)) can't be ranked but
set([1,2]) < set([1,3)) returns False just as set([1,2]) > set([1,3))
does.

Breaking my resolution already, but you're ignoring the fact that the
set type uses the '<' and '>' operators from a set-theoretic, not
number-theoretic point of view. Saying "set(1,3) is greater than
set(1,2)" is meaningless (and not false), because the mathematical basis
of the operator in this context is superset -- "set(1,3) is a superset
of set(1,2)" is well-defined and false.

Set uses '<' and '>' because the superset and subset symbols aren't on
the keyboard.

In languages that allow operator overloading, there are always some
well-defined cases where the operator is the simplest, clearest notation
yet the operator has a meaning very distinct from the arithmetical
operation. As another example, Pyparsing uses '<<' to "load" a Forward
declaration, for recursive grammars -- this obviously has nothing to do
with bit-shifting.

Of course, cases like these two are fairly textbook examples for the
argument that operator overloading is unclear; Python accepts the
occasional ambiguity and allows (indeed encourages, to a reasonable
degree) operator overloading for conciseness and expressiveness.
To reply to your other argument, Antoon: Maybe python should adopt both approaches and introduce a new family
of comparators. Then one family will always succeed and the other
family can throw an exception. [snip] I think it is usefull because when I am looking for 1 in a list,
I'm not necessarily happy when I find 1.0 or decimal("1").

I personally feel that the use cases for this "other" comparison (===?)
are very restricted. In fact, your example itself isn't even a use-case
for this operator, because integer/float/decimal have well-defined
equality comparisons already (that explicitly account for different
types) -- the implicit "not is implies !=, if __eq__ isn't defined"
behaviour isn't triggered.

The use-case for a "===" operator would seem to be restricted to when
program behaviour is determined soley by "a" not equalling "b." If a
"wrong" object is referenced by "b," then the program might do a Bad
Thing, because it expects "b" to be something else... except that the
error would be caught later anyway -- probably by calling "b.method() "
or somesuch.

In fact, even in more esoteric cases the behaviour of "==" as-is is
useful; in the itertools.izip_ longest discussion, this behaviour is
implicitly used in the sentinel-stopping method
(izip(chain(ite r,sent),chain(i ter,sent),...,s top=(sent,sent, sent,...)),
to badly mangle the syntax).

[Fuzzyman]

Peter Decker wrote:

On 10 Jan 2006 13:33:20 GMT, Antoon Pardon <ap*****@forel. vub.ac.be> wrote:

IMO if they aren't of the same type then the answer to:

a < b

is just as obviously False as

a == b

Yet how things are proposed now, the first will throw an exception
and the latter will return False.
I don't see the two comparisons as equivalent at all. If two things
are different, it does not follow that they can be ranked. If we have
two objects, such as a spark plug and a cam shaft, it is one thing to
say that the two are not the same object; it is quite another to say
that one is 'greater than' or 'less than' the other.

I agree.

If a and b are of incomparable types, then a != b is True but a < b is
meaningless.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

--

# p.d.

[Mike Meyer]

Antoon Pardon <ap*****@forel. vub.ac.be> writes:

There is no way in python now to throw an exception when you
think comparing your object to some very different object
is just meaningless and using such an object in a container
that can be searched via the "in" operator. I claim that comparing for equality is *never* meaningless. Either two
objects are equal, or they aren't. It may be that they are of
different types - like the enum example above - in which case they
will never compare equal.
Note that this is different from an ordering. It's possible to have a
pair of objects - maybe even of the same type - that can't be ordered
in anyway. In this case, raising an exception when you try that
comarison makes sense.

IMO you have the choice between taking the mathematical route or
the practical route.

The behavior proposed for Py3k *is* the practical route. It gives a
reasonable behavior, and one that leads to simple implemenations for
container operations.
Now you can take the practical option and decide that programmaticall y
it make no sense to compare a specific couple of values and throw an
exception in this case, but it doesn't matter much which test you are
conducting at that point.
Can you provide a case where having a test for equality throw an
exception is actually useful?

BTW, the case you're arguing for is *different* from the case the OP
proposed. By my reading, he wanted equality testing to throw an
exception for two objects unless a comparison was explicitly coded. So
that even a == a could cause an exception.
Maybe python should adopt both approaches and introduce a new family
of comparators. Then one family will always succeed and the other
family can throw an exception.
Comparators - including equality comparators - can already throw
exceptions. The enum case proved that.

Also, every container type now has this split between identity and
equality has to be dealt with for *every* container class. If you want
identity comparisons on objects, you have to store them in an idlist
for the in operator and index methods to work properly.
I also think your basic idea is wrong. The expression "x == y" is
intuitively False if x and y aren't comparable.
But there are certainly circumstances that I would prefer 1 == (1,2)
to throw an exception instead of simply turning up False.

So what are they?
Again - give us real use cases.

I would say some more thinking is needed in this area. Now we can
have weird circumstances where A == B and B == C but A != C.

Nothing wierd about that at all. Anyone who's dealt with floats at all
should be used to it.

With floats that is entirely a problem of precision. When you are
working with discrete types such circumstances remain weird.

Floats *are* a discrete type. The equality *relationship* is what's
fuzzy. There are lots of non-transitive relationships around. I don't
find them wierd at all.

I think such cases can be troublesome too for containers and the
"in" operator.

I don't. Can you provide an example of where it is?

Well not with the "in" operator but with the index method of lists
which seems related enough.

The index method of list is already a bit fuzzy.
If the "in" operator returns true one can use index to find out
an element in the container that compares equal. Now normally
it wouldn't make a difference whether you would make further
comparisons against the original object or against the object
in the list. But in this case it can make a difference and
it isn't obvious what one should do.

That's because in this case there's no on "right" answer. What you
should do will depend on what you are trying to accomplish. That's the
normal state of affairs when programming.

IMO some more thinking about this is needed before deciding this
would be a good idea or not.

Actually, what's need are examples of usages where breaking equality
into two (or more - most LISPs have three different definitions of
equality) different relations is usefull.

I think it is usefull because when I am looking for 1 in a list,
I'm not necessarily happy when I find 1.0 or decimal("1").

That's an argument for a more *precise* equality operator. That's
certainly worth considering, but has nothing to do with whether or not
it makes sense for an equality comparison to throw an exception.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[spam.noam]

> Can you provide a case where having a test for equality throw an > exception is actually useful? Yes. It will be useful because: 1. The bug of not finding a key in a dict because it was implicitly hashed by identity and not by value, would not have happened. 2. You wouldn't get the weird 3.0 != Decimal("3.0") - you'll get an exception which explains that these types aren't comparable. 3. If, in some time, you will decide that float and Decimal could be compared, you will be able to implement that without being concerned about backwards compatibility issues. >>>> But there are certainly circumstances that I would prefer 1 == (1,2) >>>> to throw an exception instead of simply turning up False. >>> So what are they? > > Again - give us real use cases. You may catch bugs earlier - say you have a multidimensiona l array, and you forgot one index. Having comparison raise an exception because type comparison is meaningless, instead of returning False silently, will help you catch your problem earlier. Noam

[spam.noam]

It seems to me that both Mike's and Fuzzyman's objections were that
sometimes you want the current behaviour, of saying that two objects
are equal if they are: 1. the same object or 2. have the same value
(when it's meaningful). In both cases this can be accomplished pretty
easily: You can do it with a try..except block, and you can write the
try...except block inside the __contains__ method. (It's really pretty
simple: try: return a == b except TypeError: return a is b )
Also, Mike said that you'll need an idlist object too - and I think
he's right and that there's nothing wrong with it. Note that while you
can easily define the current == behaviour using the proposed
behaviour, you can't define the proposed behaviour using the current
behaviour. Also note that using the current behaviour, you can't easily
treat objects that do define a meaningful value comparison, by
identity. Also note that in the cases that you do want identity-based
behaviour, defining it explicitly can result in a more efficient
program: explicit identity-based dict doesn't have to call any __hash__
and __eq__ protocols - it can compare the pointers themselves. The same
if you want to locate a specific object in a list - use the proposed
idlist and save yourself O(n) value-based comparisons, which might be
heavy. Noam

[Mike Meyer]

sp*******@gmail .com writes:

It seems to me that both Mike's and Fuzzyman's objections were that
sometimes you want the current behaviour, of saying that two objects
are equal if they are: 1. the same object or 2. have the same value
(when it's meaningful). In both cases this can be accomplished pretty
easily: You can do it with a try..except block, and you can write the
try...except block inside the __contains__ method. (It's really pretty
simple: try: return a == b except TypeError: return a is b )
This isn't "easy". It's an ugly hack you have to use everytime you
want to iterate through a heterogenous set doing equality tests.

You're replacing "false" with an "emphatheti c false", that *all*
containers to change for the worse to deal with it.
Also, Mike said that you'll need an idlist object too - and I think
he's right and that there's nothing wrong with it.
Except that we now need four versions of internal data structures,
instead of two: list, tuple, idlist, idtuple; set, idset, frozenset,
frozenidset, and so on. What's wrong with this is that it's ugly.
Note that while you
can easily define the current == behaviour using the proposed
behaviour, you can't define the proposed behaviour using the current
behaviour.
Yes you can, and it's even easy. All you have to do is use custom
classes that raise an exception if they don't
Also note that using the current behaviour, you can't easily
treat objects that do define a meaningful value comparison, by
identity.

Yes you can. Just use the "is" operator.

Note that this behavior also has the *highly* pecular behavior that a
doesn't necessarily equal a by default.

I will point out why your example usages aren't really usefull if
you'll repeat your post with newlines.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Steven Bethard]

Mike Meyer wrote:

sp*******@gmail .com writes:

My question is, what reasons are left for leaving the current default
equality operator for Py3K, not counting backwards-compatibility?
(assume that you have idset and iddict, so explicitness' cost is only
two characters, in Guido's example)

Yes. Searching for items in heterogenous containers. With your change
in place, the "in" operator becomes pretty much worthless on
containers of heterogenous objects. Ditto for container methods that
do searches for "equal" members. Whenever you compare two objects that
don't have the same type, you'll get an exception and terminate the
search. If the object your searching for would have been found
"later", you lose - you'll get the wrong answer.

Not to advocate one way or the other, but how often do you use
heterogeneous containers? I couldn't find any in my (admittedly small)
codebase. Could you post some examples of what kind of problems lend
themselves to being solved by heterogeneous containers?

Thanks,

STeVe

[Mike Meyer]

Steven Bethard <st************ @gmail.com> writes:

Not to advocate one way or the other, but how often do you use
heterogeneous containers?

Pretty much everything I do has heterogenous containers of some sort
or another. SQL queries made to DP API compliant modules return
homogenous lists of heterogenous containers. The cgi module turns the
request string into a dictionary-like container of objects with values
of different types. Higher-level web interfaces go even further in
this direction.

The last thing I did that was both more than a script and didn't use
either a database or a web front end was (IIRC) a media player for
multiple media types. It revolved around lists of things to play, and
the "things" in question could be any "playable" object - video or
audio files, track on a CD, or a DVD, or even a playlist.

Come to think of it, recursive data structures of this type - a
container that contains a heterogenous list of things, possibly
including instances of the container type itself - are pretty
common. Pretty much every GUI package has something like it. All
processors of SGML-based markup languages I've ever dealt with
included something like it. MIME-encoded email does this. Page layout
programs do this. Block-structured programming languages do this. And
probably lots of others.

These are things that in a language that used classes for carrying
(and enforcing) type, all of these cases would be heterogenous lists
of objects that were subtypes of some type, so maybe they would
"really" be heterogenous. But then you're stuck with the interesting
question: What's the type relationship between two objects a and b
that allows them to be compared.

This question is still interesting with duck typing. If anything, it's
even more interesting.

We have *at least two* different proposals for a different typing
system in hand. For one, the answer is obvious. The OP proposed that
equality only be allowed when the types explicitly allow it, instead
of defaulting to typing by identity. In that one the answer is that
all the types on the list have to boilerplate so they play
together. Phrasing it that way makes it seem contrary to the spirit of
Python, as not needing boilerplat is an oft-touted strength of Python.

The other proposal - if I have it right - would not change the
behavior of equality comparisons between objects of the same class,
but would make comparisons between objects of different classes raise
an exception instead of returning false by default. Since I didn't
raise this proposal, I'll leave it up to someone else to explain under
what conditions two objects that are both instances of some class are
"the same type" or not.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Antoon Pardon]

Op 2006-01-10, Christopher Subich schreef <cs************ ****@spam.subic h.block.com>:

Antoon Pardon wrote:

Op 2006-01-10, Peter Decker schreef <py******@gmail .com>:

I don't see the two comparisons as equivalent at all. If two things
are different, it does not follow that they can be ranked.

That a < b returns false doesn't imply that a and b can be ranked.
take sets. set([1,2]) and set([1,3)) can't be ranked but
set([1,2]) < set([1,3)) returns False just as set([1,2]) > set([1,3))
does.

Breaking my resolution already, but you're ignoring the fact that the
set type uses the '<' and '>' operators from a set-theoretic, not
number-theoretic point of view.

That is irrelevant. the '<' and '>' symbols are usable to denote any
mathematical order and are often enough used for even other order
relations. The only reason that other symbols like the subset symbol
are used is to avoid confusion about which order you are talking
because numbers and sets are used together often enough.

But the superset relationship is mathematically just as much an
order relation as is the greater than relationship.
Saying "set(1,3) is greater than
set(1,2)" is meaningless (and not false), because the mathematical basis
of the operator in this context is superset -- "set(1,3) is a superset
of set(1,2)" is well-defined and false.

No it is not meaningless. The superset relationship is just as much
an order relationship and thus can mathematically make use of the
'<' and '>' symbol just as any mathematical order relation can.

--
Antoon Pardon

[Antoon Pardon]

Op 2006-01-10, Mike Meyer schreef <mw*@mired.org> :

Antoon Pardon <ap*****@forel. vub.ac.be> writes:

There is no way in python now to throw an exception when you
think comparing your object to some very different object
is just meaningless and using such an object in a container
that can be searched via the "in" operator.
I claim that comparing for equality is *never* meaningless. Either two
objects are equal, or they aren't. It may be that they are of
different types - like the enum example above - in which case they
will never compare equal.
Note that this is different from an ordering. It's possible to have a
pair of objects - maybe even of the same type - that can't be ordered
in anyway. In this case, raising an exception when you try that
comarison makes sense. IMO you have the choice between taking the mathematical route or
the practical route.

The behavior proposed for Py3k *is* the practical route. It gives a
reasonable behavior, and one that leads to simple implemenations for
container operations.

Then I have to ask, practical for who, user of python or the
implementor, because I don't find it practical that a language
says at the same times that two objects can use a comparision
and can't.

Now you can take the practical option and decide that programmaticall y
it make no sense to compare a specific couple of values and throw an
exception in this case, but it doesn't matter much which test you are
conducting at that point.

Can you provide a case where having a test for equality throw an
exception is actually useful?

I'm not going to bother with that. Sure uses cases are interesting
but if you always wait for a use case before implementing something,
whatever the other arguments are, you will disappoint the future
people with a use case because they can't do what they want yet.

I haven't seen a case where testing for unequality throwing an
exception would be actually usefull, yet that is considered,
why do I have to provide a use case.
BTW, the case you're arguing for is *different* from the case the OP
proposed. By my reading, he wanted equality testing to throw an
exception for two objects unless a comparison was explicitly coded. So
that even a == a could cause an exception.

Why not? If a is of a type where == is a meaningless operation then
a == a is meaningless too.

Maybe python should adopt both approaches and introduce a new family
of comparators. Then one family will always succeed and the other
family can throw an exception.

Comparators - including equality comparators - can already throw
exceptions. The enum case proved that.

Your point? Your remark says nothing for or against python having
two families of comparators, one that is defined as never throwing
an exception and one defined as exception throwable.

> Also, every container type now has this split between identity and
> equality has to be dealt with for *every* container class. If you want
> identity comparisons on objects, you have to store them in an idlist
> for the in operator and index methods to work properly.
> I also think your basic idea is wrong. The expression "x == y" is
> intuitively False if x and y aren't comparable.
But there are certainly circumstances that I would prefer 1 == (1,2)
to throw an exception instead of simply turning up False.
So what are they?
Again - give us real use cases.

I didn't see a real use case for 1 < (1,2) throwing an exception either.
The only argument seems to be that the current behaviour confuses
beginners. But I don't see that as such a strong argument because
a number of other things confuse beginners too and are not up
for a change. I also think that 1 == (1,2) returning False but
1 < (1,2) throwing an excpetion masy not be that less confusing
as the current behaviour.

I don't care that much what it will be, but I would prefer a consistent
approach for all comparators. No either all throw an exception when
the two operands are of differnt type or None does (or two families)

I would say some more thinking is needed in this area. Now we can
have weird circumstances where A == B and B == C but A != C.
Nothing wierd about that at all. Anyone who's dealt with floats at all
should be used to it.

With floats that is entirely a problem of precision. When you are
working with discrete types such circumstances remain weird.

Floats *are* a discrete type. The equality *relationship* is what's
fuzzy. There are lots of non-transitive relationships around. I don't
find them wierd at all.

That there are a lot of non-transitive relationships and that there
is nothing weird about them, says nothing about one specific
relationship, == which normaly is considered to be transitive
and turns out not to be.

Beside I think the == comparison is transitive on the floats. It
is just that if you do your calculations that the imprecision in
the numbers can give you a result that give false where you expect
true when comparing for equality, but that is because you got
a different float as a result than you expected, not because
the float == relationship is not transitive.

--
Antoon Pardon