467,146 Members | 990 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,146 developers. It's quick & easy.

Tuple assignment and generators?

Just as a pedantic exercise to try and understand Python a
bit better, I decided to try to make a generator or class
that would allow me to unpack an arbitrary number of
calculatible values. In this case, just zeros (though I
just to prove whatever ends up working, having a counting
generator would be nice). The target syntax would be
something like
a,b,c = zeros()
q,r,s,t,u,v = zeros()
where "zeros()" returns an appropriately sized tuple/list of
zeros.

I've tried a bit of googling, but all my attempts have just
ended up pointing to pages that blithly describe tuple
assignment, not the details of what methods are called on an
object in the process.

My first thought was to get it to use a generator:

def zeros():
while 1: yield 0

However, I get back a "ValueError: too many values to
unpack" result.

As a second attempt, I tried a couple of attempts at classes
(I started with the following example class, only derived
from "object" rather than "list", but it didn't have any
better luck):
class zeros(list): .... def __getitem__(self,i):
.... return 0
.... z = zeros()
a,b,c = z

Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: need more than 0 values to unpack
It looks like I need to have a pre-defined length, but I'm
having trouble figuring out what sorts of things need to be
overridden. It seems like I sorta need a

def __len__(self):
return INFINITY

so it doesn't choke on it. However, how to dupe the
interpreter into really believing that the object has the
desired elements is escaping me. Alternatively if there was
a "doYouHaveThisManyElements" pseudo-function that was
called, I could lie and always return true.

Any hints on what I'm missing?

Thanks,

-tkc


May 4 '06 #1
  • viewed: 3035
Share:
43 Replies
Tim Chase wrote:
Just as a pedantic exercise to try and understand Python a
bit better, I decided to try to make a generator or class
that would allow me to unpack an arbitrary number of
calculatible values. In this case, just zeros (though I
just to prove whatever ends up working, having a counting
generator would be nice). The target syntax would be
something like


<snip/>

By using this:

http://aspn.activestate.com/ASPN/Coo.../Recipe/284742
I came up with a small decorator doing that:

import inspect, dis

def expecting():
"""Return how many values the caller is expecting"""
f = inspect.currentframe()
f = f.f_back.f_back
c = f.f_code
i = f.f_lasti
bytecode = c.co_code
instruction = ord(bytecode[i+3])
if instruction == dis.opmap['UNPACK_SEQUENCE']:
howmany = ord(bytecode[i+4])
return howmany
elif instruction == dis.opmap['POP_TOP']:
return 0
return 1

def variably_unpack(f):
def d(*args, **kwargs):
r = f(*args, **kwargs)
exp = expecting()
if exp < 2:
return exp
return (r.next() for i in xrange(exp))
return d

@variably_unpack
def test():
def gen():
i = 0
while True:
yield i
i += 1
return gen()

a, b, c = test()

print a,b,c

a, b, c, d = test()

print a,b,c, d

Diez
May 4 '06 #2

Tim Chase wrote:
Just as a pedantic exercise to try and understand Python a bit better, I
decided to try to make a generator or class that would allow me to
unpack an arbitrary number of calculatible values. In this case, just
zeros (though I just to prove whatever ends up working, having a
counting generator would be nice). The target syntax would be something
like
a,b,c = zeros()
q,r,s,t,u,v = zeros()
where "zeros()" returns an appropriately sized tuple/list of zeros.

I've tried a bit of googling, but all my attempts have just ended up
pointing to pages that blithly describe tuple assignment, not the
details of what methods are called on an object in the process.

My first thought was to get it to use a generator:

def zeros():
while 1: yield 0

However, I get back a "ValueError: too many values to unpack" result.

As a second attempt, I tried a couple of attempts at classes (I started
with the following example class, only derived from "object" rather than
"list", but it didn't have any better luck):
class zeros(list): ... def __getitem__(self,i):
... return 0
... z = zeros()
a,b,c = z

Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: need more than 0 values to unpack
It looks like I need to have a pre-defined length, but I'm having
trouble figuring out what sorts of things need to be overridden. It
seems like I sorta need a

def __len__(self):
return INFINITY

so it doesn't choke on it. However, how to dupe the interpreter into
really believing that the object has the desired elements is escaping
me. Alternatively if there was a "doYouHaveThisManyElements"
pseudo-function that was called, I could lie and always return true.

Any hints on what I'm missing?

Thanks,

-tkc

While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]

of if you like:

def zeros(num):
return num*[0]

a,b,c = zeros(3)
q,r,s,t,u,v = zeros(6)

I think the reason I don't every use anything like this is that
you don't need to initialize variables in Python to zero and I would
probably use a list instead of individual variables like q,r,s,t,u,v.

-Larry Bates

May 4 '06 #3
Ant
I don't think he was explicitly wanting to initialize things to zero,
but rather unpack an arbitrary sequence into a tuple (could perhaps be
the fibonnacci sequence for example).

How about:
def zeros(count): .... for i in range(count):
.... yield 0
.... a,b,c = zeros(3)
a 0 b 0 c

0

May 4 '06 #4
In article <0K******************************@comcast.com>,
Larry Bates <la*********@websafe.com> wrote:
While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]


This is (IMO) fairly idiomatic:

a = b = c = 0
q = r = s = t = u = v = 0

Just
May 4 '06 #5
Just wrote:
In article <0K******************************@comcast.com>,
Larry Bates <la*********@websafe.com> wrote:
While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]


This is (IMO) fairly idiomatic:

a = b = c = 0
q = r = s = t = u = v = 0

Just


You must be careful with this as they all point to
exactly the same object. Example:
q = r = s = t = u = v = 0
id(q) 3301924 id(r) 3301924 id(s) 3301924
Notice that all of them point to exactly the same object,
not 6 copies of zero which is "probably" what the poster
was thinking.

Most of the time when I see this, it is because people are
thinking of variables having values which is mostly a
carry-over from old Fortran/Cobol/Basic programming ideas.
In python variables are pointers to objects. Objects could
be values, but they are not placeholders where you store
stuff.

I read on this list (quite frequently) that people
think they are getting 6 separate variables each with
a zero stored in them. They are not. They are getting
six pointers that all point to an integer zero (IMHO it
would be a rather odd application for a programmer
to want this). Here is where multiple assignments
causes problems for beginners:
a=[]
b=c=a
a.append(6)
b

[6]

What?? 'b' should contain an empty list, right? Nope.
a, b, and c all point to the SAME list just like the
poster's q, r, s, t, u, v all point to the SAME zero.

What they meant to write was:

c=a[:] # Shallow copy of list
b=a[:]

My rule, don't do it unless you know exactly why you
want to do it. It will trip you up at some point and
be VERY hard to find.

-Larry Bates
May 4 '06 #6
> I don't think he was explicitly wanting to initialize
things to zero, but rather unpack an arbitrary sequence
into a tuple (could perhaps be the fibonnacci sequence
for example).
Ant is correct here...Fibonnaci, digits of pi, the constant
42, an incrementing counter, a series of squares, whatever.
That was my original intent in the matter. Zeros just
happened to be a nice sort of place to start my exercise.
How about:

def zeros(count):
... for i in range(count):
... yield 0


One of the things I was trying to avoid was having to know
(and maintain) the count of items initialized. In the
theoretical world of my example, one could do something like

def counter():
counter = 0
while 1:
yield counter
counter += 1

and then initialize several variables, such as
a,b,c,d,e,f,g = counter()
a,b,c,d,e,f,g

(0,1,2,3,4,5,6)

It's similar to C's auto-numbering of enum values...If I
want to add another entry to a C enum, I just put it there,
and let the language take care of matters. With most of the
provided solutions, I also have to increment the count each
time I add an item to the list.

Diez provided an elegant solution with a decorator
(employing an incredibly ugly sort of hack involving
sniffing the opcode stack behind the scenes) that does what
I was looking for.

I was hoping that there was just some __foo__ property I was
missing that would have been called in the process of tuple
unpacking that would allow for a more elegant solution such
as a generator (or generator method on some object) rather
than stooping to disassembling opcodes. :)

Ah well.

-tkc


May 4 '06 #7

Larry Bates wrote:
Just wrote:
In article <0K******************************@comcast.com>,
Larry Bates <la*********@websafe.com> wrote:
While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]
This is (IMO) fairly idiomatic:

a = b = c = 0
q = r = s = t = u = v = 0

Just


You must be careful with this as they all point to
exactly the same object. Example:
q = r = s = t = u = v = 0
id(q) 3301924 id(r) 3301924 id(s) 3301924
Notice that all of them point to exactly the same object,
not 6 copies of zero which is "probably" what the poster
was thinking.


Numbers are immutable. They're never copied. Zero, in particular, is
the same variable all throughout a Python interpreter.
Most of the time when I see this, it is because people are
thinking of variables having values which is mostly a
carry-over from old Fortran/Cobol/Basic programming ideas.
Most of the time when I see it, it's written by someone who's used
Python for quite some time. It's a standard Python idiom. You'll find
it all over the standard library. It's not a carry-over from
Fortran/Cobol/Basic at all.
In python variables are pointers to objects. Objects could
be values, but they are not placeholders where you store
stuff.
And all immutable objects are indistinguishable from values. Immutable
objects include ints, longs, strings, unicode objects, tuples,
frozensets, and perhaps some others that I'm forgetting.
I read on this list (quite frequently) that people
think they are getting 6 separate variables each with
a zero stored in them.
That's because they are. They're getting 6 different pointers
(bindings) to zero. If you change one, the others remain pointed at
(bound to) zero.
They are not. They are getting
six pointers that all point to an integer zero
Six *different* pointers. Six *different* bindings.
(IMHO it
would be a rather odd application for a programmer
to want this).
No, it wouldn't be. It's exactly how a person works with immutable
(value) objects.
Here is where multiple assignments
causes problems for beginners:
a=[]
b=c=a
a.append(6)
b
[6]


Yes, that does sometimes trouble beginners to programming. But so do
regular expressions. Programmers ought not to restrict themselves by
what beginners will have no difficulty learning.
What?? 'b' should contain an empty list, right? Nope.
a, b, and c all point to the SAME list just like the
poster's q, r, s, t, u, v all point to the SAME zero.
There is only one zero in Python! It can never change!
What they meant to write was:

c=a[:] # Shallow copy of list
b=a[:]

My rule, don't do it unless you know exactly why you
want to do it. It will trip you up at some point and
be VERY hard to find.


Your rule should only be followed by people who obstinately refuse
either to understand the way variable bindings work in Python, or by
people who refuse to know or care whether a given kind of object is
immutable. And that group of people doesn't seem to include any of the
good Python programmers I know.

Jeremy

May 5 '06 #8
> Zero, in particular, is the same variable all throughout a Python interpreter.

For the sake of accuracy let me note that I ought to have said, "is the
same *value* all throughout a Python interpreter."

Jeremy

May 5 '06 #9
je******@gmail.com writes:
There is only one zero in Python! It can never change!


+0.5 QOTW

--
\ "Madness is rare in individuals, but in groups, parties, |
`\ nations and ages it is the rule." -- Friedrich Nietzsche |
_o__) |
Ben Finney

May 5 '06 #10

Larry Bates wrote:
You must be careful with this as they all point to
exactly the same object. Example:
q = r = s = t = u = v = 0
id(q) 3301924 id(r) 3301924 id(s) 3301924
But:
q = 0
r = 0
s = 0
id(q) 134536636 id(r) 134536636 id(s)

134536636
[snip] My rule, don't do it unless you know exactly why you
want to do it. It will trip you up at some point and
be VERY hard to find.


It's ok to do it with constant objects, really.
Carl Banks

May 5 '06 #11
Carl Banks wrote:
q = 0
r = 0
s = 0
id(q) 134536636 id(r) 134536636 id(s) 134536636
It is okay with constant object, really.


No:
r=100001
s=100001
t=100001
id(r) 135620508 id(s) 135620532 id(t)

135104688

It worked with the number 0 because of an implementation accident,
in general Python can use different ids for constant objects that are
equals in the == sense.

Michele Simionato

May 5 '06 #12
Wow, so, to see if I understand correctly:
r = 0
s = 0
t = 100001
u = 100001
r == s True t == u True r is s True t is u False ... ?


what the...?
does anybody else get mighty uncomfortable about this?
s.

May 5 '06 #13
vdrab wrote:
what the...?
does anybody else get mighty uncomfortable about this?


No. Why should you ever care about whether two integers representing
values are the same object? Your tests should be with `==`, not `is`.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
More fodder for the new lost generation
-- Nik Kershaw
May 5 '06 #14
vdrab wrote:
Wow, so, to see if I understand correctly:
r = 0
s = 0
t = 100001
u = 100001
r == s True t == u True r is s True t is u False ... ?


what the...?
does anybody else get mighty uncomfortable about this?


#include <stdio.h>

int main(int argc, char **argv) {
int a = 1;
int b = 1;
printf("a == b: %i\n", a == b);
printf("&a == &b: %i\n", &a == &b);
return 0;
}

droggisch@ganesha:/tmp$ ./test
a == b: 1
&a == &b: 0
Feeling the same might uncomfortableness? Get used to it: object identity
and two objects being part of an equality-relation are two different
beasts. It can get even worse: I can define an object (in C++ as well as in
python) that is not even equal to itself. Not that I felt the need for that
so far....

So: don't use object identity where you want equality. In all languages.

Diez

May 5 '06 #15
Tim Chase wrote:
Just as a pedantic exercise to try and understand Python a bit better, I
decided to try to make a generator or class that would allow me to
unpack an arbitrary number of calculatible values. In this case, just
zeros (though I just to prove whatever ends up working, having a
counting generator would be nice). The target syntax would be something
like
>>> a,b,c = zeros()
>>> q,r,s,t,u,v = zeros()
where "zeros()" returns an appropriately sized tuple/list of zeros.

I've tried a bit of googling, but all my attempts have just ended up
pointing to pages that blithly describe tuple assignment, not the
details of what methods are called on an object in the process.

My first thought was to get it to use a generator:

def zeros():
while 1: yield 0

However, I get back a "ValueError: too many values to unpack" result.

As a second attempt, I tried a couple of attempts at classes (I started
with the following example class, only derived from "object" rather than
"list", but it didn't have any better luck):
>>> class zeros(list): ... def __getitem__(self,i):
... return 0
... >>> z = zeros()
>>> a,b,c = z

Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: need more than 0 values to unpack
It looks like I need to have a pre-defined length, but I'm having
trouble figuring out what sorts of things need to be overridden. It
seems like I sorta need a

def __len__(self):
return INFINITY

so it doesn't choke on it. However, how to dupe the interpreter into
really believing that the object has the desired elements is escaping
me. Alternatively if there was a "doYouHaveThisManyElements"
pseudo-function that was called, I could lie and always return true.

Any hints on what I'm missing?


Think the use case, you present, is (pythonically) ill as such. if you
want to fill literal variable's do it like: a=b=c=...=0. Otherwise
you'd handle variable lists zeros(7)

Yet there is a real freqent need for adding transparent extra/decorating
return values in cases, where you don't want to break the simple return
scheme (in maybe big existing code). For such use cases check out this
"RICHVALUE" approach with _named_ extra values - no puzzles about
non-matching tuple assignments:

http://aspn.activestate.com/ASPN/Coo.../Recipe/496676 :

....
def f():
return RICHVALUE(7, extra='hello')

ret=f()
print ret, ret+1, ret.extra
....
As I have this need very very often I'd like to see an even more natural
builtin method for enriching return values (both positional and by name)
in Python3000 - maybe by making calling and returning almost symetric!
Yet, this ideas maybe strange ? ...

def f_P3K(*args,**kwargs):
xreturn((7,8), 3, extra=5, *args, **kwargs) # first is main

v = f_P3K() # (7,8)
v,x = f_P3K() # (7,8),3
v,x,*pret,**kwret= f_P3K()
extra=kwret.get('extra',-1)
-robert
May 5 '06 #16
> No. Why should you ever care about whether two integers representing
values are the same object? Your tests should be with `==`, not `is`.


Given this though, what other such beauties are lurking in the
interpreter, under the name of 'implementation accidents'? One of the
things that drew me to python is the claimed consistency and
orthogonality of both language and implementation, not sacrificing
clarity for performance, minimizing ad-hoc design hacks and weird
gotcha's, etc...
In fact, I think my code contains things like "if len(arg) is 0:" and
so on, and I feel I should be able to do so given the way python treats
(claims to treat?) constant objects, even if I don't care whether the
values actually represent the same object.
s.

May 5 '06 #17
> beasts. It can get even worse: I can define an object (in C++ as well as in
python) that is not even equal to itself. Not that I felt the need for that
so far....


hehe... now you've picked my curiosity... how?

ps.
def __eq__(self, other): return False
does not count !

May 5 '06 #18
vdrab wrote:
beasts. It can get even worse: I can define an object (in C++ as well as
in python) that is not even equal to itself. Not that I felt the need for
that so far....


hehe... now you've picked my curiosity... how?

ps.
def __eq__(self, other): return False
does not count !


Sure it does! The rich comparison methods can very well be used to define
whatever semantics you want. Now I agree that

def __eq__(self, other):
if self is other:
return False
... # do something more sane
doesn't make much sense usually. Or maybe even never. Still, it illustrates
the point: object identity is not to be confused with two objects being
part of an equality relation.

Diez
May 5 '06 #19
"vdrab" wrote:
Given this though, what other such beauties are lurking in the
interpreter, under the name of 'implementation accidents'? One of the
things that drew me to python is the claimed consistency and
orthogonality of both language and implementation, not sacrificing
clarity for performance, minimizing ad-hoc design hacks and weird
gotcha's, etc...


so anything you don't understand, and cannot be bothered to look up in
the documentation, just has to be an inconsistent ad-hoc weird-gotcha
design ?

I think we can all safely *plonk* you know.

</F>

May 5 '06 #20
vdrab wrote:
No. Why should you ever care about whether two integers representing
values are the same object? Your tests should be with `==`, not `is`.


Given this though, what other such beauties are lurking in the
interpreter, under the name of 'implementation accidents'? One of the
things that drew me to python is the claimed consistency and
orthogonality of both language and implementation, not sacrificing
clarity for performance, minimizing ad-hoc design hacks and weird
gotcha's, etc...
In fact, I think my code contains things like "if len(arg) is 0:" and
so on, and I feel I should be able to do so given the way python treats
(claims to treat?) constant objects, even if I don't care whether the
values actually represent the same object.


Python doesn't claim that 0 is 0 == True. You are abusing the "is" operator.
The only (or at least 99%) occasions I use "is" are

if foo is None:
...

as None is guaranteed to be a singleton object.

The thing you observe as accident is that sometimes "0 is 0" is true just
because of an optimization of number objects allocation. Such things happen
in the "real" world - other examples are string-interning in e.g. the JVM
(and I bet they have a similar scheme to boxed number object allocation as
python has).

Diez
May 5 '06 #21
> so anything you don't understand, and cannot be bothered to look up in
the documentation, just has to be an inconsistent ad-hoc weird-gotcha
design ?


Does the documentation mention that "x is y" returns True when they are
both 0 but not when they are 100001 ? If so, I stand corrected. *plonk*
away ...
s.

May 5 '06 #22
Le 05-05-2006, Diez <de***@nospam.web.de> nous disait:
The thing you observe as accident is that sometimes "0 is 0" is true just
because of an optimization of number objects allocation. Such things happen
in the "real" world - other examples are string-interning in e.g. the JVM
(and I bet they have a similar scheme to boxed number object allocation as
python has).


String interning is available in Python too, by using the intern()
builtin function.

--
Alexandre Fayolle LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations
Dveloppement logiciel sur mesure: http://www.logilab.fr/services
Python et calcul scientifique: http://www.logilab.fr/science
May 5 '06 #23
> > Given this though, what other such beauties are lurking in the
interpreter, under the name of 'implementation accidents'? One of the
things that drew me to python is the claimed consistency and
orthogonality of both language and implementation, not sacrificing
clarity for performance, minimizing ad-hoc design hacks and weird
gotcha's, etc...


so anything you don't understand, and cannot be bothered to look up in
the documentation, just has to be an inconsistent ad-hoc weird-gotcha
design ?

I think we can all safely *plonk* you know.


I was just at a point when I thought I learned something but got
confused again after trying the following and unfortunately didn't
find an answer in the docs.
a = 10
b = 10
id(a) 134536516 id(b) 134536516

So the two memory addesses are the same, but
a = 10000
b = 10000
id(a) 134604216 id(b)

134604252

and they are not the same (I restarted the interpreter between the two
cases). So how is this now? Sorry if it's too trivial, but I simply
don't get it.
May 5 '06 #24
"vdrab" wrote:
Does the documentation mention that "x is y" returns True when they are
both 0 but not when they are 100001 ?


language reference, comparisions (is operator):

The operators is and is not test for object identity: x is y is true if and
only if x and y are the same object

language reference, objects:

"Even the importance of object identity is affected in some sense: for
immutable types, operations that compute new values may actually
return a reference to any existing object with the same type and value,
while for mutable objects this is not allowed. E.g., after "a = 1; b = 1",
a and b may or may not refer to the same object with the value one,
depending on the implementation, but after "c = []; d = []", c and d are
guaranteed to refer to two different, unique, newly created empty lists.

(note the use of "may or may not" and "depending on the implementation")

</F>

May 5 '06 #25
>
I was just at a point when I thought I learned something but got
confused again after trying the following and unfortunately didn't
find an answer in the docs.
a = 10
b = 10
id(a) 134536516 id(b) 134536516

So the two memory addesses are the same, but
a = 10000
b = 10000
id(a) 134604216 id(b)

134604252

and they are not the same (I restarted the interpreter between the two
cases). So how is this now? Sorry if it's too trivial, but I simply
don't get it.


It's an optimization scheme that will cache number objects up to a certain
value for optimized reuse. However this is impractical for larger numbers -
you only hold a table of lets say 1000 or so objects. Then the look up of
one of those objects is extremely fast, whereas the construction of
arbitrary numbers is somewhat more expensive.

And as "is" is the operator for testing if objects are identical and _not_
the operator for testing of equality (which is ==), the above can happen.
And is totally irrelevant from a practical POV (coding-wise that is - it
_is_ a relevant optimization).

Diez
May 5 '06 #26
> > I was just at a point when I thought I learned something but got
confused again after trying the following and unfortunately didn't
find an answer in the docs.
> a = 10
> b = 10
> id(a)

134536516
> id(b)

134536516

So the two memory addesses are the same, but
> a = 10000
> b = 10000
> id(a)

134604216
> id(b)

134604252

and they are not the same (I restarted the interpreter between the two
cases). So how is this now? Sorry if it's too trivial, but I simply
don't get it.


It's an optimization scheme that will cache number objects up to a certain
value for optimized reuse. However this is impractical for larger numbers-
you only hold a table of lets say 1000 or so objects. Then the look up of
one of those objects is extremely fast, whereas the construction of
arbitrary numbers is somewhat more expensive.

And as "is" is the operator for testing if objects are identical and _not_
the operator for testing of equality (which is ==), the above can happen.
And is totally irrelevant from a practical POV (coding-wise that is - it
_is_ a relevant optimization).


Thanks a lot! So after all I really learned something :)
May 5 '06 #27
>
language reference, objects:

"Even the importance of object identity is affected in some sense:
for
immutable types, operations that compute new values may actually
return a reference to any existing object with the same type and
value,
while for mutable objects this is not allowed. E.g., after "a = 1;
b = 1",
a and b may or may not refer to the same object with the value one,
depending on the implementation, but after "c = []; d = []", c and
d are
guaranteed to refer to two different, unique, newly created empty
lists.

(note the use of "may or may not" and "depending on the
implementation")

</F>

That, I knew. What I did not know, nor get from this explanation, is
that this behaviour "may" differ
not only within the same implementation, but with instances of the same
class or type (in this case, 'int'). Is this really a case of me being
too dumb or too lazy, or could it just be that this behaviour is not
all that consistent ?
v.
v.

May 5 '06 #28
vdrab wrote:
That, I knew. What I did not know, nor get from this explanation, is
that this behaviour "may" differ
not only within the same implementation, but with instances of the same
class or type (in this case, 'int').


"""
E.g., after "a = 1;
b = 1",
* * a and b may or may not refer to the same object with the value one,
* * depending on the implementation,
"""

Diez
May 5 '06 #29
Daniel Nogradi wrote:
a = 10
b = 10
id(a) 134536516 id(b) 134536516

So the two memory addesses are the same, but
a = 10000
b = 10000
id(a) 134604216 id(b)

134604252

and they are not the same (I restarted the interpreter between the two
cases). So how is this now? Sorry if it's too trivial, but I simply
don't get it.

If two immutable values are the same, then the interpreter has the right to
simply reuse the same value. Apart from the guarantee that it will do this
with None everything else is left open.

The current C-Python implementation will reuse small integers but not large
integers, it also reuses some strings. It reuses the empty tuple but not
(so far as I know) any other tuples. This could change at any time and
other Python implementations may do totally different things here.

Just because you saw it reusing a small value such as 10 doesn't mean that
there cannot be other small integers with the value 10 which aren't the
same as that one. Back before Python had a separate bool type, it used to
use two different integers for 0 (and two for 1), so you could (by an
accident of implementation) tell whether a value had been created by a
comparison operator. So far as I know, there is nothing stopping the author
of an extension written in C continuing to create their own versions of
small numbers today.
May 5 '06 #30
> """
E.g., after "a = 1;
b = 1",
a and b may or may not refer to the same object with the value one,
depending on the implementation,
"""


But when in a specific implementation this property _does_ hold for
ints having value 1, I expect the
same behaviour for ints with other values than 1.
I guess I'm kind of weird that way.

May 5 '06 #31
Tim Chase wrote:

I was hoping that there was just some __foo__ property I was
missing that would have been called in the process of tuple
unpacking that would allow for a more elegant solution such
as a generator (or generator method on some object) rather
than stooping to disassembling opcodes. :)


I suppose you wanted something like this...

a, b, c, d, e = xyz # first, let's consider a plain object

....to use the iterator "protocol" to populate the tuple elements, like
this:

_iter = xyz.__iter__()
a = _iter.next()
b = _iter.next()
c = _iter.next()
d = _iter.next()
e = _iter.next()

For such plain objects, it is possible to use such a mechanism. Here's
a "countdown" iterator:

class A:
def __init__(self, size):
self.size = size
def __iter__(self):
return B(self.size)

class B:
def __init__(self, size):
self.n = size
def next(self):
if self.n > 0:
self.n -= 1
return self.n
else:
raise StopIteration

xyz = A(5)
a, b, c, d, e = xyz

In fact, similar machinery can be used to acquire new values from a
generator:

def g(size):
while size > 0:
size = size - 1
yield size

a, b, c, d, e = g(5)

Note that if the iterator (specifically, the generator in the second
case) doesn't provide enough values, or provides too many, the tuple
unpacking operation will fail with the corresponding exception message.
Thus, generators which provide infinite or very long sequences will not
work unless you discard trailing values; you can support this either by
adding support for slicing to whatever is providing your sequences or,
if you're using generators anyway, by employing an additional
"limiting" generator:

def limit(it, size):
while size > 0:
yield it.next()
size -= 1

xyz = A(5)
a, b, c, d = limit(iter(xyz), 4)

The above generator may be all you need to solve your problem, and it
may be the case that such a generator exists somewhere in the standard
library.

Paul

May 5 '06 #32
vdrab wrote:
"""
E.g., after "a = 1;
b = 1",
a and b may or may not refer to the same object with the value one,
depending on the implementation,
"""


But when in a specific implementation this property _does_ hold for
ints having value 1, I expect the
same behaviour for ints with other values than 1.
I guess I'm kind of weird that way.


Are you telling us that you *had* read that doc,
and tripped because it says "depending on the implementation",
when it should say "at the choice of the implementation" ?

That's indeed a bit weird, imo.
May 5 '06 #33
> Are you telling us that you *had* read that doc,
and tripped because it says "depending on the implementation",
when it should say "at the choice of the implementation" ?


no.
let's see, where to start ... ?
let's say there's a certain property P, for the sake of this loooong
discussion, something
more or less like a class or type's property of "having immutable
values, such that any instance with value X has a single, unique
representation in memory and any two instantiations of objects with
that value X are in fact references to the same object".

Then, for example, python strings have property P whereas python lists
do not:
x = "test"
y = "test"
x is y True x = []
y = []
x is y False


Now, as it turns out, whether or not python integers have property P
_depends_on_their_value_.
For small values, they do. For large values they don't. Yes, I
understand about the interpreter optimization. I didn't know this, and
I find it neither evident nor consistent. I don't think the above post
explains this, regardless of how you read "implementation".

In fact, the whole string of replies after my initial question reminded
me of something I read not too long ago, but didn't quite understand at
the time.
source :
http://www.oreillynet.com/ruby/blog/...iantihype.html

'''
Pedantry: it's just how things work in the Python world. The status
quo is always correct by definition. If you don't like something, you
are incorrect. If you want to suggest a change, put in a PEP,
Python's equivalent of Java's equally glacial JSR process. The
Python FAQ goes to great lengths to rationalize a bunch of broken
language features. They're obviously broken if they're frequently
asked questions, but rather than 'fessing up and saying "we're
planning on fixing this", they rationalize that the rest of the world
just isn't thinking about the problem correctly. Every once in a
while some broken feature is actually fixed (e.g. lexical scoping), and
they say they changed it because people were "confused". Note that
Python is never to blame.
'''

taking this rant with the proverbial grain of salt, I did think it was
funny.

Anyway, thanks for all the attempts to show me.
I will get it in the end.
v.

May 5 '06 #34
vdrab wrote:
"""
E.g., after "a = 1;
b = 1",
a and b may or may not refer to the same object with the value one,
depending on the implementation,
"""
But when in a specific implementation this property _does_ hold for
ints having value 1, I expect the
same behaviour for ints with other values than 1.


That is an assumption you made. The above sentence is true for that
assumption, but also - and that is the key point here - for the current
implementation.

And to put it frankly: if you'd had spend only half the time it took you to
participate in this argument to think about how one could possibly
implement the behavior you'd thought of, you'd realize that its totally
unfeasible. Try stuffing 2^64 python long objects in your memory to make
that guarantee hold... And then 2^65
I guess I'm kind of weird that way.


Maybe.

Diez
May 5 '06 #35
vdrab <st***********@gmail.com> wrote:
let's say there's a certain property P, for the sake of this loooong
discussion, something
more or less like a class or type's property of "having immutable
values, such that any instance with value X has a single, unique
representation in memory and any two instantiations of objects with
that value X are in fact references to the same object".

Then, for example, python strings have property P whereas python lists
do not:
Er, no:
x = "test!"
y = "test!"
x == y True x is y

False

Strings only get a unique instance if they are valid identifiers.
Again, it's an optimisation issue. As with ints, it
_depends_on_their_value_. I find it neither evident nor consistent. I don't think the above post
explains this, regardless of how you read "implementation".


"Implementation dependent" => "Any behaviour you observe which is not
explicitly documented is not to be relied upon". Also, "Implementation
dependent" => "How this is implemented should be transparent and
irrelevant to the normal user". No, it's not particularly consistent.
Because it doesn't matter.

--
\S -- si***@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu become se bera eadward ofdun hlddre heafdes bce bump bump bump
May 5 '06 #36
oh wow... it gets better...
x = "test!"
y = "test!"
x is y False x = "test"
y = "test"
x is y True


.... I had no clue.
I guess the take-away lesson is to steer clear from any reliance on
object identity checks, if at all possible. Are there any other such
"optimizations" one should like to know about?
v.

May 5 '06 #37
On 5 May 2006 05:23:24 -0700 in comp.lang.python, "vdrab"
<st***********@gmail.com> wrote:
Are you telling us that you *had* read that doc,
and tripped because it says "depending on the implementation",
when it should say "at the choice of the implementation" ?
no.
let's see, where to start ... ?
let's say there's a certain property P, for the sake of this loooong
discussion, something
more or less like a class or type's property of "having immutable
values, such that any instance with value X has a single, unique
representation in memory and any two instantiations of objects with
that value X are in fact references to the same object".


IOW, property P is "(x == y) => (x is y)" (read "=>" as "implies").

Note that only immutable objects can have property P.

Then, for example, python strings have property P whereas python lists
do not:
x = "test"
y = "test"
x is yTrue x = []
y = []
x is yFalse

Note this last relationship is _guaranteed_. Lists are not immutable,
and therefore can not have property P.

Now, as it turns out, whether or not python integers have property P
_depends_on_their_value_.
From the zen, I believe this falls out from "practicality beats
purity."
For small values, they do. For large values they don't. Yes, I
Even that's not necessarily true. The implementation is free to
always create a new immutable object, even for small values.
understand about the interpreter optimization. I didn't know this, and
I find it neither evident nor consistent. I don't think the above post
explains this, regardless of how you read "implementation".
Think about implementation for a moment. Consider the statement

x = some_arbitrary_integer()

Do you really want the interpreter to go through all the existing
integer objects in the program to see if that particular value exists,
just to guarantee some some later statement

x is y

returns True if x == y?

OK, maybe we can change the "is" operator on immutable objects such
that x is y returns True if x == y. But then you can encounter a
situation where "x is y" but "id(x) != id(y)" Now what?

Perhaps the solution would be to disable the "is" operator and "id"
function for immutable objects. But then _they_ lose generality.
There doesn't seem to be a way to win.

So it all comes down to "who cares?" Immutable objects are immutable.
You can't change them. Object identity is a non-issue.

This is not the case for mutable objects. Consider

a = [1,2,3]
b = [1,2,3]
c = a

a==b
a==c
b==c
a is not b
b is not c
c is a

c.append(4)

In fact, the whole string of replies after my initial question reminded
me of something I read not too long ago, but didn't quite understand at
the time.
source :
http://www.oreillynet.com/ruby/blog/...iantihype.html
[...whinge elided...]
taking this rant with the proverbial grain of salt, I did think it was
funny.
Your original post in its entirety (ignoring the example) was "what
the...? does anybody else get mighty uncomfortable about this? "

The first response (paraphrased) was "No. Why should I? With
immutable objects, I care about ==, not is."

Your response seemed to want to cast doubt on the integrity of the
entire language: "Given this though, what other such beauties are
lurking in the interpreter, under the name of 'implementation
accidents'?"

Anyway, thanks for all the attempts to show me.
I will get it in the end.


I will ignore the double entendre, and simply hope I was of help, and
wish you luck. Regards,
-=Dave

--
Change is inevitable, progress is not.
May 5 '06 #38
"vdrab" wrote:
I guess the take-away lesson is to steer clear from any reliance on
object identity checks, if at all possible. Are there any other such
"optimizations" one should like to know about?


so in your little world, an optimization that speeds things up and saves
memory isn't really an optimization ?

good luck with your future career in programming.

*plonk*

May 5 '06 #39
> ... I had no clue.

We figured that....
I guess the take-away lesson is to steer clear from any reliance on
object identity checks, if at all possible.


You've been told that quite a few times before that "is" is not intended for
what you used it.

Some people actually listen to what others tell. Others seem to be driven by
the deep desire to make even the tiniest bit of getting-a-grasp a public
affair.

Diez
May 5 '06 #40
> You've been told that quite a few times before that "is" is not intended for
what you used it.
I got that. I was cleaning up some code that used "is" incorrectly
immediately after.
Some people actually listen to what others tell. Others seem to be driven by
the deep desire to make even the tiniest bit of getting-a-grasp a public
affair.


Not really. I always found python to be true to that -- admittedly
elusive -- principle of least surprise up to now ("special cases aren't
special enough to break the rules", maybe? I don't know. but then you
figured that, right?), and was thrown off quite a bit by the behaviour
described in one of the earlier posts, that is all. I wanted to ask
people's explanations about it and learnt a few things on the way
(thanks Dave). What did you get from all of this?

May 5 '06 #41

vdrab wrote:
I guess the take-away lesson is to steer clear from any reliance on
object identity checks, if at all possible.
BINGO!

Are there any other such
"optimizations" one should like to know about?


You don't have to know about them, as long as you use the operators
correctly.

== tests equality. is tests identity. Use is ONLY when you are
testing whether two things are the same object. Otherwise, use ==.
When deciding which operator to use, ask yourself this: would the
result still be true if they were different objects with the same
value? If yes, then use ==. 0 == 0 should be true even if the two
zeros are different objects.

Corrollary:

You should test for singleton objects with is. None, NotImplemented,
and Ellipsis are singleton objects; this is part of the language and
not an implementation detail. You can rely on it.
Carl Banks

May 5 '06 #42
"vdrab" <st***********@gmail.com> writes:
[...]
In fact, I think my code contains things like "if len(arg) is 0:" and
so on,
So you made a mistake. It's OK, you can forgive yourself, nobody will
notice <wink>

and I feel I should be able to do so given the way python treats
(claims to treat?) constant objects, even if I don't care whether the
values actually represent the same object.


(By "constant" I assume you mean immutable.)

I'm afraid it sounds like your assumptions about "the way python
treats (claims to treat?) constant objects" were "made up out of your
own head" as Tim Peters once put it :-) Python does not make such
guarantees about the identity of separately-constructed immutable
objects, and that's good, careful design, in my book (you might be
able to see why if you think about what "is" *means* and what it would
take to ensure it *always* remains true for e.g. x is 10E9, where x ==
10E9 but is assigned elsewhere, and at another time during program
execution).

OTOH, it's good practice in Python to use "is" to compare values with
the immutable singleton None ("if x is None"). No harm comes of that,
simply because there is only one None object.
John

May 5 '06 #43
vdrab wrote:
I guess the take-away lesson is to steer clear from any reliance on
object identity checks, if at all possible. Are there any other such
"optimizations" one should like to know about?


Object identity checks are just the thing/numero uno/ichiban
for checking object identity. A code snipped like

def broadcast (self, message):
"Broadcast a message to all the other game players."
for p in all_players:
if p is not self:
p.send (message)

does just what I want and expect it to.

Mel.
May 5 '06 #44

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by sosman | last post: by
50 posts views Thread by Will McGugan | last post: by
8 posts views Thread by Paul McGuire | last post: by
5 posts views Thread by Chris | last post: by
12 posts views Thread by Kay Schluehr | last post: by
4 posts views Thread by Peter Notebaert | last post: by
2 posts views Thread by Ministeyr | last post: by
11 posts views Thread by montyphyton@gmail.com | last post: by
reply views Thread by Hatem Nassrat | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.