Can't get around "IndexError: list index out of range"

[erikwickstrom]

Hi all,

I'm sorry about the newbie question, but I've been searching all
afternoon and can't find the answer!

I'm trying to get this bit of code to work without triggering the
IndexError.

import shutil, os, sys

if sys.argv[1] != None:
ver = sys.argv[1]
else:
ver = '2.14'

Of course, whenever I run it, I get list index out of range.

I'm coming from the php world where I can do:
if $_GET['var'] != Null {
$ver = $_GET['var'];
} else {
$ver = '2.14';
}

Can anyone tell me how to make this work in python?

Thanks!
Erik

[Leif K-Brooks]

er***********@gmail.com wrote:

I'm trying to get this bit of code to work without triggering the
IndexError.

import shutil, os, sys

if sys.argv[1] != None:
ver = sys.argv[1]
else:
ver = '2.14'

Catch it:

try:
ver = sys.argv[1]
except IndexError:
ver = '2.14'

[Steven Bethard]

er***********@gmail.com wrote:

I'm trying to get this bit of code to work without triggering the
IndexError.

import shutil, os, sys

if sys.argv[1] != None:
ver = sys.argv[1]
else:
ver = '2.14'

Something like::

if len(sys.argv) 1:
ver = sys.argv[1]
else:
ver = '2.14'

It looks like you're trying to do argument parsing, though and for
anything more complicated than the above, you might prefer to use a real
argument parsing library like argparse_::

import argparse

parser = argparse.ArgumentParser(description='print the version')
parser.add_argument('ver', nargs='?', default='2.14',
help='the version number to print')

args = parser.parse_args()
print args.ver

Then from the command line::

$ script.py
2.14

$ script.py 3.14159265359
3.14159265359

$ script.py -h
usage: script.py [-h] [ver]

print the version

positional arguments:
ver the version number to print

optional arguments:
-h, --help show this help message and exit

And then you can feel good about yourself for also documenting your
command-line interface. ;-)

... _argparse: http://argparse.python-hosting.com/

STeVe

[Terry Reedy]

"Leif K-Brooks" <eu*****@ecritters.bizwrote in message
news:45***********************@news.sover.net...

er***********@gmail.com wrote:

>I'm trying to get this bit of code to work without triggering the
IndexError.

import shutil, os, sys

if sys.argv[1] != None:
ver = sys.argv[1]
else:
ver = '2.14'

Catch it:

try:
ver = sys.argv[1]
except IndexError:
ver = '2.14'

Or slice it:

if sys.argv[1:2] != []:
<etc>

tjr

[John Machin]

Terry Reedy wrote bloated code:

if sys.argv[1:2] != []:

if sys.argv[1:2]:

:-)

[Steve Holden]

er***********@gmail.com wrote:

Hi all,

I'm sorry about the newbie question, but I've been searching all
afternoon and can't find the answer!

I'm trying to get this bit of code to work without triggering the
IndexError.

import shutil, os, sys

if sys.argv[1] != None:
ver = sys.argv[1]
else:
ver = '2.14'

Of course, whenever I run it, I get list index out of range.

I'm coming from the php world where I can do:
if $_GET['var'] != Null {
$ver = $_GET['var'];
} else {
$ver = '2.14';
}

Can anyone tell me how to make this work in python?

Well all the advice you've had so far seems good, but of course the
simplest way is just to test the length of the sequence before you try
to address its second element:

if len(sys.argv) 1:
ver = sys.argv[1]
else:
ver = "2.14"

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[erikcw]

I ended up using len(sys.argv) 1 for this particular problem. But I
think slicing is closer to the tool I was looking for.

I found a.has_key(k) or "k in a" for dictionaries - but haven't found
anything similar for lists. Does it exist?

I guess my example from php would technically be a dictionary in python
and not a list, it would nice to be able to quickly tell if a list key
existed or not.

Thanks!
Erik
Steve Holden wrote:

er***********@gmail.com wrote:

Hi all,

I'm sorry about the newbie question, but I've been searching all
afternoon and can't find the answer!

I'm trying to get this bit of code to work without triggering the
IndexError.

import shutil, os, sys

if sys.argv[1] != None:
ver = sys.argv[1]
else:
ver = '2.14'

Of course, whenever I run it, I get list index out of range.

I'm coming from the php world where I can do:
if $_GET['var'] != Null {
$ver = $_GET['var'];
} else {
$ver = '2.14';
}

Can anyone tell me how to make this work in python?

Well all the advice you've had so far seems good, but of course the
simplest way is just to test the length of the sequence before you try
to address its second element:

if len(sys.argv) 1:
ver = sys.argv[1]
else:
ver = "2.14"

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[Gabriel Genellina]

At Friday 6/10/2006 20:57, erikcw wrote:

>I ended up using len(sys.argv) 1 for this particular problem. But I
think slicing is closer to the tool I was looking for.

I found a.has_key(k) or "k in a" for dictionaries - but haven't found
anything similar for lists. Does it exist?

if 2 in [1,2,3]: print "Use the same (in) operator"
elif 'E' in ('E','r','i','k'): print "Works for any sequence"
elif 'o' in 'hello': print "Even strings"

Gabriel Genellina
Softlab SRL

__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

[hanumizzle]

On 6 Oct 2006 16:57:23 -0700, erikcw <er***********@gmail.comwrote:

I ended up using len(sys.argv) 1 for this particular problem. But I
think slicing is closer to the tool I was looking for.

I found a.has_key(k) or "k in a" for dictionaries - but haven't found
anything similar for lists. Does it exist?

As Gabriel Genellina observed, lists also may use the 'in' syntax.
Actually, any object that supports the method __contains__ can do
this, which means you may furnish your own in user objects.

What hasn't been addressed is efficiency. This is often a secondary
concern in a VHLL such as Python, but you may wish to know that using
the 'in' test on a large list repeatedly will slow down a program. You
may wish to use a set object instead, according to the application.

I guess my example from php would technically be a dictionary in python
and not a list, it would nice to be able to quickly tell if a list key
existed or not.

Yes, PHP arrays are associative (dicts). But lists (in the Pythonic
sense, aka 'vectors') do not have keys as such, only indices.

-- Theerasak

[MonkeeSage]

On Oct 6, 8:23 pm, Gabriel Genellina <[e-mail]:

if 2 in [1,2,3]: print "Use the same (in) operator"
elif 'E' in ('E','r','i','k'): print "Works for any sequence"
elif 'o' in 'hello': print "Even strings"

This isn't really analogous is it? For "somedict.has_key(k)" or "k in
somedict", you don't need to know the value of somedict[k] ahead of
time. So you can avoid KeyError by asking if the key exists first
before trying to get the value. Wouldn't that be more like this for
lists (and other sequences):

def has_index(seq, index):
try:
seq[index]
return True
except IndexError:
return False

I've often wondered why there is no built-in method like that for
sequence objects. And also why not the equivalent of dict.get for other
sequences:

def get(seq, index, default=None):
if has_index(seq, index):
return seq[index]
else:
return default

Seems useful...

Regards,
Jordan

[Gabriel Genellina]

At Saturday 7/10/2006 02:15, MonkeeSage wrote:

>On Oct 6, 8:23 pm, Gabriel Genellina <[e-mail]:

if 2 in [1,2,3]: print "Use the same (in) operator"
elif 'E' in ('E','r','i','k'): print "Works for any sequence"
elif 'o' in 'hello': print "Even strings"

This isn't really analogous is it? For "somedict.has_key(k)" or "k in
somedict", you don't need to know the value of somedict[k] ahead of
time. So you can avoid KeyError by asking if the key exists first
before trying to get the value.

The meaning comes from the most common usage. For a list, you want to
know if an object is contained in the list (not if an index is in
range!). For a dictionary, you usually want to know if it maps
anything to a given key (not if any key maps to that value!). These
are the most common operations, and that's why they have the simple
sintax "a in b". [BTW, usage of operator "in" as "key in dict" is
rather new to Python; has_key() were the only way to test for key
existence some time ago].

>Wouldn't that be more like this for
lists (and other sequences):

def has_index(seq, index):
try:
seq[index]
return True
except IndexError:
return False

I've often wondered why there is no built-in method like that for
sequence objects.

Because it's not needed at all: valid sequence indexes are *exactly*
range(len(seq)). This is the basic point of being a sequence: when
indexes are not contiguous, in fact you have a mapping, not a sequence.

>And also why not the equivalent of dict.get for other
sequences:

def get(seq, index, default=None):
if has_index(seq, index):
return seq[index]
else:
return default

Seems useful...

Sometimes, maybe... But since you can write it efficientely in a few
lines when needed, I don't see the need to put it into the core language.
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

[MonkeeSage]

On Oct 7, 3:27 am, Gabriel Genellina <[e-mail]:

The meaning comes from the most common usage.

I wasn't suggesting that the "in" keyword have a different sematic for
sequence types. I was just saying that regarding the question whether
there is anything similar to "dict.has_key / k in dict" for lists, the
"in" keyword is not semantically analogous.

Because it's not needed at all: valid sequence indexes are *exactly*
range(len(seq)). This is the basic point of being a sequence: when
indexes are not contiguous, in fact you have a mapping, not a sequence.

True. But valid dictionary keys are exactly d.keys(). The has_key
method is just sugar.

Sometimes, maybe... But since you can write it efficientely in a few
lines when needed, I don't see the need to put it into the core language.

Couldn't the same be said for dictionaries? I could write for
sequences:

if index < len(seq) and seq[index]: ...

But I could also write for dictionaries:

if key in dic.keys() and dic[key]: ...

I just wonder why the syntactic sugar was added for dicts, but not for
sequence types.

Regards,
Jordan

[Fredrik Lundh]

MonkeeSage wrote:

True. But valid dictionary keys are exactly d.keys(). The has_key
method is just sugar.

for what? are you sure you're using "sugar" as it is usually used when
talking about computer languages?

</F>

[MonkeeSage]

On Oct 7, 12:37 pm, Fredrik Lundh <[e-mail]:

for what?

key in self.keys()

And d.get() looks like sugar for:

if self.has_key(key):
return self[key]
else:
return default_value

Why not have the same sugar for sequence types? E.g.,

def has_index(self, index):
return index < len(self)

def get(self, index, default=None):
if self.has_index(index):
return self[index]
return default

I understand not wanting to add bloat without use cases and / or when
the programmer can easily implement the functionality themselves; but I
don't see why the sugar would be added for dict but not sequence types.
The fact of easy implementation by the programmer without the
convenience method is the same in both cases, and I would assume (but
don't claim to know) that one could find many use cases for, e.g.,
list.has_index() / . get().

Regards,
Jordan

[Duncan Smith]

MonkeeSage wrote:

On Oct 7, 12:37 pm, Fredrik Lundh <[e-mail]:

>>for what?

key in self.keys()

[snip]

No. The above constructs a list of keys and searches the list for the
key, O(n). "key in somedict" is a lookup, O(1).

Duncan

[MonkeeSage]

On Oct 7, 7:14 pm, Duncan Smith <[e-mail]:

No. The above constructs a list of keys and searches the list for the
key, O(n). "key in somedict" is a lookup, O(1).

My point wasn't in regard to implementation details, but in regard to
convenience methods. Obviously the sugary dict methods are tweaked for
the best performance (one would hope!), as would be sugary sequence
methods were they to be added. What I was wondering is why
dict.has_key() and get() are there but not list.has_index() and get().

Regards,
Jordan

[Steven D'Aprano]

On Sat, 07 Oct 2006 10:26:22 -0700, MonkeeSage wrote:

>

On Oct 7, 3:27 am, Gabriel Genellina <[e-mail]:

>The meaning comes from the most common usage.

I wasn't suggesting that the "in" keyword have a different sematic for
sequence types. I was just saying that regarding the question whether
there is anything similar to "dict.has_key / k in dict" for lists, the
"in" keyword is not semantically analogous.

Are you just making a philosophical point? In which case I agree: *if* you
make the analogy "a dictionary key is analogous to a sequence index",
*then* the operation of "in" isn't semantically analogous between mappings
and sequences. But why should it be?

In both mappings and sequences, "in" tests for membership. But *what* it
tests for membership is different. There's no shame there.

>Because it's not needed at all: valid sequence indexes are *exactly*
range(len(seq)). This is the basic point of being a sequence: when
indexes are not contiguous, in fact you have a mapping, not a sequence.

True. But valid dictionary keys are exactly d.keys(). The has_key
method is just sugar.

A little bit of history might be useful here. Originally, if you wanted to
test for key membership with a dict, you had three choices:

(1) Easier To Ask Forgiveness Than Permission:

try:
mapping[key]
except KeyError:
print "Oops, that's not a key!"
(2) mapping.has_key(key), which inspected the dict's internals to
determine whether key was valid or not.

(3) Build a list of keys, then test whether the key was in the list:

key in mappings.keys()
Obviously all three methods had their disadvantages. The first required
catching an exception, which is expensive, and is less than clear. The
second is a magic method that needs to be remembered by the developer,
and breaks the equivalence of mappings and sequences. The third means
building, and throwing away, a temporary list of keys.

But after types and classes were unified, dicts grew a __contains__
method for subclassing. If you want syntactic sugar, "x in obj" is
"just" syntactic sugar for obj.__contains__(x) (but note the scare quotes
around just: one person's syntactic sugar is another person's essential
syntactic feature).

--
Steven.

[Steven D'Aprano]

On Sat, 07 Oct 2006 17:25:15 -0700, MonkeeSage wrote:

My point wasn't in regard to implementation details, but in regard to
convenience methods. Obviously the sugary dict methods are tweaked for
the best performance (one would hope!), as would be sugary sequence
methods were they to be added. What I was wondering is why
dict.has_key() and get() are there but not list.has_index() and get().

Because they aren't needed often, and when they are, they are easy to
implement?

Instead of list.has_index(n), write 0 <= n < len(list).

If you want to allow for negative indices as well, write
-len(list) <= n < len(list).
It is hard to see where list.get(index, default) would be useful, since
accessing an index out of range of a list is generally an error, unlike
accessing a missing key. But I like symmetry, and for symmetry I wouldn't
object to sequences gaining a get method too. Not that I care about it
enough to put in a feature request or a PEP, but if somebody else did, I
wouldn't object.

(Note, though, that accessing a *slice* out of range is not an error, but
returns an empty list.)

get() is easy enough to implement. Feel free to turn it into a method
of a collection class if you like.

def get(mapping_or_sequence, key_or_index, default=None):
try:
return mapping_or_sequence.get(key_or_index, default)
except AttributeError:
# probably a sequence
try:
return mapping_or_sequence[key_or_index]
except IndexError:
return default

Not every piece of functionality needs to be a built-in.
--
Steven.

[MonkeeSage]

On Oct 7, 7:41 pm, Steven D'Aprano
<[e-mail]:

Are you just making a philosophical point? In which case I agree: *if* you
make the analogy "a dictionary key is analogous to a sequence index",
*then* the operation of "in" isn't semantically analogous between mappings
and sequences. But why should it be?

It shouldn't, and I'm making a pragmatic (not merely philosophic) point
regarding the OP's question whether there is a "similar" list method to
dict.has_key.

In both mappings and sequences, "in" tests for membership. But *what* it
tests for membership is different. There's no shame there.

I know / agree.

Originally, if you wanted to
test for key membership with a dict, you had three choices:

I know.

On Oct 7, 7:59 pm, Steven D'Aprano
<[e-mail]:

Because they aren't needed often, and when they are, they are easy to
implement?

More often and easier to implement than dict.has_key / get?

It is hard to see where list.get(index, default) would be useful, since
accessing an index out of range of a list is generally an error, unlike
accessing a missing key.

Uh, no. KeyError.

But I like symmetry, and for symmetry I wouldn't
object to sequences gaining a get method too. Not that I care about it
enough to put in a feature request or a PEP, but if somebody else did, I
wouldn't object.

Me neither. :)

get() is easy enough to implement. Feel free to turn it into a method
of a collection class if you like.

I already did, see previous posts.

Not every piece of functionality needs to be a built-in.

Agreed. but why implement a certain functionality in one place but
leave it out of another?

Regards,
Jordan

[Duncan Smith]

MonkeeSage wrote:

>
On Oct 7, 7:14 pm, Duncan Smith <[e-mail]:

>>No. The above constructs a list of keys and searches the list for the
key, O(n). "key in somedict" is a lookup, O(1).

My point wasn't in regard to implementation details, but in regard to
convenience methods. Obviously the sugary dict methods are tweaked for
the best performance (one would hope!), as would be sugary sequence
methods were they to be added. What I was wondering is why
dict.has_key() and get() are there but not list.has_index() and get().

Regards,
Jordan

All I can say is that, I personally, would have little use for those
list methods. Those involved in the design of the language will no
doubt have much better reasons than this.

Duncan

[MonkeeSage]

On Oct 7, 8:06 pm, "MonkeeSage" <[e-mail]:

More often and easier to implement than dict.has_key / get?

More -Less

[Steven D'Aprano]

On Sat, 07 Oct 2006 18:06:47 -0700, MonkeeSage wrote:

On Oct 7, 7:59 pm, Steven D'Aprano
<[e-mail]:

>Because they aren't needed often, and when they are, they are easy to
implement?

More often and easier to implement than dict.has_key / get?

No, *less* often. That's the point -- it is fairly common for people to
want dictionary lookup to return a default value, but quite rare for them
to want sequence lookup to return a default value. A sequence with a
default value would be, in some sense, equivalent to an infinite list:

[A, B, ... , Z, default, default, default, ... ]

where the A...Z are actual values. (Obviously you don't actually have an
infinite list.)

I can't think of any practical use for such a thing; the only thing that
comes close is in some Cellular Automata it is sometimes useful to imagine
cells out of range to be always in some default state. But generally
speaking, you don't implement Cellular Automata with lists.

>It is hard to see where list.get(index, default) would be useful, since
accessing an index out of range of a list is generally an error, unlike
accessing a missing key.

Uh, no. KeyError.

dict.get() doesn't raise KeyError. That's the whole point of get(), it
returns a default value instead of raising KeyError.

>Not every piece of functionality needs to be a built-in.

Agreed. but why implement a certain functionality in one place but leave
it out of another?

Because implementation requires time and effort. It makes the regression
tests more complicated and bloats the code base. If the feature doesn't
scratch anybody's itch, if there isn't a clear use for it, it won't be
implemented. If you don't care enough to even make a formal feature
request, let alone a PEP, then why should people who care even less
actually write the code?
--
Steven.

[MonkeeSage]

On Oct 8, 5:57 am, Steven D'Aprano
<[e-mail]:

No, *less* often. That's the point -- it is fairly common for people to
want dictionary lookup to return a default value, but quite rare for them
to want sequence lookup to return a default value. A sequence with a
default value would be, in some sense, equivalent to an infinite list:

Ah, yes. Infinite dictionaries are much better! I guess you could think
of it like providing a infinitely indexed list (or infinitely keyed
dict), but a better way to think of it is as providing a
non-terminating exception where the default value is the exceptional
case. And I don't see why it would be so rare to do something like:

if sys.argv.get(1): ...

With list.has_index() / get(), the following (pretty common I think)
idiom:

try:
data = some_unknown_seq[2]
except IndexError:
data = None
if data: ...

Can become:

data = some_unknown_seq.get(2)
if data: ...

Perhaps list.get() wouldn't be used as often as dict.get(), but it
would be used a fair amount I think. Looking at the standard library
(2.5 source), I find 30 places where "except IndexError" appears, and
two places where a comment says that some method "raises IndexError" on
some condition. I haven't looked at the context of them, but I'd wager
that many of them would benefit from list.has_index() and / or get().
Here is my script to search the libs:

import os, re
found = {}
for path, dirs, files in os.walk('./Lib'):
for afile in files:
afile = open(os.path.join(path, afile))
lines = afile.readlines()
afile.close()
for line in lines:
match = re.search(r'((except|raises) IndexError)', line)
if match:
found[afile.name] = match.group(1)
for item in found.items():
print '%s (%s)' % item
print 'Found %d matches' % len(found)

dict.get() doesn't raise KeyError. That's the whole point of get(), it
returns a default value instead of raising KeyError.

Right. Exactly. Accessing a non-existent key raises a KeyError, but
dict.get() short-curcuits the exception and gives you a default value
(which is None unless explicitly changed). So instead of trying the key
and catching a KeyError, you can use a simple conditional and ask if
d.has_key(key), or assign d.get(key) and test the assignee. So, why
isn't there a list.has_index() / get()?

If you don't care enough to even make a formal feature
request, let alone a PEP, then why should people who care even less
actually write the code?

I'm thinking about it. I just wanted to see if anyone knew of, or could
come up with, a good reason why it isn't / shouldn't be there.
Apparently not (at least not one that doesn't also bite the dict
convenience methods), so I'll probably go ahead and make a feature
request in the next few days.

Regards,
Jordan

[Fredrik Lundh]

MonkeeSage wrote:

With list.has_index() / get(), the following (pretty common I think)
idiom:

try:
data = some_unknown_seq[2]
except IndexError:
data = None
if data: ...

umm. you could at least write:

try:
data = some_unknown_seq[2]
except IndexError:
pass
else:
... deal with data ...

but "let's hypergeneralize and treat sequences and mappings as the same
thing" proposals are nothing new; a trip to the archives might be help-
ful.

</F>

[MonkeeSage]

On Oct 8, 1:44 pm, Fredrik Lundh <[e-mail]:

but "let's hypergeneralize and treat sequences and mappings as the same
thing" proposals are nothing new; a trip to the archives might be help-
ful.

Huh? I don't want to treat sequences and mappings as the same thing.
I'm talking about adding two similar convenience methods for sequences
as already exist for mappings. That may make the two APIs closer, but
that's not necessarily a bad thing (think the square-bracket accessor).
Besides, has_index is sufficiently different already. If it's really a
problem, change get() to at() for sequences. seq.at(2).

So far the reasons offered against adding those convenience methods
are:

Reason: It's unnecessary / bloat.
- Then the same thing is true of the dict methods.

Reason: It's not useful.
- I know of several use cases and could probably find others.

Reason: It takes effort to implement it. Why don't you do it yourself
if it's such a great idea!
- Mabye I will. But that is only a reason why they aren't currently
implemented, not why they *shouldn't* be.

Reason: It makes sequences and mapping to much alike.
- Then change the names for the sequences methods.

That is to say, no good reason has been offered for why these methods
shouldn't be implemented.

Regards,
Jordan

[Steve Holden]

MonkeeSage wrote:

On Oct 8, 1:44 pm, Fredrik Lundh <[e-mail]:

>>but "let's hypergeneralize and treat sequences and mappings as the same
thing" proposals are nothing new; a trip to the archives might be help-
ful.

Huh? I don't want to treat sequences and mappings as the same thing.
I'm talking about adding two similar convenience methods for sequences
as already exist for mappings. That may make the two APIs closer, but
that's not necessarily a bad thing (think the square-bracket accessor).
Besides, has_index is sufficiently different already. If it's really a
problem, change get() to at() for sequences. seq.at(2).

So far the reasons offered against adding those convenience methods
are:

Reason: It's unnecessary / bloat.
- Then the same thing is true of the dict methods.

No: you are proposing to add features to the sequence interface for
which there are few demonstrable use cases.

Reason: It's not useful.
- I know of several use cases and could probably find others.

Well I certainly didn't find your last one particularly convincing: the
attempt to reference a non-existent sequence member is almost always a
programming error.

Reason: It takes effort to implement it. Why don't you do it yourself
if it's such a great idea!
- Mabye I will. But that is only a reason why they aren't currently
implemented, not why they *shouldn't* be.

Reason: It makes sequences and mapping to much alike.
- Then change the names for the sequences methods.

That is to say, no good reason has been offered for why these methods
shouldn't be implemented.

I would argue exactly the opposite: the reason why they shouldn't be
implemented is because no good reason has been presented why they *should*.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[Fredrik Lundh]

MonkeeSage wrote:

>but "let's hypergeneralize and treat sequences and mappings as the same
thing" proposals are nothing new; a trip to the archives might be help-
ful.

Huh? I don't want to treat sequences and mappings as the same thing.
I'm talking about adding two similar convenience methods for sequences
as already exist for mappings.

so what makes you think you're the first one who's ever talked about that?

</F>

[MonkeeSage]

On Oct 8, 3:05 pm, Steve Holden <[e-mail]:

No: you are proposing to add features to the sequence interface for
which there are few demonstrable use cases.

If I really wanted to find them, how many instances do you think I
could find [in the standard lib and community-respected third-party
libs] of sequence index checking like "if 2 < len(seq)" and / or
try-excepting like "try: seq[2] ..."? Judging by the fact that there
isn't any other way to *safely* handle dynamic sequences (esp.
sequences which grow based on side-effects which may or may not be
present, depending on the execution path through the code); I'd guess
the number is alot higher than you seem to think.

Well I certainly didn't find your last one particularly convincing: the
attempt to reference a non-existent sequence member is almost always a
programming error.

Unless you are interacting with user input, or other anomalous data
source. And in that case you need to do explicit index checking or wrap
your code with a try...except; that or you need a convenience function
or method that implements a non-terminating exception, and you just
check for the exceptional case, like dictionaries have with get(). I
find the latter approach to be easier to read and write (see link
below), as well as understand.

I would argue exactly the opposite: the reason why they shouldn't be
implemented is because no good reason has been presented why they *should*.

Pretend like there are no dict.has_key and dict.get methods. Can you
provide a good reason(s) why they should be implemented? Not necessity
-- since you can do the same thing more verbosely. Usefulness? --
Probably; but I think the list methods would also be useful (see
above). Succinctness [1]? -- The list methods have the same advantage.
Anything else?

[1] http://mail.python.org/pipermail/pyt...ly/000594.html
On Oct 8, 3:11 pm, Fredrik Lundh <[e-mail]:

Huh? I don't want to treat sequences and mappings as the same thing.
I'm talking about adding two similar convenience methods for sequences
as already exist for mappings.so what makes you think you're the first one who's ever talked about that?

I looked yesterday and only found a few posts. A couple involved
Marc-Andre Lemburg, and mxTools, where he has a get() function that
works for sequences and mappings; that's not what I'm suggesting.
However, I found one from 1997 where he mentioned a patch to python 1.5
which added list.get, but I couldn't find the patch or any discussion
of why it was (presumably) rejected. The only other post I found that
was relevant was one on the dev-python list (mentioned in the July 1-15
summery [1]). And the only thing mentioned there as a reason against it
is that "It encourages bad coding. You
shouldn't be searching lists and tuples like that unless you know what
you're doing." (Whatever that is supposed to mean!).

Just point me to the discussion where the good reasons (or any at all)
against my suggestion can be found and I'll be glad to read it. I
couldn't find it.

[1]
http://www.python.org/dev/summary/20...-and-tuple-get

Regards,
Jordan

[Steve Holden]

MonkeeSage wrote:

On Oct 8, 3:05 pm, Steve Holden <[e-mail]:

>>No: you are proposing to add features to the sequence interface for
which there are few demonstrable use cases.

If I really wanted to find them, how many instances do you think I
could find [in the standard lib and community-respected third-party
libs] of sequence index checking like "if 2 < len(seq)" and / or
try-excepting like "try: seq[2] ..."? Judging by the fact that there
isn't any other way to *safely* handle dynamic sequences (esp.
sequences which grow based on side-effects which may or may not be
present, depending on the execution path through the code); I'd guess
the number is alot higher than you seem to think.

Keep right on guessing.

>

>>Well I certainly didn't find your last one particularly convincing: the
attempt to reference a non-existent sequence member is almost always a
programming error.

Unless you are interacting with user input, or other anomalous data
source. And in that case you need to do explicit index checking or wrap
your code with a try...except; that or you need a convenience function
or method that implements a non-terminating exception, and you just
check for the exceptional case, like dictionaries have with get(). I
find the latter approach to be easier to read and write (see link
below), as well as understand.

OK, so now we appear to be arguing about whether a feature should go
into Python because *you* find it to be easier to read and write. But I
don't see a groundswell of support from other readers saying "Wow, I've
always wanted to do it like that".

>

>>I would argue exactly the opposite: the reason why they shouldn't be
implemented is because no good reason has been presented why they *should*.

Pretend like there are no dict.has_key and dict.get methods. Can you
provide a good reason(s) why they should be implemented? Not necessity
-- since you can do the same thing more verbosely. Usefulness? --
Probably; but I think the list methods would also be useful (see
above). Succinctness [1]? -- The list methods have the same advantage.
Anything else?

[1] http://mail.python.org/pipermail/pyt...ly/000594.html

Nope. In fact d.has_key(k) is a historical spelling, retained only for
backwards compatibility, of k in dict. As to the d.get(k, default)
method I really don't see a compelling use case despite your
protestations, and I don't seem to be alone. Please feel free to start
recruiting support.

>
On Oct 8, 3:11 pm, Fredrik Lundh <[e-mail]:

>>>Huh? I don't want to treat sequences and mappings as the same thing.
I'm talking about adding two similar convenience methods for sequences
as already exist for mappings.so what makes you think you're the first one who's ever talked about that?

I looked yesterday and only found a few posts. A couple involved
Marc-Andre Lemburg, and mxTools, where he has a get() function that
works for sequences and mappings; that's not what I'm suggesting.
However, I found one from 1997 where he mentioned a patch to python 1.5
which added list.get, but I couldn't find the patch or any discussion
of why it was (presumably) rejected. The only other post I found that
was relevant was one on the dev-python list (mentioned in the July 1-15
summery [1]). And the only thing mentioned there as a reason against it
is that "It encourages bad coding. You
shouldn't be searching lists and tuples like that unless you know what
you're doing." (Whatever that is supposed to mean!).

Just point me to the discussion where the good reasons (or any at all)
against my suggestion can be found and I'll be glad to read it. I
couldn't find it.

[1]
http://www.python.org/dev/summary/20...-and-tuple-get

The fact that nobody has listed the good reasons why I shouldn't try to
make a computer from mouldy cheese doesn't make this a good idea.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[MonkeeSage]

On Oct 9, 2:31 am, Steve Holden <[e-mail]:

Keep right on guessing.

I hope I'm not offending one whom I consider to be much more skilled
and versed than I am, not only in python, but in programming in
general; but I must say: it seems you are being rather obtuse here. I
think I laid out the principal clearly enough, and I know you have the
mental capacity to extrapolate from the principal to general use cases.
But even so, here is a simple use case from the standard library
(python 2.5 release source):

In Libs/site.py, lines 302-306:

try:
for i in range(lineno, lineno + self.MAXLINES):
print self.__lines[i]
except IndexError:
break

With my proposal, that could be written as:

for i in range(lineno, lineno + self.MAXLINES):
if self.__lines.has_index(i):
print self.__lines[i]
else:
break

Granted, in this particular case the amount of code is not reduced, but
(and I would hope you'd agree) the control flow is certainly easier to
follow.

OK, so now we appear to be arguing about whether a feature should go
into Python because *you* find it to be easier to read and write. But I
don't see a groundswell of support from other readers saying "Wow, I've
always wanted to do it like that".

*Someone* (other than me!) obviously found it nice to have the dict
convenience methods. As for garnishing support, I almost see that as
more of a cultural, rather than pragmatic issue. I.e., if it's not
there already, then it shouldn't be there: "what is is what should be".
Of course, consistently following that naive presumption would totally
stiffle *any* extension to python. However, (I think) I do understand
the psychology of the matter, and don't fault those who cannot see past
what already is (not meaning to implicate you or Fredrick or anyone
else -- the comment is innocent).

In fact d.has_key(k) is a historical spelling, retained only for
backwards compatibility, of k in dict. As to the d.get(k, default)
method I really don't see a compelling use case despite your
protestations, and I don't seem to be alone. Please feel free to start
recruiting support.

As I stated to another poster; I'm not really concerned with
implementation details, only with the presence or absence of
convenience methods. You can write "if k in d" as easily as "if index <
len(seq)". But semantically, they are similar enough, in my (admittedly
lowly) estimation, to desevere similar convenience methods.

The fact that nobody has listed the good reasons why I shouldn't try to
make a computer from mouldy cheese doesn't make this a good idea.

Heh. True. I didn't mean to imply that I was arguing from the negative.
I was only tring to shift the perspective to include the reasons for
the dict convenience methods. If C is good for A, and A is sufficiently
similar to B, then C is good for B. But if C is just crud, then forget
it all around. ;)
On Oct 9, 12:24 pm, Dennis Lee Bieber <[e-mail]:

But how do you handle the case of:

l = []
i = 10

l[i] = l.get(i, 0) + 1

You don't; you just let the IndexError fall through. Same as a KeyError
for d[k]. My propopsal is in regard to convencience methods, not to
direct access.

Ps. Sorry if this comes through twice, Google is being wierd right now.

Regards,
Jordan

[Steve Holden]

MonkeeSage wrote:

On Oct 9, 2:31 am, Steve Holden <[e-mail]:

>>Keep right on guessing.

I hope I'm not offending one whom I consider to be much more skilled
and versed than I am, not only in python, but in programming in
general; but I must say: it seems you are being rather obtuse here. I
think I laid out the principal clearly enough, and I know you have the
mental capacity to extrapolate from the principal to general use cases.
But even so, here is a simple use case from the standard library
(python 2.5 release source):

In Libs/site.py, lines 302-306:

try:
for i in range(lineno, lineno + self.MAXLINES):
print self.__lines[i]
except IndexError:
break

With my proposal, that could be written as:

for i in range(lineno, lineno + self.MAXLINES):
if self.__lines.has_index(i):
print self.__lines[i]
else:
break

Granted, in this particular case the amount of code is not reduced, but
(and I would hope you'd agree) the control flow is certainly easier to
follow.

>>OK, so now we appear to be arguing about whether a feature should go
into Python because *you* find it to be easier to read and write. But I
don't see a groundswell of support from other readers saying "Wow, I've
always wanted to do it like that".

*Someone* (other than me!) obviously found it nice to have the dict
convenience methods. As for garnishing support, I almost see that as
more of a cultural, rather than pragmatic issue. I.e., if it's not
there already, then it shouldn't be there: "what is is what should be".
Of course, consistently following that naive presumption would totally
stiffle *any* extension to python. However, (I think) I do understand
the psychology of the matter, and don't fault those who cannot see past
what already is (not meaning to implicate you or Fredrick or anyone
else -- the comment is innocent).

>>In fact d.has_key(k) is a historical spelling, retained only for
backwards compatibility, of k in dict. As to the d.get(k, default)
method I really don't see a compelling use case despite your
protestations, and I don't seem to be alone. Please feel free to start
recruiting support.

As I stated to another poster; I'm not really concerned with
implementation details, only with the presence or absence of
convenience methods. You can write "if k in d" as easily as "if index <
len(seq)". But semantically, they are similar enough, in my (admittedly
lowly) estimation, to desevere similar convenience methods.

>>The fact that nobody has listed the good reasons why I shouldn't try to
make a computer from mouldy cheese doesn't make this a good idea.

Heh. True. I didn't mean to imply that I was arguing from the negative.
I was only tring to shift the perspective to include the reasons for
the dict convenience methods. If C is good for A, and A is sufficiently
similar to B, then C is good for B. But if C is just crud, then forget
it all around. ;)
On Oct 9, 12:24 pm, Dennis Lee Bieber <[e-mail]:

> But how do you handle the case of:

l = []
i = 10

l[i] = l.get(i, 0) + 1

You don't; you just let the IndexError fall through. Same as a KeyError
for d[k]. My propopsal is in regard to convencience methods, not to
direct access.

Ps. Sorry if this comes through twice, Google is being wierd right now.

I think we'll just have to agree to differ in this repsecrt, as I don't
see your suggestions for extending the sequence API as particularly
helpful. The one case you quote doesn't actually use the construct to
supply a default value, it just terminates a loop early if the sequence
length is below expectation (unless I've misread the code: that does
happen).

I don't really think it's a biggy, I just don't see the parallels that
you do. Your point about blindness to the need for change is one that
we do have to be careful of (not that I'm the person to consult about
what goes into the language anyway). But if you think you've had an
argument here, try writing a PEP for this and see if you can persuade
the denizens of python-dev to accept (and implement) it.

regard
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[Terry Reedy]

"MonkeeSage" <Mo********@gmail.comwrote in message
news:11**********************@b28g2000cwb.googlegr oups.com...

But even so, here is a simple use case from the standard library
(python 2.5 release source):

In Libs/site.py, lines 302-306:

try:
for i in range(lineno, lineno + self.MAXLINES):
print self.__lines[i]
except IndexError:
break

Is there an outer loop being 'break'ed? If not, it should be pass instead.

With my proposal, that could be written as:

for i in range(lineno, lineno + self.MAXLINES):
if self.__lines.has_index(i):
print self.__lines[i]
else:
break

This break is swallowed by the for loop, so not exactly equivalent, I
think. In any case, these are both clumsy and I believe I would instead
write something like

for i in range(lineno, len(self.__lines)):
print self.__lines[i]

or even better, use islice -- for line in islice(...): print line
So not a persuasive use case to me.

Terry Jan Reedy

[Fredrik Lundh]

MonkeeSage wrote:

In Libs/site.py, lines 302-306:

try:
for i in range(lineno, lineno + self.MAXLINES):
print self.__lines[i]
except IndexError:
break

With my proposal, that could be written as:

for i in range(lineno, lineno + self.MAXLINES):
if self.__lines.has_index(i):
print self.__lines[i]
else:
break

Granted, in this particular case the amount of code is not reduced, but
(and I would hope you'd agree) the control flow is certainly easier to
follow.

so to "improve" a piece of code that's been optimized for the common
case, you're adding an extra method call and a test to the inner loop?

and this because you think Python programmers don't understand try-
except statements ?

I think we can all safely ignore you now.

</F>

[Fredrik Lundh]

Terry Reedy wrote:

Is there an outer loop being 'break'ed?

yes.

This break is swallowed by the for loop, so not exactly equivalent, I
think.

the code is supposed to break out of the outer loop when it runs out of
lines, so yes, monkeeboy's code is broken in more than one way.

In any case, these are both clumsy and I believe I would instead
write something like

for i in range(lineno, len(self.__lines)):
print self.__lines[i]

that doesn't do the same thing, either.

and you both seem to be missing that

try:
for loop over something:
do something with something
except IndexError:
...

is a common pydiom when you expect to process quite a few somethings,
and you don't want to waste time on calculating end conditions up front
or checking for end conditions inside the loop or otherwise looking
before you leap (LBYL); the pydiom lets you process things as quickly as
you possibly can, as long as you possibly can, and deal with the end of
the sequence when you hit it (EAFP*).

</F>

*) insert martelli essay here

[Fredrik Lundh]

*) insert martelli essay here.

for example:

http://mail.python.org/pipermail/pyt...ay/163820.html

[MonkeeSage]

On Oct 10, 1:57 am, Steve Holden <[e-mail]:

I think we'll just have to agree to differ in this repsecrt, as I don't
see your suggestions for extending the sequence API as particularly
helpful.

No worries. :)
On Oct 10, 11:22 am, Fredrik Lundh <[e-mail]:

so to "improve" a piece of code that's been optimized for the common
case, you're adding an extra method call and a test to the inner loop?

and this because you think Python programmers don't understand try-
except statements ?

Uh, yeah, "You're all idiots and I'm not really a 'Python
Programmer'(TM)" -- that's exactly what I was meaning to say. I'm
suprised your telepathic powers let you pick up on that, since I didn't
say anything that could even remotely be construed that way. Freaky.

And, FWIW, the "optimized" version is not much faster than that put
forth by the stupid peon (me), even when my suggestion is implemented
in pure python:

$ cat test.py
import timeit
ary = ['blah'] * 10

def has_index(seq, idx):
return idx < len(seq)
def at(seq, idx):
if has_index(seq, idx):
return seq[idx]

def t1():
while 1:
try:
for i in range(11):
ary[i]
except IndexError:
break
def t2():
go = True
while go:
for i in range(11):
if has_index(ary, i):
ary[i]
else:
go = False
def t3():
go = True
while go:
for i in range(11):
val = at(ary, i)
if val:
val
else:
go = False

print 't1 time:'
print timeit.Timer('t1()', 'from __main__ import t1').timeit()
print 't2 time:'
print timeit.Timer('t2()', 'from __main__ import t2').timeit()
print 't3 time:'
print timeit.Timer('t3()', 'from __main__ import t3').timeit()

$ python test.py
t1 time:
15.9402189255
t2 time:
18.6002299786
t3 time:
23.2494211197

I think we can all safely ignore you now.

You could have done that all along. I'm no Max Planck, and this isn't
quantum mechanics. But more interesting than whether it's "safe" to
ignore suggestions for improvement is whether its actually beneficial
to do so.
On Oct 10, 11:35 am, Fredrik Lundh <[e-mail]:

*) insert martelli essay here.for example:

http://mail.python.org/pipermail/pyt...ay/163820.html

I don't think this is strictly a question of EAFP vs. LBYL, but more a
question of convenience. This very moment in python I can say both
"try: d[k] ..." and "if d.has_key[k] / k in d".

Regards,
Jordan