Why not just show the out-of-range index?

Russ

Every Python programmer gets this message occasionally:

IndexError: list index out of range

The message tells you where the error occurred, but it doesn't tell you
what the range and the offending index are. Why does it force you to
determine that information for yourself when it could save you a step
and just tell you? This seems like a "no-brainer" to me. Am I missing
something?

Dec 4 '06

Subscribe Post Reply

4216

rurpy

Martin v. Löwis wrote:

ru***@yahoo.com schrieb:

>Rather, they (like I) will encourage to OP to submit
a patch that fixes the problem.
Now, that would be rather silly. I would have to familiarize
myself with the code for the Python interpreter,
Seems to me he called the suggestion (made without any
knowlage of the OP's abilities regarding C and Python's
internals) that he summit a patch, silly.

I aggree.

His response was well within the bounds of normal
usenet discourse.

Maybe I'm unusually picky, but I also feel insulted if
my suggestions are called silly - this is just like calling
myself silly. I rarely make silly suggestions deliberately
(and try to mark them as ironic in usenet if I do); so if
somebody puts them down as "silly", I'll feel insulted.

I personally don't think it is silly to suggest that an
IT professional becomes familiar with the implementation
of the Python interpreter. That code is well-written,
well-documented, so it should be feasible (rather than
being silly) for anybody with a programming background
and sufficient determination to familiarize with that code.

I think you are vastly overestimating the programming
abilities of many Python users.
The set of IT professionals includes a large number
of people who aren't programmers at all. The set of
programmers includes a large number of people who don't
know C. Even for someone who knows C, there are many
places in the Python code that require considerable
general knowlage of Python's overall architecure and
way of doing things that is not obtained without a fair
bit of time spent working with the code. I think your
existing familiarity with it is inteferring with your
ability to make an objective judgement of this.

To say a Python user should learn C, understand the
internals of Python sufficiently to create a patch as
a prerequisite to making a suggestion about how to
improve the user experience of Python, is, I personally
think, silly.

I take the same position for about any open-source software:
you *can* get into Apache, Mozilla, the Linux kernel,
and now the Java virtual machine if you want to. If you
don't, it's not because you can't, but because you don't
want to.

That's stretching the meaning of "want to" quite a bit.
Theere are lots of constraints on people that prevent
even the technically capapable of doing that. You can
write them off as not being sufficiently motivated if
you want, but I don't think that is being realistic.
Nor is it a good reason to disregard ideas for improving
Python coming from them.

It would be unrealistic (but not silly) to suggest that
if the source code weren't available at all. It is *not*
silly to suggest that people should make efforts to
contribute to open source software.

"Suggestion" covers a wide range of meanings. To
politely ask someone to consider submitting a patch
if they are able, with no implication of sanction
if they are not is one thing. To suggest that any
IT professional read and understand Python's internals
or be publically castigated is something else.

If I had meeting at work, and person X said I think
if department Y did things this way, we could save
$Z/year. Should I tell that person, "implement it
yourself, or be quiet". Sorry, I don't buy that.

Dec 5 '06 #51

rurpy

Robert Kern wrote:

ru***@yahoo.com wrote:
I saw no posts where there OP insulted anybody without being
insulted first. It is ironic the Mr. Kern was the most consistent
insulter while at the same time accusing the OP of rudeness.

As I said, insult is in the ear of the listener, so I apologize if anyone
construed my comments as insults. However, facts are facts, and I stated them as
I believe them. If you can pick out the precise comments that you felt were
insulting, I will be happy to attempt clarifying them in a way that you do not
find insulting.

Facts?

As I explained in another post, "encouraging" someone to submit
a patch without a clue about the poster's abilities or resources
is offensive.

"begging for a fix on comp.lang.python is ..."

I didn't see the OP "begging" for anything.

>"His suggestion that his time is worth more than that of anyone else..."

again (as the OP pointed out) no such claim was
made.

"and yes, you are being incredibly rude and insulting"

By responding in kind? I thought he was quite restrained
given the provocation.

"The way that you have been acting..."

Funny, I thought he was posting to c.l.p. Sounds like
he is a petulant 6 year old child.

In virtually every one of your posts you said things that
I certainly would have been offended by.

"You're also missing that *I'm trying to help you*"

I hope you understand that I'm trying to help *you*?

You don't need to bother clarifying, your meaning is clear.
It was the insulting tone you (and some others) expressed
that meaning in, while attacking the OP for being "rude"
that I found grossly unfair.

Dec 5 '06 #52

Robert Kern

ru***@yahoo.com wrote:

Robert Kern wrote:
>ru***@yahoo.com wrote:
>>I saw no posts where there OP insulted anybody without being
insulted first. It is ironic the Mr. Kern was the most consistent
insulter while at the same time accusing the OP of rudeness.
As I said, insult is in the ear of the listener, so I apologize if anyone
construed my comments as insults. However, facts are facts, and I stated them as
I believe them. If you can pick out the precise comments that you felt were
insulting, I will be happy to attempt clarifying them in a way that you do not
find insulting.

Facts?

As I explained in another post, "encouraging" someone to submit
a patch without a clue about the poster's abilities or resources
is offensive.

I can't even begin to fathom that. I pointed him to the one path that will
(nearly) ensure that he gets the fix that he wants.

>"begging for a fix on comp.lang.python is ..."
I didn't see the OP "begging" for anything.

As you wish. I'll admit that the choice of words had more emotional baggage than
perhaps the situation warranted. But you can't tell me that his choice of words
had any less. Or yours.

>"His suggestion that his time is worth more than that of anyone else..."
again (as the OP pointed out) no such claim was
made.

In your words, his "meaning is clear".

>"and yes, you are being incredibly rude and insulting"
By responding in kind? I thought he was quite restrained
given the provocation.

Given that I don't understand how pointing him to appropriate actions to take to
get his feature request implemented is a provocation, I don't buy this one, either.

>"The way that you have been acting..."
Funny, I thought he was posting to c.l.p. Sounds like
he is a petulant 6 year old child.

Pick any synonym you like, but I do believe that adults are capable of "acting".
I thought "act" was the least emotional and most appropriate of the bunch.
Perhaps I was wrong.

In virtually every one of your posts you said things that
I certainly would have been offended by.

I'll admit that you do seem easily offended.

>"You're also missing that *I'm trying to help you*"

I hope you understand that I'm trying to help *you*?

Yes, thank you. I specifically asked for clarification, and you provided it. I'm
not sure why you think that I wouldn't understand that. As Russ has been
fighting every practical suggestion (which I hope he can separate out from what
he believes are insults), I have no way of knowing that he understands that.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Dec 5 '06 #53

James Stroud

Russ wrote:

Fredrik Lundh wrote:

>>>Sorry I haven't thought this through 100%

obviously not.

And you didn't like the "tone" of some of my earlier posts?

Some people consider themselves above a sensible conversation.

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

Dec 5 '06 #54

James Stroud

Russ wrote:

Fredrik Lundh wrote:

>>>Sorry I haven't thought this through 100%

obviously not.

And you didn't like the "tone" of some of my earlier posts?

Some people consider themselves above a sensible conversation.

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

Dec 5 '06 #55

OKB (not okblacke)

John Machin wrote:

3. The OP asked only for values; you are asking for names and
values. If you have a magic flak jacket, please let me know; I'd
like to borrow it occasionally :-)

On reflection I think my alternative suggestion might be just as
good: the interpreter could indicate the character position within the
line. This also handles cases where the operands aren't simple names.
(Not that I have any idea how much harder/easier this would be to
implement.)

I ask for names and values because that is what I want. If I have
some expression containing a bunch of variables and an exception is
raised, the most helpful information for me is the source code token
that is causing the error, because I can see the source code in my text
editor. The value is not as important, because there may be many
different variables that could be holding the same incorrect value. I
only want to look at the one that is ACTUALLY holding the incorrect
value.

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown

Dec 5 '06 #56

Russ

Folks,

I'm truly sorry that so many feathers got ruffled in this thread. Let's
see if I can put this thing to rest gracefully.

Russ wrote:

My suggestion is trivial to implement and would benefit every Python
programmer (even if only slightly), so I don't think it is too much to
ask for.

Apparently I was wrong about the implementation being trivial. Tasks
often seem trivial -- until you actually try to do them. Yes, I should
have known better. Had I not wrote that little gem, I suppose a lot of
bad feelings could have been avoided.

Would it be nice to get the specifics about an index error? Yes. But is
it actually worth implementing? Well, since I have no intention of
implementing it myself, I can't make that call. I can only hope that
the Python maintainers decide that it is worth their effort to make
Python more convenient for the user. But even if they don't, I still
intend to continue using Python. The call is theirs to make. As I said
earlier, the "issue" is hardly a major one, and I'm sorry it got blown
out of proportion.

At this point I don't even plan to submit a formal request. I have too
many accounts and passwords already. If someone else wants to do it,
please have at it.

Dec 5 '06 #57

Fredrik Lundh

Russ wrote:

>you're forgetting that you're dealing with "squeaky wheel contributors"
here, not the kind of nice and helpful persons that actually make open
source work.

Please refrain from dishing out gratutious insults until you have a
clue what you are talking about. It just so happens that I *have* made
significant contributions to open-source software.

http://sourceforge.net/tracker/?grou...70&atid=355470

</F>

Dec 5 '06 #58

John Machin

OKB (not okblacke) wrote:

John Machin wrote:

3. The OP asked only for values; you are asking for names and
values. If you have a magic flak jacket, please let me know; I'd
like to borrow it occasionally :-)

On reflection I think my alternative suggestion might be just as
good: the interpreter could indicate the character position within the
line. This also handles cases where the operands aren't simple names.
(Not that I have any idea how much harder/easier this would be to
implement.)

OTTOMH, it would be easier. The compiler would "only" have to ensure
that the line number and position within the line of each operator was
available at run time (currently keeping only the line number of the
first line in each statement). The cost/benefit ratio would IMHO still
be far too high.

>
I ask for names and values because that is what I want. If I have
some expression containing a bunch of variables and an exception is
raised, the most helpful information for me is the source code token
that is causing the error, because I can see the source code in my text
editor. The value is not as important, because there may be many
different variables that could be holding the same incorrect value. I
only want to look at the one that is ACTUALLY holding the incorrect
value.

Firstly, you may say that you want to look only at the ONE that is
actually bound to THE incorrect value, but the interpreter has in
general no way of telling which one is bad. For example:

foo = "a"
bar = [1]
baz= "z"
goo = [26]
x = foo + bar

This causes: "TypeError: cannot concatenate 'str' and 'list' objects"

Which ONE out of foo and bar is actually bound to THE "incorrect"
value? Did you mean to write "foo + baz", or did you mean to write "goo
+ bar"?

Secondly, if you have so many variables referenced in one statement all
with the same operator (e.g. +) that you have difficulty in working out
where the problem is, I'd say that you have a statement that is far too
long, and you have far too many variables in your function.

Can you give a real example from your code where the location of such a
human error (you cause a variable to bound to a value of inappropriate
type) would be difficult to find, plus an estimate of how often you
would make such an error?

Cheers,
John

Dec 5 '06 #59

Fredrik Lundh

BJörn Lindqvist wrote:

Maybe so, but that doesn't mean that it is not possible to make the
IndexError messages Pythons sequence objects throws better. You don't
need to change the semantics of x[i].

are you claiming that the fact that you can change every single index
check somehow prove that you *had* thought this through when you argued
that it could be implemented as a global PyObject_IsIndexOutOfBounds
helper, that relies on ob_size ?

(maybe I should stop using a threaded newsreader to read this group;
it's obvious that nobody else ever bothers to check what subthread a
message is appearing in.)

have you posted your patch to the patch tracker, btw?

</F>

Dec 5 '06 #60

Hendrik van Rooyen

"Carsten Haese" <ca*****@uniqsys.comwrote:
8<-----------------------------------------

My sarcasm meter just exploded.

-Carsten

Will you miss it a lot?

- Hendrik

Dec 5 '06 #61

Ben Finney

"Russ" <uy*******@sneakemail.comwrites:

At this point I don't even plan to submit a formal request. I have too
many accounts and passwords already.

If the implication here (that submitting a change request requires an
account on the server) is true, then I must concur with Russ that it's
a significant barrier to entry against a user who has one suggestion
to make.

We all use software from many different vendors (or projects, or
whatever), so requiring separate, rarely-used accounts on each one's
tracking system is plenty enough to ensure that a great deal of
suggestions never go through that channel.

I hope that, instead, it's possible to perform the research needed to
describe the requested change, submit it as an email or online form,
and never have to care about an account and password on a server that
likely will never be visited again. This way, the mechanics of
submitting the request are a low enough burden that it's reasonable to
encourage people to go through that channel.

--
\ "I know you believe you understood what you think I said, but I |
`\ am not sure you realize that what you heard is not what I |
_o__) meant." -- Robert J. McCloskey |
Ben Finney

Dec 5 '06 #62

Fredrik Lundh

Ben Finney wrote:

I hope that, instead, it's possible to perform the research needed to
describe the requested change, submit it as an email or online form

are you perhaps volunteering to help setup and monitoring such a sub-
mission channel ?

it's a certain amount of work to keep out the spammers and scammers, as
should be obvious for anyone who's looked at an unmonitored wiki or bug
tracker (or looked at his mail program's spam folder), but nothing that
cannot be fit into a reasonably structured developer's daily routine.

</F>

Dec 5 '06 #63

stdazi

Usually, when I make some coding mistake (index out of range - in this
case) I just care to fix the mistake and I usually don't mind to
inspect by how much the index was overflowed. It really seems like a
feature that should be embedded in some Python debugger than a feature
in the interpreter itself.

Dec 5 '06 #64

Soni Bergraj

stdazi wrote:

It really seems like a
feature that should be embedded in some Python debugger than a feature
in the interpreter itself.

+1
--
Soni Bergraj
http://www.YouJoy.org/

Dec 5 '06 #65

Thomas Ploch

stdazi wrote:

Usually, when I make some coding mistake (index out of range - in this
case) I just care to fix the mistake and I usually don't mind to
inspect by how much the index was overflowed. It really seems like a
feature that should be embedded in some Python debugger than a feature
in the interpreter itself.

I read the whole thread and this is more or less the first post which
actually has a good thing to say without saying any bad thing about anyone.

Thomas

Dec 5 '06 #66

Aahz

In article <ma***************************************@python. org>,
Fredrik Lundh <fr*****@pythonware.comwrote:

>
(maybe I should stop using a threaded newsreader to read this group;
it's obvious that nobody else ever bothers to check what subthread a
message is appearing in.)

Hey! I've been using trn3.6 for more than fifteen years, and what's good
enough for me is good enough for you!
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

Member of the Groucho Marx Fan Club

Dec 5 '06 #67

Russ

stdazi wrote:

Usually, when I make some coding mistake (index out of range - in this
case) I just care to fix the mistake and I usually don't mind to
inspect by how much the index was overflowed. It really seems like a
feature that should be embedded in some Python debugger than a feature
in the interpreter itself.

Then why have the interpreter generate error messages at all? I think
it makes sense to make the interpreter as self-contained as possible,
within reason, so you don't depend on a debugger any more than
necessary.

And it's not that I care "how much the index was overflowed." It's that
something unexpected happened, and I want a clue what it was. I'll need
to find out somehow anyway. What is the next logical step if not to
find out what the out-of-range index was?

Having said that, I reiterate that I don't know the implementation
difficulty, so I defer on the decision as to whether it is worth
implementing or not.

Dec 5 '06 #68

rurpy

Russ wrote:

Folks,

I'm truly sorry that so many feathers got ruffled in this thread. Let's
see if I can put this thing to rest gracefully.

I too am tired of this and I apologize to you
(Russ) for jumping into it and for this (hopefullly
last) followup. But the same thing happened here
a short time ago and in that incident too, someone
who made a suggestion was attacked and charged
unjustifiably with rudeness.

In neither that case nor this did anyone protest
the unjustness and hypocrisy that was blatant.
So this time I could not sit idly and watch.

To summarize the "insults" subtread (for closure)

An observation was made about a suboptimal error
message and a suggestion about how to improve it
made in good faith. (Even harebrained suggestions
deserve a non-insulting explanation of why they're
not a good idea, and this was not harebrained.)

This resulted in a suggestion to "submit a patch".

The response was (correctly) that it was not
a viable suggestion. (characterized by "silly"
hardly a strong epithet on usenet), that it would
be "trivial" to fix and should "take 2 minutes" by
someone with the requisite experience (obviously
an opinion, not a claim, since the OP said in
the same post that he was unfamiliar with Python
internals).

Obviously it takes more than two minutes. This
was a usenet post for god's sake, not a PhD
dissertation. And "trivial" is used almost as slang
by many computer and science types and means
"very easy" (as if you didn't know).

o it is reasonable to question the description
"trivial" and ask what actually would need
to be done. It is not reasonable to be
insulted by it.
o Anyone who honestly interprets "two minutes"
as 120 seconds should seek clinical help.
A reasonable reading if this is that "it would
take someone who works on Python internals
regularly orders of magnitude less time than
me, and with a higher probability of getting
it right." Sorry folks. that is a factually
true statement. Nothing to do with your time
is less valuable than mine. More imaginary
insults.

The OP was blameless in this thread and this
was another case the regulars here overreacting
to imagined (or deliberately misinterpreted)
"insults" that were not there.

One can speculate that this serves a purpose
of suppressing interference from "outsiders"
in the development process. Or maybe it is
just the alpha dogs asserting their authority.
Either way, it should not be allowed to pass
uncriticised.

Dec 5 '06 #69

Ben Finney

Fredrik Lundh <fr*****@pythonware.comwrites:

Ben Finney wrote:

I hope that, instead, it's possible to perform the research needed
to describe the requested change, submit it as an email or online
form

are you perhaps volunteering to help setup and monitoring such a
sub- mission channel ?

I have done for other projects. I don't have the resources to do it
for every project that I'm interested in, and Python is one of many
that miss out.

I also wasn't insisting that anyone else do so; merely pointing out
some negative consequences of (what I understand to be) the current
situation, that may not be apparent to those who are closest to the
issue tracker.

--
\ "Everything is futile." -- Marvin of Borg |
`\ |
_o__) |
Ben Finney

Dec 6 '06 #70

OKB (not okblacke)

John Machin wrote:

Firstly, you may say that you want to look only at the ONE that is
actually bound to THE incorrect value, but the interpreter has in
general no way of telling which one is bad. For example:

foo = "a"
bar = [1]
baz= "z"
goo = [26]
x = foo + bar

This causes: "TypeError: cannot concatenate 'str' and 'list'
objects"

Which ONE out of foo and bar is actually bound to THE "incorrect"
value? Did you mean to write "foo + baz", or did you mean to write
"goo + bar"?

My amended proposal handles this, because the error message should
point to the line position of the operand. (Or if it points to one of
the operands, that's fine too, at least I know what's going on.)

Secondly, if you have so many variables referenced in one statement
all with the same operator (e.g. +) that you have difficulty in
working out where the problem is, I'd say that you have a statement
that is far too long, and you have far too many variables in your
function.

See my example below.

Can you give a real example from your code where the location of
such a human error (you cause a variable to bound to a value of
inappropriate type) would be difficult to find, plus an estimate of
how often you would make such an error?

Here is an example:

self.outFile.write(str(len(self.hits)) + ' in ' + searchInfo.recordName
+ '\t' + ']['.join((a. group() for a in self.hits)) + '\n')

This is from a text-searching tool I have written to search
linguistic corpora. This statement writes a summary line to the output
file indicating how many hits were found in that file.

The problem in this line was that I'd forgotten to put str() around
the len(). Now, it's not impossible to find the problem, because given
my knowledge of the program I know that that's the only part of the line
that would reasonably contain the number, but I still think it would be
a lot easier if there were a caret in the error message pointing to the
offending summand.

I'm glad you asked this question, though, because in searching for
examples in my code, I discovered that most of my beefs aren't actually
of this type. A lot of them are things like this:

someStr.split(someList)

Here I meant to write someList[0] (say), but have inadvertently
passed the list instead of one of its elements.

In this case I receive an error message that says "TypeError:
expected a character buffer object". My question is: why can this error
message not say what was encountered INSTEAD of a character buffer
object?

A similar situation occurs when I do someVar[0] and get an
"unsubscriptable object" error. The error does not even tell me the
type of the offending value, just that it is unsubscriptable.

So my conclusion from this is: is there a reason that every error
message of the form "expected foo" or "this object cannot be frotzed"
cannot be changed to something like "expected foo but found bar" or
"this FooType object cannot be frotzed"?

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown

Dec 7 '06 #71

Gabriel Genellina

At Thursday 7/12/2006 01:17, OKB (not okblacke) wrote:

So my conclusion from this is: is there a reason that every error
message of the form "expected foo" or "this object cannot be frotzed"
cannot be changed to something like "expected foo but found bar" or
"this FooType object cannot be frotzed"?

Some arguments from my head:
- because error messages should be short
- because error messages are not debugging tools (better use unit
tests to avoid most common errors)
- because it's a lot of work to actually obtain such information in
each posible case, and sometimes, it may not be posible.
- because none of the core developers has interest on it
- because nobody else cares to submit a patch
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

Dec 7 '06 #72

John Machin

OKB (not okblacke) wrote:

John Machin wrote:

Can you give a real example from your code where the location of
such a human error (you cause a variable to bound to a value of
inappropriate type) would be difficult to find, plus an estimate of
how often you would make such an error?

Here is an example:

self.outFile.write(str(len(self.hits)) + ' in ' + searchInfo.recordName
+ '\t' + ']['.join((a. group() for a in self.hits)) + '\n')

This is from a text-searching tool I have written to search
linguistic corpora. This statement writes a summary line to the output
file indicating how many hits were found in that file.

The problem in this line was that I'd forgotten to put str() around
the len(). Now, it's not impossible to find the problem, because given
my knowledge of the program I know that that's the only part of the line
that would reasonably contain the number, but I still think it would be
a lot easier if there were a caret in the error message pointing to the
offending summand.

It should be extremely easy (rather than "not impossible") for anybody
with half a clue to find the problem given only the offending statement
(rather than your "knowledge of the program"). There are 6
possibilities; 3 are string constants. That leaves us with
len(something), searchInfo.recordName, and "][".join(something). len()
very definitely returns an integer (unless some lunatic has bound the
name to something else) and it's the *first* of the possibilities --
one can normally expect execution to proceed from left to right.
recordName smells like a string. "][".join(blahblah) likewise unless
the rebinding mania is pandemic.

Sheesh.

>
I'm glad you asked this question, though, because in searching for
examples in my code, I discovered that most of my beefs aren't actually
of this type. A lot of them are things like this:

someStr.split(someList)

Here I meant to write someList[0] (say), but have inadvertently
passed the list instead of one of its elements.

In this case I receive an error message that says "TypeError:
expected a character buffer object". My question is: why can this error
message not say what was encountered INSTEAD of a character buffer
object?

A similar situation occurs when I do someVar[0] and get an
"unsubscriptable object" error. The error does not even tell me the
type of the offending value, just that it is unsubscriptable.

So my conclusion from this is: is there a reason that every error
message of the form "expected foo" or "this object cannot be frotzed"
cannot be changed to something like "expected foo but found bar" or
"this FooType object cannot be frotzed"?