By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,602 Members | 1,507 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,602 IT Pros & Developers. It's quick & easy.

spliting a list by nth items

P: n/a
I feel like this has probably been answered before, but I couldn't
find something quite like it in the archives. Feel free to point me
somewhere if you know where this has already been answered.

I have a list in a particular order that I want to split into two
lists: the list of every nth item, and the list of remaining items.
It's important to maintain the original order in both lists.

So the first list is simple:

nth_items = items[::n]

The second list is what's giving me trouble. So far, the best I've
come up with is:

non_nth_items = list(items)
del non_nth_items[::n]

Is this the right way to do this?

Steve
--
You can wordify anything if you just verb it.
- Bucky Katt, Get Fuzzy
Jul 18 '05 #1
Share this Question
Share on Google+
33 Replies


P: n/a
Steven Bethard wrote:
I feel like this has probably been answered before, but I couldn't
find something quite like it in the archives. Feel free to point me
somewhere if you know where this has already been answered.

I have a list in a particular order that I want to split into two
lists: the list of every nth item, and the list of remaining items.
It's important to maintain the original order in both lists.

So the first list is simple:

nth_items = items[::n]

The second list is what's giving me trouble. So far, the best I've
come up with is:

non_nth_items = list(items)
del non_nth_items[::n]

Is this the right way to do this?


There's probably no "right" way. Your way isn't the way I
would have thought to do it, I think. But it's brilliant. :-)
(Also simple and clear.)

-Peter
Jul 18 '05 #2

P: n/a
Steven Bethard wrote:
I have a list in a particular order that I want to split into two
lists: the list of every nth item, and the list of remaining items.
It's important to maintain the original order in both lists.

So the first list is simple:

nth_items = items[::n]

The second list is what's giving me trouble.


non_nth_items = [x for x in items if x % n]
--
Michael Hoffman
Jul 18 '05 #3

P: n/a
I believe you meant somthing like:

non_nth_items = [items[i-1] for i in range(1,len(items)-1) if i % n]

I don't think the "if x % n" won't work on his list of items

Larry Bates

"Michael Hoffman" <m.*********************************@example.com > wrote in
message news:ci**********@pegasus.csx.cam.ac.uk...
Steven Bethard wrote:
I have a list in a particular order that I want to split into two
lists: the list of every nth item, and the list of remaining items. It's
important to maintain the original order in both lists.

So the first list is simple:

nth_items = items[::n]

The second list is what's giving me trouble.


non_nth_items = [x for x in items if x % n]
--
Michael Hoffman

Jul 18 '05 #4

P: n/a
Larry Bates wrote:
I believe you meant somthing like:

non_nth_items = [items[i-1] for i in range(1,len(items)-1) if i % n]

I don't think the "if x % n" won't work on his list of items


D'oh! I wasn't thinking--the test list of items I used was
range(something), so of course it worked.

What I meant <wink> was:

non_nth_items = [x for index, x in enumerate(items) if index % n]

Thanks for the correction.
--
Michael Hoffman
Jul 18 '05 #5

P: n/a
Larry Bates <lbates <at> swamisoft.com> writes:

I believe you meant somthing like:

non_nth_items = [items[i-1] for i in range(1,len(items)-1) if i % n]


Yeah, the items list is not of the form range(x) -- in fact, my current use
for this is with a list of filenames -- so Michael Hoffman's solution:
non_nth_items = [x for x in items if x % n]


wouldn't work. In the full fledged problem, I actually have a start parameter
for the slice as well as the step parameter (n), so I could probably do
something along these lines (inspired by your two suggestions):

non_nth_items = [item for i, item in enumerate(items) if (i - start) % n]

instead of my original

non_nth_items = list(items)
del non_nth_items[start::n]

The enumerate/range solution is a little more verbose, but it does only go
through the list once. Thanks for the good suggestions!

STeve
Jul 18 '05 #6

P: n/a
[Steven Bethard]
so I could probably do
something along these lines (inspired by your two suggestions):

non_nth_items = [item for i, item in enumerate(items) if (i - start) % n]

instead of my original

non_nth_items = list(items)
del non_nth_items[start::n]

The enumerate/range solution is a little more verbose, but it does only go
through the list once. Thanks for the good suggestions!


The enumerate solution has a lot going for it. It is more flexible and easily
accomodates criteria other than every nth item. More importantly, it
demonstrates an important guiding principle: multiple list deletions are a code
smell indicating that building a new list is a better approach. On this
newsgroup, I've seen people tie themselves in knots with multiple in-place
deletions during iteration when the new list approach was clearer, more robust,
and faster.

The runtime behavior of the non_nth_items[start::n] approach is implementation
dependent. One can easily imagine a O(n**2) process running behind the scenes.
CPython is a smarter than that, but the code for list_ass_subscript() in
listobject.c is not a pretty sight.
Raymond Hettinger
Jul 18 '05 #7

P: n/a
Raymond Hettinger writes:
The enumerate solution has a lot going for it. It is more flexible and
easily accomodates criteria other than every nth item.
Yeah, I'm not so much worried about this, or I wouldn't be using
items[start::n]
to make the nth_items list in the first place. (If I was planning to allow
for other criteria, I'd probably use filter or ifilter instead of the
slicing...)
More importantly, it demonstrates an important guiding principle: multiple
list deletions are a code smell indicating that building a new list is a
better approach.
This was kinda my feeling -- hence why I posted. Part of the problem for me
was that because the two lists were so closely related semantically, I was
looking for a syntax that was also closely related. Sometimes this is a good
intuition to have, sometimes not. ;)
The runtime behavior of the non_nth_items[start::n] approach is
implementation dependent.


Ahh, I didn't know that. Good thing to know...

Thanks,

Steve

Jul 18 '05 #8

P: n/a
>>> x = [0,1,2,3,4,5,6,7,8,9]
n = 3
y = [x[i] for i in range(len(x)) if i%n==0]
z = [x[i] for i in range(len(x)) if i%n<>0]
x [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] y [0, 3, 6, 9] z

[1, 2, 4, 5, 7, 8]
"Steven Bethard" <st************@gmail.com> wrote in message
news:ma**************************************@pyth on.org...
| I feel like this has probably been answered before, but I couldn't
| find something quite like it in the archives. Feel free to point me
| somewhere if you know where this has already been answered.
|
| I have a list in a particular order that I want to split into two
| lists: the list of every nth item, and the list of remaining items.
| It's important to maintain the original order in both lists.
|
| So the first list is simple:
|
| nth_items = items[::n]
|
| The second list is what's giving me trouble. So far, the best I've
| come up with is:
|
| non_nth_items = list(items)
| del non_nth_items[::n]
|
| Is this the right way to do this?
|
| Steve
| --
| You can wordify anything if you just verb it.
| - Bucky Katt, Get Fuzzy
Jul 18 '05 #9

P: n/a
I was one of the frequent posters on the many tools/ides/gui threads
that appeared here over the past weeks. I'm now partly satisfied. My
search for a good IDE that is usable in my lowend machine is still not
completed. But I found a good setup using a free editor and PythonWin
working together.

I'm using Crimson Editor, a small freeware editor. The most
interesting part is that it isn't the most featured of all editors
I've tried; it isn't as powerful as vi or emacs, but has one feature
that I wanted the most, that is a fairly simple project management
tool. It simply allows me to organize a list of files that I'm using
for a project on the left pane. I can click on a file to open it. It's
amazingly useful when you're dealing with "hot" files stored on
several different directories, and you don't have to remember where is
it located, or to manually walk the open file dialog looking for it.

I'm using PythonWin for testing, mostly because the Windows console is
so braindead, and because PythonWin help is quick well organized (it's
about the same that you find in the web, but a little bit more
convenient).

I'm also evaluating Wing IDE 2.0; I had a few issues with it that were
solved, but for some tasks I found it too heavy for my limited
resources. Anyway, I'm keeping an eye on it.

--
http://mail.python.org/mailman/listinfo/python-list

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com
Jul 18 '05 #10

P: n/a
Carlos Ribeiro <ca********@gmail.com> writes:
I'm using Crimson Editor, a small freeware editor. The most
interesting part is that it isn't the most featured of all editors
I've tried; it isn't as powerful as vi or emacs, but has one feature
that I wanted the most, that is a fairly simple project management
tool. It simply allows me to organize a list of files that I'm using
for a project on the left pane. I can click on a file to open it. It's
amazingly useful when you're dealing with "hot" files stored on
several different directories, and you don't have to remember where is
it located, or to manually walk the open file dialog looking for it.


I'm an Emacs user, but you might try looking at http://ecb.sf.net as a
'plugin' to (X)Emacs. It comes installed with XEmacs...

Also, I can't help with Windows. I find that tools there are very poor
when compared to what we have on unices :-)
--
Godoy. <go***@ieee.org>
Jul 18 '05 #11

P: n/a
Carlos Ribeiro wrote:
I'm using PythonWin for testing, mostly because the Windows console is
so braindead, and because PythonWin help is quick well organized (it's
about the same that you find in the web, but a little bit more
convenient).


<blatant plug>

You may want to try ipython (http://ipython.scipy.org). It gives you under
Windows (or linux, or OSX) colored tracebacks, tab-completion, directory
management, integrated pdb loading on uncaught exceptions, profiler support,
and quite a bit more.

I don't use windows, but under linux my development setup is Xemacs+ a terminal
with ipython running. Thanks to ctypes and UNCreadline, you get exactly the
same functionality under windows (along with your editor of choice). I
honestly find it to be a _very_ efficient setup.

</plug>

Cheers,

f
Jul 18 '05 #12

P: n/a
Carlos Ribeiro wrote:
I was one of the frequent posters on the many tools/ides/gui threads
that appeared here over the past weeks. I'm now partly satisfied. My
search for a good IDE that is usable in my lowend machine is still not
completed. But I found a good setup using a free editor and PythonWin
working together.

I'm using Crimson Editor, a small freeware editor. The most
interesting part is that it isn't the most featured of all editors
I've tried; it isn't as powerful as vi or emacs, but has one feature
that I wanted the most, that is a fairly simple project management
tool. It simply allows me to organize a list of files that I'm using
for a project on the left pane. I can click on a file to open it. It's
amazingly useful when you're dealing with "hot" files stored on
several different directories, and you don't have to remember where is
it located, or to manually walk the open file dialog looking for it.

I'm using PythonWin for testing, mostly because the Windows console is
so braindead, and because PythonWin help is quick well organized (it's
about the same that you find in the web, but a little bit more
convenient).

I'm also evaluating Wing IDE 2.0; I had a few issues with it that were
solved, but for some tasks I found it too heavy for my limited
resources. Anyway, I'm keeping an eye on it.

--
http://mail.python.org/mailman/listinfo/python-list


i do a lot of python (and jython) programming and i use crimson editor and pythonwin also. in fact, i use crimson
editor for everything. i've even associated just about every file type in windows to crimson editor :)
bryan

Jul 18 '05 #13

P: n/a
Jeremy Jones wrote:
Another tool that I use that I just started using but I'm finding
indispensable is IPython. Features:
<blush> :) I'm glad you like it, though.
* tab completion - this is probably the hugest benefit.
* executes (some? all?) OS commands without having the exit out -
tab completion works on filesystem paths here
The @rehash magic command will load _all_ of your $PATH as aliases, by default
only the most common things are loaded. You can just type alias to see what's
active. Note that the pysh profile (ipython -p pysh) preloads your whole path,
and adds a few more modifications, so you can use ipython as a system shell,
but with python syntax (and full two-way communication of variables). Here's
an example:

fperez@maqroll[~/test]|8> $$files=ls
fperez@maqroll[~/test]|9> type(files)
<9> <type 'list'>
fperez@maqroll[~/test]|10> len files
-------------------------> len(files)
<10> 33
fperez@maqroll[~/test]|11> for f in files:
|..> if len(f) > 10:
|..> wc -l $f
|..>
4 inspectbug.py
73 ramptest.py

Granted, it's silly, but I don't know in the shell how I could easily say "run
wc -l over all files whose name is longer than 10 characters". I can _never_
remember how to do complicated logic/looping in the shell, while I can
certainly do such things trivially in python. The pysh profile puts ipython in
a mode which bridges this gap. Incidentally, pysh is actually just a special
configuration mode for ipython, consisting of about only 36 lines of code (the
rest is regular ipython).
* probably more....but I'm still digging into this


Lots more :)

Best,

f

Jul 18 '05 #14

P: n/a

"Carlos Ribeiro" <ca********@gmail.com> wrote in message
news:ma**************************************@pyth on.org...
I'm also evaluating Wing IDE 2.0; I had a few issues with it that were
solved, but for some tasks I found it too heavy for my limited
resources. Anyway, I'm keeping an eye on it.


I didn't follow the entire IDE dispute and I apologize if my comments are
now redundant, but here are my 2 cents.

I just bought the Komodo Personal edition. I had already used the trial
download for a month and I've been quite satisfied with it. That's after
using PythonWin for a few months, but it would be hard to go back to
PythonWin now.

Before making the decision to buy Komodo, I also evaluated Wing IDE 2.0,
their Personal edition. Please indulge me, but I have to take a load off my
chest about Wing IDE.

Right off the bat, the trial period was only 10 days. I've evaluated quite
a few tools that had a free trial period and I don't think I can remember of
ever having only 10 days before. Then, the GUI is kind of awkward in a few
places (the "project tree view", or something like that, and the toolbars).
There is more. The Personal edition does not have an object browser (it's
only in their more expensive editions). With all due respect, that's a
minimal requirement in a commercial IDE these days and Komodo has one in its
Personal edition (for the same price as Wing IDE).

I left my biggest beef with Wing IDE for the end. I debugged one of my
scripts (that I knew it works) and got an unexplainable exception. It turns
out that Wing IDE falsely detects some exceptions. They know about it and
they suggest that if you KNOW that it's a false exception you should flag it
to be ignored. Yes, Wingware people, I know it's a false exception, but
what am I using an IDE with debugging for if I have to figure out false
exceptions and ignore them? And how do I know that that particular
exception will not be valid some day in one of my scripts? And please, do
not insult our intelligence with some marketing mumbo-jumbo that Wing IDE's
exception detection is so powerful that it even detects exceptions that are
not there. It's a BUG, it's not a feature, folks! And no excuse will make
up for it!

The choice between Wing IDE and Komodo was clear for me.

As a side note, among free open-source IDEs, I thought that pydev (a plugin
for eclipse) shows promise, but it's not ready yet for the big leagues.

There. Thanks for listening.

Dan
Jul 18 '05 #15

P: n/a
Dan Perl wrote:
I debugged one of my
scripts (that I knew it works) and got an unexplainable exception. It turns
out that Wing IDE falsely detects some exceptions. They know about it and
they suggest that if you KNOW that it's a false exception you should flag it
to be ignored. Yes, Wingware people, I know it's a false exception, but


What exactly is a "false exception"? I have never heard of such
a thing, and can't imagine what it might be. Python raises
exceptions, they are subclasses of Exception or (old-style)
strings, they have tracebacks, they come from a variety of
sources, there's lots of things to know about exceptions but
"falseness" is not something about which I'm aware...

-Peter
Jul 18 '05 #16

P: n/a
On Sun, 26 Sep 2004 21:36:18 -0400, Peter Hansen <pe***@engcorp.com> wrote:
Dan Perl wrote:
I debugged one of my
scripts (that I knew it works) and got an unexplainable exception. It turns
out that Wing IDE falsely detects some exceptions. They know about it and
they suggest that if you KNOW that it's a false exception you should flag it
to be ignored. Yes, Wingware people, I know it's a false exception, but


What exactly is a "false exception"? I have never heard of such
a thing, and can't imagine what it might be. Python raises
exceptions, they are subclasses of Exception or (old-style)
strings, they have tracebacks, they come from a variety of
sources, there's lots of things to know about exceptions but
"falseness" is not something about which I'm aware...


They're false in the sense that that they're not supposed to be raised
to the attention of the user. What happens is that Wing IDE is super
sensitive with regards to exceptions; when an exception is raised in
some internal module, Wing tries to outsmart the library and warn the
user that an exception happened. A "false exception" is one that would
happen and be silently treated by the library itself, without nobody
noticing, but that's catched by Wing IDE nonetheless, in a totally
unnecessary and obstrusive way. You can see that it's a case where the
tool tries to be too smart for its own good.

p.s. Why do they do it? For what I could see, the intention seems to
be able to catch situations where a valid exception that should be
raised to the user attention is silented by a generic error handler.
It really may be useful in some situations, but it isn't a reasonable
default in my opinion.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com
Jul 18 '05 #17

P: n/a
Yes, that's what I was talking about. Thanks, Carlos, for explaining it. I
actually should have said false detection of exceptions or wrong detection
of exceptions. I don't remember what Wingware are calling it. Actually,
the way they were qualifying it was something like "if you do not see this
exception when running the script outside the IDE, then you should probably
flag it to be ignored". And they are giving a list of builtin modules where
those exceptions are usually "falsely" detected.

I didn't try to figure out what their exception detection mechanism is and
why they have this problem. I just uninstalled the IDE. So 10 days for the
trial was more than enough after all. ;-) I think I had it installed for
about 2 days. I can tolerate a bug and I wouldn't be so riled up, but what
got me was the marketing spin and how they were just excusing the bug, like
they would never fix it.

The "false exceptions" and the lack of a class browser made me decide not to
choose Wing IDE this time. Their attitude about the "false exceptions" made
me cut the trial short and never to consider Wing IDE again.

Dan

"Carlos Ribeiro" <ca********@gmail.com> wrote in message
news:ma**************************************@pyth on.org...
On Sun, 26 Sep 2004 21:36:18 -0400, Peter Hansen <pe***@engcorp.com>
wrote:
Dan Perl wrote:
> I debugged one of my
> scripts (that I knew it works) and got an unexplainable exception. It
> turns
> out that Wing IDE falsely detects some exceptions. They know about it
> and
> they suggest that if you KNOW that it's a false exception you should
> flag it
> to be ignored. Yes, Wingware people, I know it's a false exception,
> but


What exactly is a "false exception"? I have never heard of such
a thing, and can't imagine what it might be. Python raises
exceptions, they are subclasses of Exception or (old-style)
strings, they have tracebacks, they come from a variety of
sources, there's lots of things to know about exceptions but
"falseness" is not something about which I'm aware...


They're false in the sense that that they're not supposed to be raised
to the attention of the user. What happens is that Wing IDE is super
sensitive with regards to exceptions; when an exception is raised in
some internal module, Wing tries to outsmart the library and warn the
user that an exception happened. A "false exception" is one that would
happen and be silently treated by the library itself, without nobody
noticing, but that's catched by Wing IDE nonetheless, in a totally
unnecessary and obstrusive way. You can see that it's a case where the
tool tries to be too smart for its own good.

p.s. Why do they do it? For what I could see, the intention seems to
be able to catch situations where a valid exception that should be
raised to the user attention is silented by a generic error handler.
It really may be useful in some situations, but it isn't a reasonable
default in my opinion.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

Jul 18 '05 #18

P: n/a
I've done a little more digging into the Wing IDE documentation and this is
one of the better explanations I've found on the "false exceptions":

"As you try out the various demos from the tree on the left of the wxPython
demo app, you will sometimes see Wing IDE report exceptions in its Error
List dialog, which will come to the front as the wxPython demo pauses at the
exception. These are not program errors or a malfunction of the IDE. As
described at the end of the Error List text, these are caused by Wing's
proactive exception detection algorithm, which cannot see into the C and C++
code that is handling these exceptions outside of the debugger.
To get past them, select "Ignore this exception location" in the Error List
window and click on Continue Execution. You will see 3-4 of these the first
time Wing IDE encounters them. After that, your ignore list is stored in the
project so you will never see them again, even in future debug sessions.
Subsequently, you will benefit from Wing's ability to stop immediately at
the point of exception, rather than after the fact. This makes understanding
the conditions leading to an error much easier and speeds up debug
turn-around."

I can see a benefit in stopping at the point of exception, but if that
causes false positives (term used by one of their support people) with the
builtin modules that are implemented in C/C++, then I'd rather just live
without this benefit. Apparently, there is a way to disable that behavior
(I found out about it only now), but Wing keeps recommending the "Ignore
this exception location" solution.

And here is the kind of comment that really bothers me. An actual statement
from their documentation ("Debugging Zope with Wing IDE"):

"Another useful exception related feature is the ability to ignore
exceptions selectively by line of code on which they occur. You can choose
this option in the Error List window that comes up when an exception occurs.
In Zope, as in other Python apps that contain some non-Python code, Wing
will incorrectly identify some exceptions as unhandled because it cannot see
whether enclosing C code is going to handle the exception. This usually
happens in 1-2 places in Zope and can be dealt with by ignoring subsequent
instances of the exception. The zope.wpr project distributed with the
Wing+Zope bundle already has these ignores in place."

No, this is NOT a "useful exception related feature", it's a workaround for
a bug. And it's 1-2 times for Zope, it was 3-4 times in the comments above,
how do they get these estimates? I got this problem (er, useful exception
related feature) with the first script I debugged.

Dan

"Carlos Ribeiro" <ca********@gmail.com> wrote in message
news:ma**************************************@pyth on.org...
On Sun, 26 Sep 2004 21:36:18 -0400, Peter Hansen <pe***@engcorp.com>
wrote:
Dan Perl wrote:
> I debugged one of my
> scripts (that I knew it works) and got an unexplainable exception. It
> turns
> out that Wing IDE falsely detects some exceptions. They know about it
> and
> they suggest that if you KNOW that it's a false exception you should
> flag it
> to be ignored. Yes, Wingware people, I know it's a false exception,
> but


What exactly is a "false exception"? I have never heard of such
a thing, and can't imagine what it might be. Python raises
exceptions, they are subclasses of Exception or (old-style)
strings, they have tracebacks, they come from a variety of
sources, there's lots of things to know about exceptions but
"falseness" is not something about which I'm aware...


They're false in the sense that that they're not supposed to be raised
to the attention of the user. What happens is that Wing IDE is super
sensitive with regards to exceptions; when an exception is raised in
some internal module, Wing tries to outsmart the library and warn the
user that an exception happened. A "false exception" is one that would
happen and be silently treated by the library itself, without nobody
noticing, but that's catched by Wing IDE nonetheless, in a totally
unnecessary and obstrusive way. You can see that it's a case where the
tool tries to be too smart for its own good.

p.s. Why do they do it? For what I could see, the intention seems to
be able to catch situations where a valid exception that should be
raised to the user attention is silented by a generic error handler.
It really may be useful in some situations, but it isn't a reasonable
default in my opinion.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com

Jul 18 '05 #19

P: n/a
Dan Perl wrote:
Yes, that's what I was talking about. Thanks, Carlos, for explaining it. I
actually should have said false detection of exceptions or wrong detection
of exceptions. I don't remember what Wingware are calling it. Actually,
the way they were qualifying it was something like "if you do not see this
exception when running the script outside the IDE, then you should probably
flag it to be ignored". And they are giving a list of builtin modules where
those exceptions are usually "falsely" detected.
As a recent convert to Wing I think you aren't giving them enough
credit: remember, this only happens in the 2.0 BETA product (or, if it
also happens in earlier versions, they are definitely planning to
minimize the effects in the production version).
I didn't try to figure out what their exception detection mechanism is and
why they have this problem. I just uninstalled the IDE. So 10 days for the
trial was more than enough after all. ;-) I think I had it installed for
about 2 days. I can tolerate a bug and I wouldn't be so riled up, but what
got me was the marketing spin and how they were just excusing the bug, like
they would never fix it.
Well, I'm sure I remember reading somewhere that before they go into
production they plan to add a standard list of such exceptions for the
various Python versions, so that the standard libraries are far less
troublesome in this respect.

Plus, IIRC, all you have to do is check an "ignore this exception" box
to have a specific exception ignored everafter, which didn't seem like a
huge deal to me.
The "false exceptions" and the lack of a class browser made me decide not to
choose Wing IDE this time. Their attitude about the "false exceptions" made
me cut the trial short and never to consider Wing IDE again.

I find that it's very useful to be able to select from a dropdown list
of defined classes, and then select from a further drop-down list of
methods of that class. While it may not be a full class browser it's
certainly excellent functionality.

regards
Steve
Jul 18 '05 #20

P: n/a

"Steve Holden" <st***@holdenweb.com> wrote in message
news:41************@holdenweb.com...
Dan Perl wrote:
Yes, that's what I was talking about. Thanks, Carlos, for explaining it.
I actually should have said false detection of exceptions or wrong
detection of exceptions. I don't remember what Wingware are calling it.
Actually, the way they were qualifying it was something like "if you do
not see this exception when running the script outside the IDE, then you
should probably flag it to be ignored". And they are giving a list of
builtin modules where those exceptions are usually "falsely" detected.
As a recent convert to Wing I think you aren't giving them enough credit:
remember, this only happens in the 2.0 BETA product (or, if it also
happens in earlier versions, they are definitely planning to minimize the
effects in the production version).


According to the documentation this was happening in much earlier,
production, versions. You are saying "they are definitely planning to
minimize the effects in the production version". Can you please qualify
that? Minimize how? What does "minimizing the effects" mean anyway?
I didn't try to figure out what their exception detection mechanism is
and why they have this problem. I just uninstalled the IDE. So 10 days
for the trial was more than enough after all. ;-) I think I had it
installed for about 2 days. I can tolerate a bug and I wouldn't be so
riled up, but what got me was the marketing spin and how they were just
excusing the bug, like they would never fix it.

Well, I'm sure I remember reading somewhere that before they go into
production they plan to add a standard list of such exceptions for the
various Python versions, so that the standard libraries are far less
troublesome in this respect.


A "standard" list? I don't see what's "standard" about that. And the
"standard libraries" are not "troublesome", the IDE is. On the other hand,
I suppose that this would be an "effect minimization". I don't know what
their plans are, but they could include a list of all the possible false
positives and automatically ignore them. Forgive me, but that is a hack and
I will not have any respect for that.

You are using the same kind of marketing spin speak that Wingware is using.
I can understand that you like the tool and you feel you have to defend it.
But I think this is the wrong way to do it, both for you and for Wingware.

I probably shouldn't get so worked up about it. To be fair, since I tried
the tool I have discovered that Wing IDE also has the option to disable that
behavior and to debug in a mode where unhandled exceptions are just treated
like a normal run would treat them.
Plus, IIRC, all you have to do is check an "ignore this exception" box to
have a specific exception ignored everafter, which didn't seem like a huge
deal to me.


It's just too bad I don't have a checkbox for "Ignore marketing spin at this
location".

Dan
Jul 18 '05 #21

P: n/a
Dan Perl wrote:
"Steve Holden" <st***@holdenweb.com> wrote in message
news:41************@holdenweb.com...
[...]

As a recent convert to Wing I think you aren't giving them enough credit:
remember, this only happens in the 2.0 BETA product (or, if it also
happens in earlier versions, they are definitely planning to minimize the
effects in the production version).

According to the documentation this was happening in much earlier,
production, versions. You are saying "they are definitely planning to
minimize the effects in the production version". Can you please qualify
that? Minimize how? What does "minimizing the effects" mean anyway?

All I was doing was pointing out htat they are aware of hte phenomenon,
and that I (unlike you, apparently) am prepared to live with a minor nit
in order to get the advantages the tool gives me.
I didn't try to figure out what their exception detection mechanism is
and why they have this problem. I just uninstalled the IDE. So 10 days
for the trial was more than enough after all. ;-) I think I had it
installed for about 2 days. I can tolerate a bug and I wouldn't be so
riled up, but what got me was the marketing spin and how they were just
excusing the bug, like they would never fix it.

Well, I'm sure I remember reading somewhere that before they go into
production they plan to add a standard list of such exceptions for the
various Python versions, so that the standard libraries are far less
troublesome in this respect.

A "standard" list? I don't see what's "standard" about that. And the
"standard libraries" are not "troublesome", the IDE is. On the other hand,
I suppose that this would be an "effect minimization". I don't know what
their plans are, but they could include a list of all the possible false
positives and automatically ignore them. Forgive me, but that is a hack and
I will not have any respect for that.

Well, fine.
You are using the same kind of marketing spin speak that Wingware is using.
I can understand that you like the tool and you feel you have to defend it.
But I think this is the wrong way to do it, both for you and for Wingware.
Nope, I don;t feel I have to defend it. The tool does what it does, I
was merely sayiong that some people (such as me) can work with it as it
is. You apparently can't. Fine.
I probably shouldn't get so worked up about it. To be fair, since I tried
the tool I have discovered that Wing IDE also has the option to disable that
behavior and to debug in a mode where unhandled exceptions are just treated
like a normal run would treat them.
Well, yes, but I thought we were focusing ont he deficiencies of the
tool, not the advantages :-)
Plus, IIRC, all you have to do is check an "ignore this exception" box to
have a specific exception ignored everafter, which didn't seem like a huge
deal to me.

It's just too bad I don't have a checkbox for "Ignore marketing spin at this
location".

Indeed. Hope you feel better now.
Dan

regards
Steve
Jul 18 '05 #22

P: n/a
Dan Perl wrote:
Right off the bat, the trial period was only 10 days. I've evaluated quite
a few tools that had a free trial period and I don't think I can remember of
ever having only 10 days before. Then, the GUI is kind of awkward in a few
places (the "project tree view", or something like that, and the toolbars).


In fairness, the initial period is 10 days but it can be renewed at the
end of that for another 10 days. The renewal procedure is done online
and takes about 10 seconds. I also *think* that during the beta period
you can repeatedly renew anyway.

I agree with your comments about the GUI - it's just so busy but I feel
that with all the alternatives. Apart from Emacs of course ;-).

Ian
Jul 18 '05 #23

P: n/a
On Mon, 27 Sep 2004 12:04:47 -0400, Dan Perl <da*****@rogers.com> wrote:

I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.

It beats Komodo 3.01 in some aspect:
- it shines when it comes to auto-completion. No need to feed some
"intelligent database" for each and every file you introduce like in
Komodo (first crash while building the database).
- in my case it works seamlessly with wxPython. Komodo crashed constantly.
Auto-completion with wxPython did never work (tried on 2 machines)
- WingIDE is much more robust although beta than Komodo, which crashed
about 5 times during 3h of work (Win2K; about 20 times because of wxPython
are not taken into account). WingIDE does not crash since around patch 3
(we are now at patch 8).
So I would say:

Use Komodo if you:
- are programming as well in other languages that Komodo supports (PHP,
Perl, Python, HTML, ...)
- need built-in cvs access
- can live with an IDE that crashes from time to time

Use Wingware if you:
- need a good "intelligent" auto-completion even for wxPython, pyQt, ...
- need an extensible IDE (you may in future add your own tools easily)
- need an IDE that focusses on Python and that tries to do the best on
this single language
- can live with an IDE that does not look as polished as Komodo

This comparison is not perfect. But you may realize that I turned it down
for it constantly crashing on me (which might be a problem of me and my 2
machines) and the useless auto-completion (constantly feeding the
database; does not work with wxPython). Those 2 points were in my case the
show stopper. Sad, because the idea behind Komodo is just great, the looks
is fantastic... I will give it another chance with 4.0...

Happy wing user,
Marco

Jul 18 '05 #24

P: n/a

"Marco Aschwanden" <PP**********@spammotel.com> wrote in message
news:ma**************************************@pyth on.org...
On Mon, 27 Sep 2004 12:04:47 -0400, Dan Perl <da*****@rogers.com> wrote:

I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.
The 10 day trial period just gave me a bad first impression and the
explanations for the false positives just left me with a bad taste. I gave
up on Wing because of the false positives (their simple existence, even
without the explanations) and the lack of a class browser. I encountered
the false positives first and I almost gave up right there. But it was just
the first day and I thought I should look more. Like you say, I wanted to
be fair. When I found that the class browser is not supported in the
Personal edition I threw my arms in the air.

To be fair again to Wing, I gave up on it based on the false positives
before finding out that I could have switched to another debug mode. But
that's their fault too because the widget that came up with the false
positive just says you can ignore the exception and then there was something
else just praising the tool, but there was no mention of the different debug
modes. I found that option only once I searched the manual. Maybe it's
just me, but I'm not reading the manuals when I'm trying out several tools.

My "nagging" on the false positives is mostly on the spin that Wingware puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a beef
with it.

I'm curious to know what disturbed you in my comments.
It beats Komodo 3.01 in some aspect:
- it shines when it comes to auto-completion. No need to feed some
"intelligent database" for each and every file you introduce like in
Komodo (first crash while building the database).
- in my case it works seamlessly with wxPython. Komodo crashed constantly.
Auto-completion with wxPython did never work (tried on 2 machines)
- WingIDE is much more robust although beta than Komodo, which crashed
about 5 times during 3h of work (Win2K; about 20 times because of wxPython
are not taken into account). WingIDE does not crash since around patch 3
(we are now at patch 8).
So I would say:

Use Komodo if you:
- are programming as well in other languages that Komodo supports (PHP,
Perl, Python, HTML, ...)
- need built-in cvs access
- can live with an IDE that crashes from time to time

Use Wingware if you:
- need a good "intelligent" auto-completion even for wxPython, pyQt, ...
- need an extensible IDE (you may in future add your own tools easily)
- need an IDE that focusses on Python and that tries to do the best on
this single language
- can live with an IDE that does not look as polished as Komodo

This comparison is not perfect. But you may realize that I turned it down
for it constantly crashing on me (which might be a problem of me and my 2
machines) and the useless auto-completion (constantly feeding the
database; does not work with wxPython). Those 2 points were in my case the
show stopper. Sad, because the idea behind Komodo is just great, the looks
is fantastic... I will give it another chance with 4.0...
I used Komodo for about a month and it never crashed. As a matter of fact,
when the license expired, I let it run for several days and then I just had
to reboot my machine for other reasons (I am using Win XP, BTW). And I have
used it with wxPython already, although probably not as extensively as you
have.

I actually prefer the auto-completion in Komodo. Yes, it doesn't always
work because sometimes it has to learn first, but on the other hand I felt
that Wing tries to do too much. I don't need auto-completion for every
little name that I'm using and Wing's auto-completion was rather annoying me
even after I adjusted the pause period. I didn't find any option to disable
auto-completion selectively for things like locals. I also didn't see a
function usage showing up in Wing the way I am getting it in Komodo (and I
want that), but maybe that was a preference that I had to turn on and I
didn't know about it. I later saw something similar in a screen shot but
maybe that's also not in Wing's Personal edition.

So your 2 show stoppers do not apply for me. As for your other arguments, I
am using Komodo only for Python and not for any other languages and I am
using winCvs for CVS access. Still, during one month of use (more than a
month now), Komodo satisfied all my needs.
Happy wing user,
Marco


Jul 18 '05 #25

P: n/a
"Dan Perl" <da*****@rogers.com> writes:
"Marco Aschwanden" <PP**********@spammotel.com> wrote in message
news:ma**************************************@pyth on.org...
On Mon, 27 Sep 2004 12:04:47 -0400, Dan Perl <da*****@rogers.com> wrote:

I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.
The 10 day trial period just gave me a bad first impression and the
explanations for the false positives just left me with a bad taste. I gave
up on Wing because of the false positives (their simple existence, even
without the explanations) and the lack of a class browser. I encountered
the false positives first and I almost gave up right there. But it was just
the first day and I thought I should look more. Like you say, I wanted to
be fair. When I found that the class browser is not supported in the
Personal edition I threw my arms in the air.


For whatf it's worth, the "false positives" you refer to was a
significant reason that I really liked Wing.

If you ever use any code that runs Python through an extension
boundary (my key use is with wxPython, where all of your GUI objects
and event handlers run from within the wxPython extension), Wing is
one of the few debuggers that catches exceptions in that code. That's
because normally the exceptions are handled by the wxPython extension
module, and then suppressed so that they don't kill the main event
loop.

In my experience, there are minimal other false positives that a
normal application triggers (a consistent one I recall was in the sre
code), although the current 2.0 beta may still be getting tuned with
respect to its default "known" list, and I was more than willing to
mark those to be ignored in order to be able to be sure of catching
any exceptions within my own code, regardless of whether it was
triggered through an extension or not.
(...)
My "nagging" on the false positives is mostly on the spin that Wingware puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a beef
with it.


The actually good for you part is arguably true, since otherwise you
have the potential to be generating exceptions in your own code and
not even know it. But I agree that different people will react
differently (note how differently you and I reacted to this feature),
so the more that can be done to explain the feature (and its
configurability) the better.

-- David
Jul 18 '05 #26

P: n/a

"David Bolen" <db**@fitlinxx.com> wrote in message
news:ue***********@fitlinxx.com...
"Dan Perl" <da*****@rogers.com> writes:
"Marco Aschwanden" <PP**********@spammotel.com> wrote in message
news:ma**************************************@pyth on.org...
> On Mon, 27 Sep 2004 12:04:47 -0400, Dan Perl <da*****@rogers.com>
> wrote:
>
> I myself felt disturbed by your comments on WingIDE. But yes, I do
> understand your nagging on "false exceptions" which can be turned off.
> You
> turned it down because of this and because the short trial period
> (which
> can be prolonged with 1 button click)... well, it didn't seem fair
> towards
> the tool. It has its shortcomings but the ones you mention are
> ridiculous.
The 10 day trial period just gave me a bad first impression and the
explanations for the false positives just left me with a bad taste. I
gave
up on Wing because of the false positives (their simple existence, even
without the explanations) and the lack of a class browser. I encountered
the false positives first and I almost gave up right there. But it was
just
the first day and I thought I should look more. Like you say, I wanted
to
be fair. When I found that the class browser is not supported in the
Personal edition I threw my arms in the air.


For whatf it's worth, the "false positives" you refer to was a
significant reason that I really liked Wing.

If you ever use any code that runs Python through an extension
boundary (my key use is with wxPython, where all of your GUI objects
and event handlers run from within the wxPython extension), Wing is
one of the few debuggers that catches exceptions in that code. That's
because normally the exceptions are handled by the wxPython extension
module, and then suppressed so that they don't kill the main event
loop.


I'm new to wxPython so I cannot relate yet to what you are saying. The
wxPython extension module may catch the exceptions and suppress them, but
doesn't it generate a message for that or something? I hope that they are
not hiding the exception completely (and that they generate something) and
then you can still debug the problem with well placed breakpoints. Am I
wrong here or is there something I'm missing?
In my experience, there are minimal other false positives that a
normal application triggers (a consistent one I recall was in the sre
code), although the current 2.0 beta may still be getting tuned with
respect to its default "known" list, and I was more than willing to
mark those to be ignored in order to be able to be sure of catching
any exceptions within my own code, regardless of whether it was
triggered through an extension or not.
The false positive that I encountered was indeed in 'sre', invoked by
pickle. I think it was an IndexError which should be a normal thing if they
are using it to stop at the end of an iteration. So I'm not convinced this
would constitute "minimal" false positives.

Maybe if I would have been convinced first with a personal experience about
the advantages of Wing's exception detection at the point of raise, then I
would have accepted the false positives more easily. But seeing the false
exception first (actually right away and with positive spins on it) and then
finding out the right technical explanation later, it made it much harder to
swallow.

Here is my suggestion to Wingware. Make the "Never Stop" exception
reporting mode the default in the Personal edition or at least for the trial
download. A tutorial could then lead to the "Stop on Unhandled" mode.
(...)
My "nagging" on the false positives is mostly on the spin that Wingware
puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful
feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a
beef
with it.


The actually good for you part is arguably true, since otherwise you
have the potential to be generating exceptions in your own code and
not even know it. But I agree that different people will react
differently (note how differently you and I reacted to this feature),
so the more that can be done to explain the feature (and its
configurability) the better.


I would need to first see an example of generating an exception without
knowing it and that exception still being relevant although it is being
handled. I can imagine something not working right and an exception being
one more indication for the problem, but then there are still other good
ways to debug the problem. And if things are working, I don't see how
hidden exceptions may be of interest. I have very little experience with
GUIs but I realize that unit testing is much more difficult for GUIs. Does
that make it necessary to "hunt for exceptions" like that instead of testing
for results correctness? It may also be just my short experience with
Python and especially my total lack of experience with complex systems in
Python (e.g, Zope), so I'm ready to listen to an argument on that point.

Dan
-- David

Jul 18 '05 #27

P: n/a
"Dan Perl" <da*****@rogers.com> writes:
I'm new to wxPython so I cannot relate yet to what you are saying. The
wxPython extension module may catch the exceptions and suppress them, but
doesn't it generate a message for that or something? I hope that they are
not hiding the exception completely (and that they generate something) and
then you can still debug the problem with well placed breakpoints. Am I
wrong here or is there something I'm missing?
Yes, the wxPython layer will generally output an exception, but only
if there's a console available to receive it.

Even if you do have a console showing exceptions, not all exceptions
are cleanly reproduceable or predictable. I may have code that
typically runs just great, but that occasionally generates an
exception. Breakpoints may not work if the code runs many times
successfully before failing (due to varieties in calling parameters at
runtime). Perhaps a watchpoint might work but you'd have to know
enough about the failure to establish the watchpoint parameters.

Instead, with Wing, just start it up, let it run (or if you need to,
run it independently of Wing but with the stub loaded) until the
exception occurs. No hassles or up front work.
The false positive that I encountered was indeed in 'sre', invoked by
pickle. I think it was an IndexError which should be a normal thing if they
are using it to stop at the end of an iteration. So I'm not convinced this
would constitute "minimal" false positives.
Did you get any others? I was talking "minimal" in terms of volume,
not necessarily significance (which may be more subjective). I didn't
have much of an issue choosing to ignore the sre case (after which it
never showed up again for my project).
Maybe if I would have been convinced first with a personal experience about
the advantages of Wing's exception detection at the point of raise, then I
would have accepted the false positives more easily. But seeing the false
exception first (actually right away and with positive spins on it) and then
finding out the right technical explanation later, it made it much harder to
swallow.
That could well be. When I was testing Wing originally being able to
effectively work with legacy wxPython apps was a high priority for me,
both in terms of the auto-completion and in terms of exception
handling (the latter of which I had been disappointed to find not
working in other IDEs I had tried to that point). So I was probably
predisposed to not worry too much about the side-effects of
functionality that at least got me the exception processing I wanted.

(...) I would need to first see an example of generating an exception without
knowing it and that exception still being relevant although it is being
handled.
I'd split that into two parts - (a) generating the exception without
knowing it, and (b) it being relevant even if handled.

For (a), I can't seem to come up with a good concise example, so maybe
I'll just agree that you may know about, but not be able to act on it
in a useful fashion. (The cases I was thinking of were GUI apps
without a console, or apps generating output where a traceback might
be missed, but I can come up with reasonable solutions in those cases
if you knew you were looking for an exception, so I'm willing to bet
you can too. The other case was with a locally developed extension, but
that's susceptible to the argument of just fixing the local extension :-))

Note however, that knowing about it in many cases may only be through
a stderr traceback, so to know it other than manually inspecting it at
runtime would require trapping stderr in general and logging that
somewhere, and then having something to recognize tracebacks in that
log.

(b) is certainly possible though, although at the risk of overuse I'll
fall back to wxPython again. In wxPython any exception that occurs in
an event handler will be handled by the wxPython extension, and is
thus invisible to top level code. So if you have an exception in an
event handler, it interrupts the exception handler, which presumably
no longer does what it is supposed to, so that's relevant. The
exception will not, however, stop the application, nor bubble up to
any top level application exception handler. The only place you'd see
an indication of it would be in the traceback to stderr - your
application has no opportunity to handle it otherwise (unless you have
exception handlers in each individual event handler).

Now in normal operation that's actually a very attractive part of
wxPython. By and large you can create problems, and the application
keeps running. Perhaps one element of the GUI might not react
properly or some button won't work, but the application survives. But
when trying to troubleshoot that problem, it's often most convenient
to interact with the application as a user until the problem occurs,
letting the debugger simply catch things at the point when it happens.
I can imagine something not working right and an exception being
one more indication for the problem, but then there are still other good
ways to debug the problem.
I'd certainly agree with that. As with everything, the debugger is
just one element in the toolkit. But even if you have an exception
being generated, there is real value to being able to stop and
dynamically inspect the state of the system at the point of exception,
particularly when an exception is sporadic or difficult to reproduce.

Having other methods to also debug it doesn't negate the value of having
a debugger that can run an application until such an event takes place and
then give you direct access to the state of the system at that point.
And if things are working, I don't see how
hidden exceptions may be of interest. I have very little experience with
GUIs but I realize that unit testing is much more difficult for GUIs. Does
that make it necessary to "hunt for exceptions" like that instead of testing
for results correctness? It may also be just my short experience with
Python and especially my total lack of experience with complex systems in
Python (e.g, Zope), so I'm ready to listen to an argument on that point.


I definitely have multiple categories of code. New projects that can
make use of TDD approaches do so and have far less of a requirement
(IMO so far) of debuggers in general. They're generally also more
amenable to "print" debugging since issues arise that tend to be more
focused, and occurring in well identified areas of the code under well
identified conditions.

Older projects not written in a TDD fashion, or those for which unit
testing is difficult (in my environment typically GUI, threading, and
network in some mixture) often run into scenarios where you need a
lengthy runtime or real world usage to tickle sporadic or
non-deterministic bugs. In those cases, a solid debugger with the
ability to stop at any moment when a problem arises can be worth its
weight in gold - even if such occurrances may be few and far between.
I believe Alex Martelli recently posted about such a case in a web
application, for example, where a debugger ended up very useful in
addition to the other techniques being applied.

-- David
Jul 18 '05 #28

P: n/a
"Dan Perl" <da*****@rogers.com> writes:
"Marco Aschwanden" <PP**********@spammotel.com> wrote in message
news:ma**************************************@pyth on.org...
On Mon, 27 Sep 2004 12:04:47 -0400, Dan Perl <da*****@rogers.com> wrote:

I myself felt disturbed by your comments on WingIDE. But yes, I do
understand your nagging on "false exceptions" which can be turned off. You
turned it down because of this and because the short trial period (which
can be prolonged with 1 button click)... well, it didn't seem fair towards
the tool. It has its shortcomings but the ones you mention are ridiculous.
[...] To be fair again to Wing, I gave up on it based on the false positives
before finding out that I could have switched to another debug mode. But
that's their fault too because the widget that came up with the false
positive just says you can ignore the exception and then there was something
else just praising the tool, but there was no mention of the different debug
modes. I found that option only once I searched the manual. Maybe it's
just me, but I'm not reading the manuals when I'm trying out several tools.

My "nagging" on the false positives is mostly on the spin that Wingware puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a beef
with it.


Dan, I suspect that (though perhaps their marketing may have room for
improvement here ;-) there may have been an unstated assumption on the
part of the Wing IDE people that, "obviously", the false-positives
issue is only a "nit" if one has a hard debugging problem on one's
hands, and that "obviously", one turns this feature off under all
other circumstances. Not necessarily obvious at all, of course.

Perhaps, like many of the rest of us, they only wheel out their
debugger when they hit a really *nasty* problem, so the exception
issue simply never arises as anything other than a minor wart in what
I can see could be a very handy feature. That would explain why this
feature is on by default. Another explanation is that it's turned on
by default just to advertise the feature -- if you don't bump into it
like this, you might never realise it existed.
John
Jul 18 '05 #29

P: n/a

"David Bolen" <db**@fitlinxx.com> wrote in message
news:u6***********@fitlinxx.com...
...................
Even if you do have a console showing exceptions, not all exceptions
are cleanly reproduceable or predictable. I may have code that
typically runs just great, but that occasionally generates an
exception. Breakpoints may not work if the code runs many times
successfully before failing (due to varieties in calling parameters at
runtime). Perhaps a watchpoint might work but you'd have to know
enough about the failure to establish the watchpoint parameters.

Instead, with Wing, just start it up, let it run (or if you need to,
run it independently of Wing but with the stub loaded) until the
exception occurs. No hassles or up front work.
Okay, I can see this point. The idea of the loaded stub sounds especially
interesting because you can just run the application normally, independent
of an IDE. In my experience with sporadic problems like the ones you are
describing, debuggers are usually not much help, except after the fact, for
instance debugging a core of a C/C++ program. If the problem is caused by a
race condition, a debugger can be useless because it affects the scheduling
of threads. How does the loaded stub work in cases like that? Does it
affect threads in any way?
The false positive that I encountered was indeed in 'sre', invoked by
pickle. I think it was an IndexError which should be a normal thing if
they
are using it to stop at the end of an iteration. So I'm not convinced
this
would constitute "minimal" false positives.


Did you get any others? I was talking "minimal" in terms of volume,
not necessarily significance (which may be more subjective). I didn't
have much of an issue choosing to ignore the sre case (after which it
never showed up again for my project).


It was the only one. My point was that even just this particular exception
should happen very often. So even volume, not only significance, is
relative. I, for one, was not comfortable ignoring an exception, especially
without a very clear and detailed explanation on why the false positive
happened in the first place and what happens with the exception if I ignore
it.
(...)
I would need to first see an example of generating an exception without
knowing it and that exception still being relevant although it is being
handled.
I'd split that into two parts - (a) generating the exception without
knowing it, and (b) it being relevant even if handled.

For (a), I can't seem to come up with a good concise example, so maybe
I'll just agree that you may know about, but not be able to act on it
in a useful fashion. (The cases I was thinking of were GUI apps
without a console, or apps generating output where a traceback might
be missed, but I can come up with reasonable solutions in those cases
if you knew you were looking for an exception, so I'm willing to bet
you can too. The other case was with a locally developed extension, but
that's susceptible to the argument of just fixing the local extension :-))

Note however, that knowing about it in many cases may only be through
a stderr traceback, so to know it other than manually inspecting it at
runtime would require trapping stderr in general and logging that
somewhere, and then having something to recognize tracebacks in that
log.

(b) is certainly possible though, although at the risk of overuse I'll
fall back to wxPython again. In wxPython any exception that occurs in
an event handler will be handled by the wxPython extension, and is
thus invisible to top level code. So if you have an exception in an
event handler, it interrupts the exception handler, which presumably
no longer does what it is supposed to, so that's relevant. The
exception will not, however, stop the application, nor bubble up to
any top level application exception handler. The only place you'd see
an indication of it would be in the traceback to stderr - your
application has no opportunity to handle it otherwise (unless you have
exception handlers in each individual event handler).

Now in normal operation that's actually a very attractive part of
wxPython. By and large you can create problems, and the application
keeps running. Perhaps one element of the GUI might not react
properly or some button won't work, but the application survives. But
when trying to troubleshoot that problem, it's often most convenient
to interact with the application as a user until the problem occurs,
letting the debugger simply catch things at the point when it happens.


You're making a very good case, assuming that the exception is at the root
of the problem. However, it's hard to say how many such problems are caused
by an exception. Even if there is an exception, the root problem may be way
before that and you still have to go and do some normal debugging. So many
times I've had memory corruption crashing an application in C++ and I had to
look back at a method that was called WAY BEFORE the crash because that was
usually corrupting memory at exit.

So, in your example of a button, my general debugging approach would still
be to put a breakpoint in the event handler of the button, because the root
of the problem can be so many different things.

In your experience with Python, what percentage of problems manifested
themselves with exceptions (especially right at the root of the problem) as
opposed to simply flawed logic that gets through without any exceptions?

I think we are actually very much agreeing. The main difference between us
is how much weight we put in debugging exceptions versus debugging all kinds
of problems and how much we are willing to put up with in exchange for that
precious advantage in a few cases.

You are saying a solid debugger "can be worth its weight in gold". How much
does Wing IDE weigh? ;-)

Dan
-- David

Jul 18 '05 #30

P: n/a

"John J. Lee" <jj*@pobox.com> wrote in message
news:87************@pobox.com...
"Dan Perl" <da*****@rogers.com> writes:
My "nagging" on the false positives is mostly on the spin that Wingware
puts
in their explanations. It's one thing to explain it the way that Stephan
did, that it was better to offer something imperfect if that was also
providing some important advantages and then also offer the other
alternative ("We weighed having a necessarily imperfect but useful
feature
with not having the feature at all."). And it's another thing to insist
that this is not a problem and it is actually good for you. This doesn't
seem to bother other people as much as it bothered me, but I do have a
beef
with it.
Dan, I suspect that (though perhaps their marketing may have room for
improvement here ;-) there may have been an unstated assumption on the
part of the Wing IDE people that, "obviously", the false-positives
issue is only a "nit" if one has a hard debugging problem on one's
hands, and that "obviously", one turns this feature off under all
other circumstances. Not necessarily obvious at all, of course.

Perhaps, like many of the rest of us, they only wheel out their
debugger when they hit a really *nasty* problem, so the exception
issue simply never arises as anything other than a minor wart in what
I can see could be a very handy feature. That would explain why this
feature is on by default. Another explanation is that it's turned on
by default just to advertise the feature -- if you don't bump into it
like this, you might never realise it existed.


If Stephan is still reading this thread he may contradict this, but my
suspicion is that your last explanation is the real one. Such an approach
can also backfire like it did in my case.
John

Jul 18 '05 #31

P: n/a
"Dan Perl" <da*****@rogers.com> writes:
(...) In my experience with sporadic problems like the ones you are
describing, debuggers are usually not much help, except after the fact, for
instance debugging a core of a C/C++ program. If the problem is caused by a
race condition, a debugger can be useless because it affects the scheduling
of threads. How does the loaded stub work in cases like that? Does it
affect threads in any way?
Yes, once the debugger is actually activated (you start up the IDE and
it attaches) it's like a normal Python debugger, so it uses the trace
hook and will have some impact on runtime. You can finesse this a little by
controlling where you import the stub and/or when you connect to the process.

Of course, in my experience just about anything you do to try to work
with a subtle threading or race condition bug is just as likely to
affect the problem (including adding a new line of code), so you can
still hope that the problem will simply exhibit itself as well - even
if slightly differently - with the debugger attached. Obviously, no
guarantees.

(the "sre" exception) It was the only one. My point was that even just this particular exception
should happen very often. So even volume, not only significance, is
relative. I, for one, was not comfortable ignoring an exception, especially
without a very clear and detailed explanation on why the false positive
happened in the first place and what happens with the exception if I ignore
it.
I don't really disagree - it would probably be good if this was a
well-defined one that Wing came configured to ignore.
(...)
I would need to first see an example of generating an exception without
knowing it and that exception still being relevant although it is being
handled.


I'd split that into two parts - (a) generating the exception without
knowing it, and (b) it being relevant even if handled.

For (a), I can't seem to come up with a good concise example, so maybe
I'll just agree that you may know about, but not be able to act on it (...) Note however, that knowing about it in many cases may only be through
a stderr traceback, so to know it other than manually inspecting it at
runtime would require trapping stderr in general and logging that
somewhere, and then having something to recognize tracebacks in that
log.

(b) is certainly possible though, although at the risk of overuse I'll
fall back to wxPython again. In wxPython any exception that occurs in
an event handler will be handled by the wxPython extension, and is
thus invisible to top level code.

(...)
You're making a very good case, assuming that the exception is at the root
of the problem. However, it's hard to say how many such problems are caused
by an exception. Even if there is an exception, the root problem may be way
before that and you still have to go and do some normal debugging. So many
times I've had memory corruption crashing an application in C++ and I had to
look back at a method that was called WAY BEFORE the crash because that was
usually corrupting memory at exit.
Yep, although if you're lucky the form of the corruption at the point
of error can help point the way. Also, since pure memory corruption
tends to be a rare occurrence in Python (buggy extension modules
notwithstanding), even if an exception is due to malformed code
earlier on, the actual memory state of the process is generally in
good enough shape to go poking around reliably in all your data
structures once the exception triggers.
So, in your example of a button, my general debugging approach would still
be to put a breakpoint in the event handler of the button, because the root
of the problem can be so many different things.
Sure, but if the button works sometimes and not others, it's just more
of a pain to hit the breakpoint over and over again until it happens
to get to a state where the problem would occur.
In your experience with Python, what percentage of problems manifested
themselves with exceptions (especially right at the root of the problem) as
opposed to simply flawed logic that gets through without any exceptions?
I'd have to separate by older and newer code again. The newer code
following a TDD pattern definitely has different characteristics than
older legacy code written using more traditional methods. The newer
stuff is much less likely to generate either kind of problem except
during active development at which point the current test generally
points right to the area of potential problem.

But I'd have to say that at least for me, for more traditional
methods, I do think there's a higher percentage of problems leading to
exceptions than to more subtle failures. I'm not entirely sure why
except that I expect bugs that in other environments might lead to
subtle mistakes (or harder to track problems like eventual crashes
from prior memory corruption) quickly end up in some state where you
are trying to interpret any object incorrectly in your code and yield
an exception. For example, a wrong pathway not constructing an
appropriate object type or contents which is then passed into another
function anticipating it. In effect, the "duck typing" approach that
is commonly used can yield exceptions in pretty quick order if you
start messing up the objects involved in your system.

Note however that I'm not saying all those exceptions require a
debugger or anything more than logging them and/or observing them
during development.
I think we are actually very much agreeing. The main difference between us
is how much weight we put in debugging exceptions versus debugging all kinds
of problems and how much we are willing to put up with in exchange for that
precious advantage in a few cases.
Yes (to the agreement). And don't let me over-emphasize things, as
I'm not necessarily suggesting that the "wait for the exception" is
extremely important. It is, however, useful enough to me that I
consider supporting it to be a "pro" feature of any Python debugger,
and at least for me, yes worth some consequences such as the
occasional false positive.
You are saying a solid debugger "can be worth its weight in gold". How much
does Wing IDE weigh? ;-)


Heh - good question. 'Course, with gold where it is at the moment, it
probably wouldn't have to weigh too much anyway :-)

-- David
Jul 18 '05 #32

P: n/a

"David Bolen" <db**@fitlinxx.com> wrote in message
news:u7***********@fitlinxx.com...
"Dan Perl" <da*****@rogers.com> writes:
(...) In my experience with sporadic problems like the ones you are
describing, debuggers are usually not much help, except after the fact,
for
instance debugging a core of a C/C++ program. If the problem is caused
by a
race condition, a debugger can be useless because it affects the
scheduling
of threads. How does the loaded stub work in cases like that? Does it
affect threads in any way?


Yes, once the debugger is actually activated (you start up the IDE and
it attaches) it's like a normal Python debugger, so it uses the trace
hook and will have some impact on runtime. You can finesse this a little
by
controlling where you import the stub and/or when you connect to the
process.


Interesting. So it's actively attached to the process, not just listening
to it. I would like to find out more about how this debug stub works. The
Wing IDE reference manual doesn't get into details like that, but I can see
that one can also set breakpoints remotely. That should mean that the stub
does quite a bit of work.

Since my previous posting on this thread I have discovered an interesting
utility for remote reporting of exceptions, although I couldn't try it:
http://egofile.com/cgi-bin/blosxom.cgi. My understanding is that it sends
an HTTP POST message. That should mean that it's much less powerful than
the Wing IDE stub but it should be also less intrusive.

And I also found win32traceutil
(http://www.python.org/windows/win32/#win32trace), which has quite a similar
use. Of course, it's only for Windows, and I'm not sure whether it works
with the listener and the debugged process on different nodes. But this is
something I may personally use.

Any experience with these tools? Especially with win32traceutil, because
that's something I may use.

Dan
Jul 18 '05 #33

P: n/a
"Dan Perl" <da*****@rogers.com> writes:
Interesting. So it's actively attached to the process, not just listening
to it. I would like to find out more about how this debug stub works. The
Wing IDE reference manual doesn't get into details like that, but I can see
that one can also set breakpoints remotely. That should mean that the stub
does quite a bit of work.
It's been a while, but if I recall the stub operates like a network
client, just connecting back to the IDE. It tries to connect when
imported. If it can't connect it sits around (in a background thread
I think) periodically trying - that shouldn't be much overhead. But
when connected, it invokes the normal wing debugging code (e.g., the
same stuff used when debugging an app locally) which I'm assuming ties
into sys.settrace or equivalent and uses the network link to exchange
the information with the IDE. I'm assuming sys.settrace or equivalent
since once connected you can explicitly interrupt the debuggee from
the IDE at any time.
Since my previous posting on this thread I have discovered an interesting
utility for remote reporting of exceptions, although I couldn't try it:
http://egofile.com/cgi-bin/blosxom.cgi. My understanding is that it sends
an HTTP POST message. That should mean that it's much less powerful than
the Wing IDE stub but it should be also less intrusive.
Depends on how it traps the exceptions. If it's using something like
sys.excepthook, then yes, it should definitely be less intrusive (but
also unable to catch anything but unhandled exceptions). But I would
expect any tool that is actively watching code at a finer granularity
is going to carry similar overhead to a debugger since they probably
all use the existing trace hooks in Python.
And I also found win32traceutil
(http://www.python.org/windows/win32/#win32trace), which has quite a similar
use. Of course, it's only for Windows, and I'm not sure whether it works
with the listener and the debugged process on different nodes. But this is
something I may personally use.


I'm pretty sure this is local only. You can get the source from the
PyWin32 package. Last time I looked it used a memory mapped file (in
swap space) to hold data pending someone reading from it. So it's
basically a simple inter-process pipe. Importing it routes all
stdout/stderr to the trace file so yes it would be a way to catch
unhandled exceptions at least for logging purposes.

It's useful (particularly for when you want to get output from
background stuff like COM servers or GUI applications that for one
reason or another can't have a console during debugging), but strictly
for providing an alternate stdout/stderr stream.

-- David
Jul 18 '05 #34

This discussion thread is closed

Replies have been disabled for this discussion.