IDLE history, Python IDE, and Interactive Python with Vim

[Ashot]

This is sort of both Python and Vim related (which is why I've posted to
both newsgroups).

Python related:
----------------------
I have been frustrated for quite some time with a lack of a history
command in IDLE (in fact with IDLE in general). Often I'll develop new
code at the command line, testing each line as I go. Currently I have to
copy and paste, removing outputs and the ">>>" at each line.
Is it perhaps possible to make some kind of hack to do this (dump a
command history)?

Idle in general isn't that great IMO, so I was wondering also if there are
better alternatives out there? What do people use mostly? I've tried
something called pyCrust, but this too didn't have history and some other
things I was looking for. On a more general note, although the agility
and simplicity of Python make programming tools like an IDE less
necessary, it still seems that Python is lacking in this departement. The
PyDev plug-in for Eclipse seems like good step in this direction, although
I haven't tried it yet. Does anyone have any experience with this, or
perhaps can point me to other tools.

Vim related:
----------------------
Ideally, it would be nice to have a command mapped to a keystroke that can
append the last executed command to a file. Even better would be a system
that would integrate the file editing and interactive command line tool
more seamlessly. Something along the lines of a debugger + file editor
+ command line utility, where file editor = vim. I know that vim has a
utility for running python commands from its command prompt, but I have
had a hard time getting this to work in windows and haven't explored it.
Has anyone seen/tried a system along these lines, perhaps incorporating
the python debugger (pdb)? I can see something that will run the file you
are editing in vim up to the cursor or a mark with a set_trace at the line
you are editing.
Any info is appreciated, thanks.

--
Ashot Petrosian
University of Texas at Austin, Computer Sciences

[Daniel Bickett]

This is certainly a worthy topic. There are several IDEs for Python
(one I like very much being Komodo) that have plenty of fancy
debugging features and advanced operations, however I have yet to
encounter (elsewhere) the convenience that comes with being able to
press F5 and have an interactive interpreter load my code and be ready
for testing.

That said, that is my only reason for my ever using IDLE. Without it I
would probably forget that IDLE exists, were it not for the obnoxious
context menu entry. ;)

--
Daniel Bickett
dbickett at gmail.com
http://heureusement.org/

[Stephen Waterbury]

Daniel Bickett wrote:

This is certainly a worthy topic. There are several IDEs for Python
(one I like very much being Komodo) that have plenty of fancy
debugging features and advanced operations, however I have yet to
encounter (elsewhere) the convenience that comes with being able to
press F5 and have an interactive interpreter load my code and be ready
for testing.

*Sigh*, I too am a vi kinda guy, and I keep badgering the WingIDE
gang to give Wing a vi mode ... :) They keep saying "it's coming",
but it's been a while ... and in fairness to them, I don't think
there is exactly a huge horde of vi barbarians at their gate!
Probably a pretty tiny horde, and not very noisy. ;)

Steve

[Ashot]

WingIDE looks nice though, and it seems that the interactive shell is
built in from the screenshots. Can you run a program to a certain point
break point and use the shell?

On Wed, 02 Feb 2005 23:40:40 -0500, Stephen Waterbury <go***@comcast.net>
wrote:

Daniel Bickett wrote:

This is certainly a worthy topic. There are several IDEs for Python
(one I like very much being Komodo) that have plenty of fancy
debugging features and advanced operations, however I have yet to
encounter (elsewhere) the convenience that comes with being able to
press F5 and have an interactive interpreter load my code and be ready
for testing.

*Sigh*, I too am a vi kinda guy, and I keep badgering the WingIDE
gang to give Wing a vi mode ... :) They keep saying "it's coming",
but it's been a while ... and in fairness to them, I don't think
there is exactly a huge horde of vi barbarians at their gate!
Probably a pretty tiny horde, and not very noisy. ;)

Steve

--
==============================
Ashot Petrosian
University of Texas at Austin, Computer Sciences
(views expressed are solely my own)
==============================

[Fuzzyman]

If you use IPython for your interactive mode stuff, you'll have a nice
history...
Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml

[Pierre Barbier de Reuille]

Fuzzyman a écrit :

If you use IPython for your interactive mode stuff, you'll have a nice
history...
Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml

Best event : if your "EDITOR" system variable in "vim", using the "ed"
command in ipython will bring "vim" with (eventually) the code you want
to edit :)

Now, I wonder if you could embed ipython inside vim ...

Pierre

[Bruno Desthuilliers]

Daniel Bickett a écrit :

This is certainly a worthy topic. There are several IDEs for Python
(one I like very much being Komodo) that have plenty of fancy
debugging features and advanced operations, however I have yet to
encounter (elsewhere) the convenience that comes with being able to
press F5 and have an interactive interpreter load my code and be ready
for testing.

Try emacs + python-mode.

[Bruno Desthuilliers]

Pierre Barbier de Reuille a écrit :

Fuzzyman a écrit :

If you use IPython for your interactive mode stuff, you'll have a nice
history...
Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml

Best event : if your "EDITOR" system variable in "vim", using the "ed"
command in ipython will bring "vim" with (eventually) the code you want
to edit :)

Now, I wonder if you could embed ipython inside vim ...

<holy-war>
Don't know, but you can embed ipython inside emacs !-)
</holy-war>
Pierre

[http://www.stani.be]

I think SPE has exactly what you need. Next to the shell there is the
session, which has only the commands typed on the interactive prompt.

Stani

http://spe.pycs.net

Ashot wrote:

This is sort of both Python and Vim related (which is why I've posted to both newsgroups).

Python related:
----------------------
I have been frustrated for quite some time with a lack of a history
command in IDLE (in fact with IDLE in general). Often I'll develop new code at the command line, testing each line as I go. Currently I have to copy and paste, removing outputs and the ">>>" at each line.
Is it perhaps possible to make some kind of hack to do this (dump a
command history)?

Idle in general isn't that great IMO, so I was wondering also if there are better alternatives out there? What do people use mostly? I've tried something called pyCrust, but this too didn't have history and some other things I was looking for. On a more general note, although the agility and simplicity of Python make programming tools like an IDE less
necessary, it still seems that Python is lacking in this departement. The PyDev plug-in for Eclipse seems like good step in this direction, although I haven't tried it yet. Does anyone have any experience with this, or perhaps can point me to other tools.

Vim related:
----------------------
Ideally, it would be nice to have a command mapped to a keystroke that can append the last executed command to a file. Even better would be a system that would integrate the file editing and interactive command line tool more seamlessly. Something along the lines of a debugger + file editor + command line utility, where file editor = vim. I know that vim has a utility for running python commands from its command prompt, but I have had a hard time getting this to work in windows and haven't explored it. Has anyone seen/tried a system along these lines, perhaps incorporating the python debugger (pdb)? I can see something that will run the file you are editing in vim up to the cursor or a mark with a set_trace at the line you are editing.
Any info is appreciated, thanks.

--
Ashot Petrosian
University of Texas at Austin, Computer Sciences

[Tim Hoffman]

Have a gander at Boa Constructor.

YOu can certainly go to a shell when debugging. (ie set breakpoint)
then the shell is attached to that context.

T

Ashot wrote:

This is sort of both Python and Vim related (which is why I've posted
to both newsgroups).

Python related:
----------------------
I have been frustrated for quite some time with a lack of a history
command in IDLE (in fact with IDLE in general). Often I'll develop new
code at the command line, testing each line as I go. Currently I have
to copy and paste, removing outputs and the ">>>" at each line.
Is it perhaps possible to make some kind of hack to do this (dump a
command history)?

Idle in general isn't that great IMO, so I was wondering also if there
are better alternatives out there? What do people use mostly? I've
tried something called pyCrust, but this too didn't have history and
some other things I was looking for. On a more general note, although
the agility and simplicity of Python make programming tools like an IDE
less necessary, it still seems that Python is lacking in this
departement. The PyDev plug-in for Eclipse seems like good step in
this direction, although I haven't tried it yet. Does anyone have any
experience with this, or perhaps can point me to other tools.

Vim related:
----------------------
Ideally, it would be nice to have a command mapped to a keystroke that
can append the last executed command to a file. Even better would be a
system that would integrate the file editing and interactive command
line tool more seamlessly. Something along the lines of a debugger +
file editor + command line utility, where file editor = vim. I know
that vim has a utility for running python commands from its command
prompt, but I have had a hard time getting this to work in windows and
haven't explored it. Has anyone seen/tried a system along these lines,
perhaps incorporating the python debugger (pdb)? I can see something
that will run the file you are editing in vim up to the cursor or a
mark with a set_trace at the line you are editing.
Any info is appreciated, thanks.

--
Ashot Petrosian
University of Texas at Austin, Computer Sciences

[Pat]

Ashot wrote:

This is sort of both Python and Vim related (which is why I've posted to both newsgroups).

Python related:
----------------------
I have been frustrated for quite some time with a lack of a history
command in IDLE (in fact with IDLE in general). Often I'll develop new code at the command line, testing each line as I go. Currently I have to copy and paste, removing outputs and the ">>>" at each line.
Is it perhaps possible to make some kind of hack to do this (dump a
command history)?

Idle in general isn't that great IMO, so I was wondering also if there are better alternatives out there? What do people use mostly? I've tried something called pyCrust, but this too didn't have history and some other things I was looking for.

PyCrust is only a shell, not a full-blown IDE, so it likely lacks
things that you were looking for. But it certainly does have history,
multi-line command recall, and cut/paste options with and without the
leading prompts. In fact, the default Copy command (Ctrl+C) strips out
the prompts, and the Copy Plus command (Shift+Ctrl+C) retains the
prompts. If you select the Session tab you'll see the entire command
history, without prompts and without the responses from the Python
interpreter. Here are some other keybindings:

shell.help()
* Key bindings:
Home Go to the beginning of the command or line.
Shift+Home Select to the beginning of the command or line.
Shift+End Select to the end of the line.
End Go to the end of the line.
Ctrl+C Copy selected text, removing prompts.
Ctrl+Shift+C Copy selected text, retaining prompts.
Ctrl+X Cut selected text.
Ctrl+V Paste from clipboard.
Ctrl+Shift+V Paste and run multiple commands from clipboard.
Ctrl+Up Arrow Retrieve Previous History item.
Alt+P Retrieve Previous History item.
Ctrl+Down Arrow Retrieve Next History item.
Alt+N Retrieve Next History item.
Shift+Up Arrow Insert Previous History item.
Shift+Down Arrow Insert Next History item.
F8 Command-completion of History item.
(Type a few characters of a previous command and
press F8.)
Ctrl+Enter Insert new line into multiline command.
Ctrl+] Increase font size.
Ctrl+[ Decrease font size.
Ctrl+= Default font size.

Hope that helps.

Pat

--
Patrick K. O'Brien
Orbtech http://www.orbtech.com
Schevo http://www.schevo.org
Pypersyst http://www.pypersyst.org

[Fernando Perez]

Hi,

Ashot wrote:

This is sort of both Python and Vim related (which is why I've posted to
both newsgroups).

[...]

I know you've been using ipython recently (the readline color bugs), so perhaps
my reply is a bit redundant. Forgive me if that's the case, I just want to
give you some useful info.

Just in case you haven't come across these features (the manual is kind of long
and dry), I should note that ipython has pretty much everything you've asked
for here. %hist -n dumps your history without line numbers (for copy/paste),
%logstart gives you an incremental log (valid python code) of your current
session, %save allows you to save a selected group of lines to a file, and
%edit will open up $EDITOR (or vi in Unix by default) at the source of any
accessible object. With %pdb, you can even trigger automatically pdb at any
uncaught exception (in Emacs, you'll even get the source simultaneously opened,
I'm sure something similar could be done for vi).

Ipython is not an IDE, but it does offer an extensible command-line environment
which tries hard to be very efficient. Hopefully this year I'll be able to
work on integrating it (optionally, since it will never lose its command-line
functionality) with GUIs as well.

Regards,

f

[Ashot]

On Sat, 05 Feb 2005 11:07:40 -0700, Fernando Perez <fp********@gmail.com> wrote:

Hi,

Ashot wrote:

This is sort of both Python and Vim related (which is why I've posted to
both newsgroups).
[...]

I know you've been using ipython recently (the readline color bugs), so perhaps
my reply is a bit redundant. Forgive me if that's the case, I just want to
give you some useful info.

Just in case you haven't come across these features (the manual is kind of long
and dry), I should note that ipython has pretty much everything you've asked
for here. %hist -n dumps your history without line numbers (for copy/paste),
%logstart gives you an incremental log (valid python code) of your current
session, %save allows you to save a selected group of lines to a file, and
%edit will open up $EDITOR (or vi in Unix by default) at the source of any
accessible object. With %pdb, you can even trigger automatically pdb at any
uncaught exception (in Emacs, you'll even get the source simultaneously opened,
I'm sure something similar could be done for vi).

yup, this is why I've been using it, its (almost exactly :) what I was looking for. I had tried it before, but was reluctant to use it because the windows terminal is not very appealing.
Some things I've noticed so far that I think could be improved, some of which are minor but annoying:

edit doesn't work with classes even though it says it should, you have to point it to a function in the class (perhaps only in windows)

under windows vim/gvim doesn't jump to the line of the function.. you said that this should work in vi, but I'm guessing its a Windows thing

**the biggest problem for me is edit forgets code in which there were errors. for example:"ed" type some nonsense, and ed -p doesn't remember. Thats putting an awful lot of pressure on getting the syntax right everytime =] Perhaps I am missing something here?

it would be really great if the code executed on save rather than exit, but I'm not sure if this is possible. This would be very useful for people using an editor not in the console.. that way you wouldn't have to keep opening/closing new editor windows.

this isn't a big deal, but it would be nice if there was an easy way to refer to the entire history (for edit/save/etc)

would be nice if you could send stuff to the clipboard.

There are a few other things I'm probably forgetting, but having said all that, I've come off a little negative here. I really like IPython so far, its got all the things I was looking for, thank you very much for your work. If I have the time in the future I would definately contribute to this project.

..a

Ipython is not an IDE, but it does offer an extensible command-line environment
which tries hard to be very efficient. Hopefully this year I'll be able to
work on integrating it (optionally, since it will never lose its command-line
functionality) with GUIs as well.

Regards,

f

--
==============================
Ashot Petrosian
University of Texas at Austin, Computer Sciences
(views expressed are solely my own)
==============================

[Ashot]

On Mon, 07 Feb 2005 00:55:07 -0600, Ashot <as***@removemolsoftthis.com> wrote:

On Sat, 05 Feb 2005 11:07:40 -0700, Fernando Perez <fp********@gmail.com> wrote:

Hi,

Ashot wrote:

This is sort of both Python and Vim related (which is why I've posted to
both newsgroups).

[...]

I know you've been using ipython recently (the readline color bugs), so perhaps
my reply is a bit redundant. Forgive me if that's the case, I just want to
give you some useful info.

Just in case you haven't come across these features (the manual is kind of long
and dry), I should note that ipython has pretty much everything you've asked
for here. %hist -n dumps your history without line numbers (for copy/paste),
%logstart gives you an incremental log (valid python code) of your current
session, %save allows you to save a selected group of lines to a file, and
%edit will open up $EDITOR (or vi in Unix by default) at the source of any
accessible object. With %pdb, you can even trigger automatically pdb at any
uncaught exception (in Emacs, you'll even get the source simultaneously opened,
I'm sure something similar could be done for vi).

yup, this is why I've been using it, its (almost exactly :) what I was looking for. I had tried it before, but was reluctant to use it because the windows terminal is not very appealing.
Some things I've noticed so far that I think could be improved, some of which are minor but annoying:

edit doesn't work with classes even though it says it should, you have to point it to a function in the class (perhaps only in windows)

under windows vim/gvim doesn't jump to the line of the function.. you said that this should work in vi, but I'm guessing its a Windows thing

**the biggest problem for me is edit forgets code in which there were errors. for example:"ed" type some nonsense, and ed -p doesn't remember. Thats putting an awful lot of pressure on getting the syntax right everytime =] Perhaps I am missing something here?

it would be really great if the code executed on save rather than exit, but I'm not sure if this is possible. This would be very useful for people using an editor not in the console.. that way you wouldn't have to keep opening/closing new editor windows.

this isn't a big deal, but it would be nice if there was an easy way to refer to the entire history (for edit/save/etc)

would be nice if you could send stuff to the clipboard.

Sorry, a few more things I forgot to mention having to do with editing multiline entries in the console:

Autotab setting doesn't seem to have any effect, I have to type "ctrl-o" manually

Is there a setting that treats multiline entries as a single command, as IDLE does? In otherwords, allowing you to edit the entire entry, going up to previous lines, and treating as one when cycling through the history? It seems like ipython recognizes these through the hist command, but not when editing or cycling.

Is it possible to use the "set editing-mode vi" option in ipython since it uses readline? From reading online, it seems like this should be possible, however it doesn't work for me. It works in bash, but once I enter either ipython or just regular python the keybindings don't work anymore.

Again, thanks for your help.

..a

[Fernando Perez]

Ashot wrote:

yup, this is why I've been using it, its (almost exactly :) what I was looking
for. I had tried it before, but was reluctant to use it because the windows
terminal is not very appealing. Some things I've noticed so far that I think
could be improved, some of which are minor but annoying:
I should note that I hardly use %edit myself, it's something that was added at
the request of users, and is an OK facility for quick and dirty work. But
getting such a system to work reliably in all cases, with the
import/execfile/reload issues at play, is really not easy. Especially because
it's not even completely clear what the 'right' behaviour should be. %edit
makes a few educated guesses, but that's it.

My personal coding environment consists of a permanently open editor with
multiple files open (XEmacs in my case), and an ipython session open. In
ipython, I use %run to run test code, and when said top-level test code relies
on modules which are also being changed, I simply put into the top-level
scripts reload() statements. This gives me the ipython tracebacks,
experimentation, tab-exploration of objects, etc, while I have the full power
of Xemacs for the real editing work.

IPython is NOT an IDE, so I think it's important to understand its limitations
to make the best possible use of it. Perhaps I haven't done a very good job of
outlining this to users, I don't know.
edit doesn't work with classes even though it says it should, you have to
point it to a function in the class (perhaps only in windows)

under windows vim/gvim doesn't jump to the line of the function.. you said
that this should work in vi, but I'm guessing its a Windows thing
Works for jed/vim under linux, that's about as much as I know. Probably a win32
thing, as you say.
**the biggest problem for me is edit forgets code in which there were errors.
for example:"ed" type some nonsense, and ed -p doesn't remember. Thats
putting an awful lot of pressure on getting the syntax right everytime =]
Perhaps I am missing something here?
Fixed. Will be in the next release, which will probably come out soon (I found
an unrelated crash case, and I consider crashes release-triggering bugfixes).
it would be really great if the code executed on save rather than exit, but
I'm not sure if this is possible. This would be very useful for people using
an editor not in the console.. that way you wouldn't have to keep
opening/closing new editor windows.
not possible in a cross-platform, cross-editor way.
this isn't a big deal, but it would be nice if there was an easy way to refer
to the entire history (for edit/save/etc)
%hist has options to print as many lines as you want. And the %log* commands
will dump all of your history straight to a file, and continue recording from
then on.
would be nice if you could send stuff to the clipboard.
Do you have a linux/OSX/win32 way to do it? If so, I'll gladly include it.
Under *nix, mouse-highlighting copies to the X11 primary selection, which I can
then paste anywhere with the middle button.
There are a few other things I'm probably forgetting, but having said all
that, I've come off a little negative here. I really like IPython so far, its
got all the things I was looking for, thank you very much for your work. If I
have the time in the future I would definately contribute to this project.

No problem, I appreciate the comments. Fair, honest criticism can only make a
project better. Note, however, that I _strongly_ suggest you post this kind of
stuff on the ipython list. While I monitor c.l.py via gmane, I'm not
subscribed and sometimes I'm too busy to read it for weeks at a time, so
there's a very good chance I'll simply miss ipython-related posts here. The
ipython lists, on the other hand, I keep a close eye on :)

Best,

f

[Fernando Perez]

Ashot wrote:

Sorry, a few more things I forgot to mention having to do with editing
multiline entries in the console:

Autotab setting doesn't seem to have any effect, I have to type "ctrl-o"
manually
I've noticed it doesn't work under win32. It's fine under *nix. There's only
so much Gary can do with his readline supplement in a win32 environment, I'm
not sure if this is something which he could add, you'd have to contact him
directly. This would require support for insertions into the current line,
which may not even be possible in a windows terminal, I just don't know.
Is there a setting that treats multiline entries as a single command, as IDLE
does? In otherwords, allowing you to edit the entire entry, going up to
previous lines, and treating as one when cycling through the history? It
seems like ipython recognizes these through the hist command, but not when
editing or cycling.
This is impossible in a line-oriented terminal program. What you are asking for
requires 2-d cursor control, which can only be provided in a gui environment,
or with curses in a terminal (or with custom terminal handling code).
Basically, it means writing a complete screen-handling program, which is far
more than ipython can do.

I keep trying to start an internal ipython cleanup, so it could be embedded into
a gui environment. Once that happens, it may be possible to use idle or
pycrust but with the ipython engine.

Within the design constraints of not taking over the terminal with a curses (or
similar) environment, restricted to onlw raw_input() communication with the
user, what you are asking is a technical impossibility.
Is it possible to use the "set editing-mode vi" option in ipython since it
uses readline? From reading online, it seems like this should be possible,
however it doesn't work for me. It works in bash, but once I enter either
ipython or just regular python the keybindings don't work anymore.

I don't know if this works even under *nix, since I don't know if the python
readline library wraps this part of GNU readline. I _do_ know that python only
wraps a subset of GNU readline, so I wouldn't be surprised if this wasn't
included. It is not something specific to ipython, you'd have to ask the
readline developers directly.

Best,

f