473,287 Members | 1,555 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

How to get all the variables in a python shell

Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!
Jun 27 '08 #1
15 2133
On 2008-05-29, li********@gmail.com <li********@gmail.comwrote:
Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.
ok, although I personally favor the style of writing and running a
script/program, since it scales much better (you can easier automate steps),
and it is much easier reproducible (something you probably want in scientific
software) and storable (in a VCS).
For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.
I do hope you have made a fair estimate of the amount of work that it costs to
change the value of a variable in this way.

I would propose to simply use the interactive Python prompt. It doesn't give
you popup icons for clicking, but you do get the entire Python interpreter, and
all its libraries for free.

The Python library has a frame work for customizing the interpreter. Have a
look at 'cmd' module.
So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?
Now you have lost me. One window is not enough for interaction?

Obviously, you'll need to have a common interpreter/storage backend. One
solution may be to have a common execution back-end, and for each window a
'frontend' which passes commands entered to the back-end, and echoes results
from the back-end to the terminal.
Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
They eat memory when you keep them in a data base? It seems, you are making
assumptions about implementations here without telling them.
(ie pick a data base that uses a disk, and your problem is solved. Why is that
not an option?)

Sincerely,
Albert

Jun 27 '08 #2
li********@gmail.com wrote:
I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?
I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

TJG

[1] http://ipython.scipy.org/moin/
Jun 27 '08 #3
li********@gmail.com wrote:
I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?
I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

TJG

[1] http://ipython.scipy.org/moin/
Jun 27 '08 #4
-----Original Message-----
From: py******************@python.org [mailto:py******************@python.org] On Behalf Of Tim Golden
Sent: Thursday, May 29, 2008 4:11 AM
To: Python-Win32 List; py*********@python.org
Cc: Python-Win32 List
Subject: Re: [python-win32] How to get all the variables in a python shell

li********@gmail.com wrote:
I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?
I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.

TJG

[1] http://ipython.scipy.org/moin/
_______________________________________________
python-win32 mailing list
py**********@python.org
http://mail.python.org/mailman/listinfo/python-win32

I'd try looking at memcached (http://www.danga.com/memcached/apis.html). No hacking or reimplementation of the interpreter would be necessary, and there's a
Python api available. I haven't used it for anything production related, but I have played with it a bit, and it's fast and stable.

DISCLAIMER:
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and
may contain legally privileged and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and
any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify
me and permanently delete the original and any copy of any e-mail and any printout thereof.
E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept
liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

NOTICE REGARDING PRIVACY AND CONFIDENTIALITY
Direct Edge ECN LLC may, at its discretion, monitor and review the content of all e-mail communications.

www.directedge.com
Jun 27 '08 #5
On 29/05/2008, Dahlstrom, Roger <rd********@directedge.comwrote:
I'd try looking at memcached (http://www.danga.com/memcached/apis.html).
No hacking or reimplementation of the interpreter would be necessary, and
there's a Python api available. I haven't used it for anything production related,
but I have played with it a bit, and it's fast and stable.
Is memcached available for Windows, then? I've heard nice things about
it, but thought it was Unix-only.

Paul.
Jun 27 '08 #6
-----Original Message-----
From: Paul Moore [mailto:p.*******@gmail.com]
Sent: Thursday, May 29, 2008 7:23 AM
To: Dahlstrom, Roger
Cc: Python-Win32 List; py*********@python.org
Subject: Re: [python-win32] How to get all the variables in a python shell

On 29/05/2008, Dahlstrom, Roger <rd********@directedge.comwrote:
I'd try looking at memcached (http://www.danga.com/memcached/apis.html).
No hacking or reimplementation of the interpreter would be necessary, and
there's a Python api available. I haven't used it for anything production related,
but I have played with it a bit, and it's fast and stable.
Is memcached available for Windows, then? I've heard nice things about
it, but thought it was Unix-only.

Paul.
----------------------------

It is available for Windows, yes. As a matter of fact, I've never used it on Unix.
DISCLAIMER:
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and
may contain legally privileged and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and
any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify
me and permanently delete the original and any copy of any e-mail and any printout thereof.
E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept
liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission.

NOTICE REGARDING PRIVACY AND CONFIDENTIALITY
Direct Edge ECN LLC may, at its discretion, monitor and review the content of all e-mail communications.

www.directedge.com
Jun 27 '08 #7
Tim Golden wrote:
li********@gmail.com wrote:
>I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

I stronly suggest you look at IPython [1]. To do what I think
you're describing, you'd need to hack or reimplement the interpreter.
And that's what they've done. ISTR that they even have a branch
which is dealing with parallel instances.
Sorry, having seen Roger D's memcached suggestion, I realise I may
have misinterpreted your requirement. I thought that you wanted
a Python interpreter session to share its objects to another window,
hence my IPython suggestion. If you're writing your own console app
and want to share stuff, then there's any number of IPC possibilities.

Roger's already mentioned memcached which I've no more than a
passing knowledge of. But Pyro [1] is often a good bet for these
things, and the pyprocessing [2] module is gaining a fair bit of
traction at the moment. (To name just two out of many).

TJG

[1] http://pyro.sf.net
[2] http://pyprocessing.berlios.de/

Jun 27 '08 #8
<<< No Message Collected >>>
Jun 27 '08 #9
[Bizarrely, my mail system seems to be randomly misfiring.
If you've already seen this, please ignore!]

Tim Golden wrote:
Sorry, having seen Roger D's memcached suggestion, I realise I may
have misinterpreted your requirement. I thought that you wanted
a Python interpreter session to share its objects to another window,
hence my IPython suggestion. If you're writing your own console app
and want to share stuff, then there's any number of IPC possibilities.

Roger's already mentioned memcached which I've no more than a
passing knowledge of. But Pyro [1] is often a good bet for these
things, and the pyprocessing [2] module is gaining a fair bit of
traction at the moment. (To name just two out of many).

TJG

[1] http://pyro.sf.net
[2] http://pyprocessing.berlios.de/

Jun 27 '08 #10
Your project interests me. Actually I was thinking about doing the
same. I hadn't worked on it at all, but I though about it and had the
idea about reading the session namespace directly, which I though
would be stored in the __dict__ attribute of something.

After reading your post, I have been trying a little bit, and I have
found a way to do it with ipython. If you open an ipython console,
press _ then hit TAB, you'll see it stores some useful information,
including all input, all output, and after some searching, a
dictionary matching all variables to its values.

__IPYTHON__.user_ns

There is a little extra stuff in there that you don't want, but that
can be easily filtered (the extra stuff is either 'In', 'Out', 'help'
or starts with '_'). I've tried it, and you can change the value in
that dict to alter the value of the real variable. Say you have a
variable 'test':

test=5
__IPYTHON__.user_ns['test']=4
print test #prints 5

If I get it right, python is a dynamic language, and you won't break
things by messing around with its inner stuff like this, but you
better check it.

Is this what you had in mind?
Jun 27 '08 #11
I meant it prints 4, which means the value of test is modified by the
access to the dict
test=5
__IPYTHON__.user_ns['test']=4
print test #prints 4
Jun 27 '08 #12
I'm not certain if this is what you want but try this for the first
window:

import __main__
localvars = __main__.__dict__
dir(localvars) # lists names of all objects available to the
interpreter.

And then you can pass localvars anywhere in the program - so after a
command is entered in one window, you can import the all locals into
the other by putting this function in the second window:

def SetWindow2Locals(localvars):
__main__.__dict__ = localvars

and do the reverse by sending the updated localvars from window 2 to
window 1. Setting up an object between the 2 windows should be enough:

# window 1, must be created first
import __main__
localvars = __main__.__dict__
# create second window (say, win2)
win2.SetLocals(localvars)

in win2 after creation, there is a method/function:

SetLocals(localvars):
__main__.__dict__ = localvars

If you want to store the object in something like a DB, you can store
this localvars dictionary rather than the actual objects. This should
be efficient because it stores the object name and ID only but not the
objects themselves. However, if the objects are deleted, after saving
and before re-loading, you may get some odd results but I'm not sure.
I would be very careful of using this in the wild.

You can also add objects singly to the dictionary by hand with a
function:

def importobject(obj):
__main__.__dict__[obj.__name__] = obj

Which should make them available in the other window easily. If a new
object is added to the __main__.__dict__, just grab it and send it to
this function. Make sure that you send the object and not just its
name because the name is only a string.

Sorry if this is not what you were after.

Alan

On May 29, 2:47 pm, lixinyi...@gmail.com wrote:
Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!
Jun 27 '08 #13
Lie
lixinyi...@gmail.com wrote:
Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, after which if you go back to the command window and
type 'a' and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!
In all kinds of code, it's best to seperate the workers code and the
UI code, in your case, you should create a backend (worker), which is
a class that stands on its own, and two foreground class (UI) that is
completely independent of each other but have the same interface. The
backend code would have an event that is raised when it is changed to
notify the UI (esp. The gui one) that it has changed since last time,
possibly passing info on what have been changed. The front ends, would
watch for this event as necessary (I don't think it is necessary for
the command line to watch this event) and react to it as necessary
like refreshing the view. The front-end window may only call functions
on the backend to interact with the data being worked on (in short the
backend class is opaque).

This obliviate the need to share data between the two (or more)
windows (UI) because all the data are contained in the backend class
that the frontend can't access directly.
Jun 27 '08 #14
Lie
On Jun 2, 1:29*am, Lie <Lie.1...@gmail.comwrote:
lixinyi...@gmail.com wrote:
Hi!
I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.
For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, *after which if you go back to the command window and
type 'a' *and press enter, you see that
varable a's value has been changed to 100.
So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?
Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.
Come on guys, give me some ideas. Thanks in advance!

In all kinds of code, it's best to seperate the workers code and the
UI code, in your case, you should create a backend (worker), which is
a class that stands on its own, and two foreground class (UI) that is
completely independent of each other but have the same interface. The
backend code would have an event that is raised when it is changed to
notify the UI (esp. The gui one) that it has changed since last time,
possibly passing info on what have been changed. The front ends, would
watch for this event as necessary (I don't think it is necessary for
the command line to watch this event) and react to it as necessary
like refreshing the view. The front-end window may only call functions
on the backend to interact with the data being worked on (in short the
backend class is opaque).

This obliviate the need to share data between the two (or more)
windows (UI) because all the data are contained in the backend class
that the frontend can't access directly.
To clarify what I meant, the front ends should never contain any
working data except the ones needed for the UI to illustrate what it
wanted to show at the moment, and even then, it is accessed in read
only fashion.

And actually because of Python's Global Interpreter Lock, which means
that your program would all be contained in the same python
interpreter instance (unless you do some workarounds), passing objects/
lists around between python program is cheap because they're just a
"reference" passing (like pointer passing in C/C++)

This approach is a simple "server-client" method (not a true server-
client method though, since a true one cannot share unserialized
data), and is extremely scalable, it's easy to add a third window for
example, there is no need for every front end to be aware that there
are other front ends, since it just watches for the "Changed" event
from the backend.
Jun 27 '08 #15
Lie
On May 29, 1:47*pm, lixinyi...@gmail.com wrote:
Hi!

I'm currently working on a scientific computation software built in
python.
What I want to implement is a Matlab style command window <->
workspace interaction.

For example, you type 'a=1' in the command window, and you see a list
item named 'a' in the workspace.
You double click the icon of the item, and you see its value. You can
modify the value of the list item,
1 -100 etc, *after which if you go back to the command window and
type 'a' *and press enter, you see that
varable a's value has been changed to 100.

So my question is : if you have two DOS command windows running under
WINDOWS OS, how can you make them share the same internal variable
buffer? Or is there any easier way to implement such kind of
interaction?

Maybe I could just build a small database to store all the values and
access them from both programs, but chances are sometimes I have to
deal with big arrays, and they will eat extra memory if I keep them in
a database. Is there anyway to access a shell's local memory buffer?
I tried to use shell.interp.locals() in wxPython, but there's too many
variables in the list which I don't actually need.

Come on guys, give me some ideas. Thanks in advance!
As an addition: Don't try to share data between windows, it's messy,
fragile, and easy to make bugs.

PS: Do not confuse Lie (Me) and Lee (OP)
Jun 27 '08 #16

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Logan | last post by:
Several people asked me for the following HOWTO, so I decided to post it here (though it is still very 'alpha' and might contain many (?) mistakes; didn't test what I wrote, but wrote it - more or...
20
by: Matthew Thorley | last post by:
My friend sent me an email asking this: > I'm attemtping to decide which scripting language I should master and > was wondering if it's possible to do > these unixy awkish commands in python:...
3
by: Greg Lindstrom | last post by:
Hello- I am running python 2.3. on an HP-9000 box running Unix and have a POSIX script that sets up my production environment. I would like to run the script from inside a python routine and...
3
by: David Durkee | last post by:
Hi, I'm trying to write a script I can run from tcsh in Terminal (on Mac OS X) that will set environment variables that can be accessed by subsequent commands I execute in that session. Not...
2
by: Xah Lee | last post by:
Python Doc Problem Example: os.system Xah Lee, 2005-09 today i'm trying to use Python to call shell commands. e.g. in Perl something like output=qx(ls) in Python i quickly located the...
30
by: bblais | last post by:
Hello, Let me start by saying that I am coming from a background using Matlab (or Octave), and C++. I am going to outline the basic nuts-and-bolts of how I work in these languages, and ask for...
5
by: Maxim Veksler | last post by:
Hello list, I'm trying to write a python script that would allow me to manipulate shell variables of the calling shell. I'm trying to write some logic that would know to add LD_LIBRARY_PATH to...
1
by: Lee | last post by:
Hi, thank your for your reply. I will try iPython. I did try sage for a while, but I found it quite heavy, and I'm not sure whether it's easy to expand like python or not. New libraries can be...
1
by: Fredrik Lundh | last post by:
John Lawrence wrote: doesn't exactly work for Python scripts, though: $ cat env.py #!/usr/bin/env python import os os.environ = "hello"
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.