Gnuplot.py and, _by far_, the weirdest thing I've ever seen on my computer

[syd]

I don't even know where to begin. This is just bizarre. I just picked
up the Gnuplot.py module (a light interface to gnuplot commands) and
was messing around with it today.

I've got a tiny script, but it only works from the command line about
half the time! In the python interpreter, 100%. Ipython, 100%. I'm
not kidding.

#!/bin/env python
import Gnuplot
g = Gnuplot.Gnuplot(debug=1)
g.title('A simple example')
g('set data style linespoints')
g('set terminal png small color')
g('set output "myGraph.png"')
g.plot([[0,1.1], [1,5.8], [2,3.3], [3,100]])

Here's just one example -- it does not work, then it works. It seems
totally random. It will work a few times, then it won't for a few
times...

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmp5LXAow' notitle

gnuplot> plot '/tmp/tmp5LXAow' notitle
^
can't read data file "/tmp/tmp5LXAow"
line 0: util.c: No such file or directory

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmpHMTkpL' notitle

(and it makes the graph image just fine)

I mean what the hell is going on? My permissions on /tmp are wide open
(drwxrwxrwt). It does the same thing when I run as root. And it
_always_ works when I use the interpreter or interactive python.

Any clues would be greatly appreciated. I'm baffled.

[James Stroud]

Are you on NFS or are you using a raid? This may have to do with the time it
takes to access your files, etc.

On Monday 04 April 2005 02:23 pm, syd wrote:

I don't even know where to begin. This is just bizarre. I just picked
up the Gnuplot.py module (a light interface to gnuplot commands) and
was messing around with it today.

I've got a tiny script, but it only works from the command line about
half the time! In the python interpreter, 100%. Ipython, 100%. I'm
not kidding.

#!/bin/env python
import Gnuplot
g = Gnuplot.Gnuplot(debug=1)
g.title('A simple example')
g('set data style linespoints')
g('set terminal png small color')
g('set output "myGraph.png"')
g.plot([[0,1.1], [1,5.8], [2,3.3], [3,100]])

Here's just one example -- it does not work, then it works. It seems
totally random. It will work a few times, then it won't for a few
times...

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmp5LXAow' notitle

gnuplot> plot '/tmp/tmp5LXAow' notitle
^
can't read data file "/tmp/tmp5LXAow"
line 0: util.c: No such file or directory

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmpHMTkpL' notitle

(and it makes the graph image just fine)

I mean what the hell is going on? My permissions on /tmp are wide open
(drwxrwxrwt). It does the same thing when I run as root. And it
_always_ works when I use the interpreter or interactive python.

Any clues would be greatly appreciated. I'm baffled.

--
James Stroud, Ph.D.
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

[David M. Cooke]

"syd" <sy*********@gmail.com> writes:

I don't even know where to begin. This is just bizarre. I just picked
up the Gnuplot.py module (a light interface to gnuplot commands) and
was messing around with it today.

I've got a tiny script, but it only works from the command line about
half the time! In the python interpreter, 100%. Ipython, 100%. I'm
not kidding.

#!/bin/env python
import Gnuplot
g = Gnuplot.Gnuplot(debug=1)
g.title('A simple example')
g('set data style linespoints')
g('set terminal png small color')
g('set output "myGraph.png"')
g.plot([[0,1.1], [1,5.8], [2,3.3], [3,100]])

Here's just one example -- it does not work, then it works. It seems
totally random. It will work a few times, then it won't for a few
times...

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmp5LXAow' notitle

gnuplot> plot '/tmp/tmp5LXAow' notitle
^
can't read data file "/tmp/tmp5LXAow"
line 0: util.c: No such file or directory

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmpHMTkpL' notitle

(and it makes the graph image just fine)

I mean what the hell is going on? My permissions on /tmp are wide open
(drwxrwxrwt). It does the same thing when I run as root. And it
_always_ works when I use the interpreter or interactive python.

Any clues would be greatly appreciated. I'm baffled.

What's your OS? Python version? Gnuplot.py version (I assume 1.7)?
Put a 'import sys; print sys.version' in there to make sure /bin/env
is using the same python as you expect it to.

It looks like any temporary file it's writing to is deleted too early.

Have a look at gp_unix.py in the Gnuplot source. There's some
customization options that might be helpful. In particular, I'd try

import Gnuplot
Gnuplot.GnuplotOpts.prefer_fifo_data = 0

.... then the data will be save to a temporary file instead of piped
through a fifo.

Alternatively, try
Gnuplot.GnuplotOpts.prefer_inline_data = 1

.... then no file will be used.

[I don't use Gnuplot myself; this is just what I came up with after a
few minutes of looking at it]

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca

[Paul Cochrane]

I had similar problems with Gnuplot.py, basically, one couldn't use ascii
mode for plotting grid data, and had to set binary=1 in the GridData()
method. (this is off the top of my head, so I might have the binary/ascii
thing the wrong way around). What fixed the problem was going to version
Gnuplot.py 1.7. What version are you using? If it's 1.6, that *might* be
the problem (although, I can't be 100% sure).

HTH

Regards,

Paul
* syd (sy*********@gmail.com) [050405 07:26]:

I don't even know where to begin. This is just bizarre. I just picked
up the Gnuplot.py module (a light interface to gnuplot commands) and
was messing around with it today.

I've got a tiny script, but it only works from the command line about
half the time! In the python interpreter, 100%. Ipython, 100%. I'm
not kidding.

#!/bin/env python
import Gnuplot
g = Gnuplot.Gnuplot(debug=1)
g.title('A simple example')
g('set data style linespoints')
g('set terminal png small color')
g('set output "myGraph.png"')
g.plot([[0,1.1], [1,5.8], [2,3.3], [3,100]])

Here's just one example -- it does not work, then it works. It seems
totally random. It will work a few times, then it won't for a few
times...

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmp5LXAow' notitle

gnuplot> plot '/tmp/tmp5LXAow' notitle
^
can't read data file "/tmp/tmp5LXAow"
line 0: util.c: No such file or directory

bash-2.05b$ ./myGnu.py
gnuplot> set title "A simple example"
gnuplot> set data style linespoints
gnuplot> set terminal png small color
gnuplot> set output "myGraph.png"
gnuplot> plot '/tmp/tmpHMTkpL' notitle

(and it makes the graph image just fine)

I mean what the hell is going on? My permissions on /tmp are wide open
(drwxrwxrwt). It does the same thing when I run as root. And it
_always_ works when I use the interpreter or interactive python.

Any clues would be greatly appreciated. I'm baffled.

--
http://mail.python.org/mailman/listinfo/python-list

--
Paul Cochrane
Computational Scientist/Software Developer
Earth Systems Science Computational Centre
Rm 703, SMI Building
University of Queensland
Brisbane
Queensland 4072
Australia

[Fernando Perez]

syd wrote:

I don't even know where to begin. This is just bizarre. I just picked
up the Gnuplot.py module (a light interface to gnuplot commands) and
was messing around with it today.

I've got a tiny script, but it only works from the command line about
half the time! In the python interpreter, 100%. Ipython, 100%. I'm
not kidding.

Nothing strange about it. When run standalone, the python interpreter quits and
cleans up after itself, possibly deleting temp files before gnuplot gets a
chance to use them. You can put a time.sleep(5) before exiting, or even safer,
a full safety check:

while 1:
if os.path.isfile(your_plot_filename):
break
time.sleeep(1)

Best,

f

[Fernando Perez]

syd wrote:

I don't even know where to begin. This is just bizarre. I just picked
up the Gnuplot.py module (a light interface to gnuplot commands) and
was messing around with it today.

I've got a tiny script, but it only works from the command line about
half the time! In the python interpreter, 100%. Ipython, 100%. I'm
not kidding.

Nothing strange about it. When run standalone, the python interpreter quits and
cleans up after itself, possibly deleting temp files before gnuplot gets a
chance to use them. You can put a time.sleep(5) before exiting, or even safer,
a full safety check:

while 1:
if os.path.isfile(your_plot_filename):
break
time.sleeep(1)

Best,

f

[syd]

Thanks for all the help, guys!

The affected computer is at home, but I'll address all these issues
before I post again with my fix.

James: It's on my personal box, and I've got reiserfs.

David:
OS: GNU/Linux 2.4 (Gentoo)
Python 2.3
GNUplot 1.6

I just checked the Gentoo Portage tree and 1.7 is listed "~x86"
(unstable), so unwittingly I had gotten 1.6 when I emerged gnuplot-py.
Definitely, the first thing I'm going to do is upgrade to 1.7.

Before I do, though, I'll try to see if this works:
Gnuplot.GnuplotOpts.prefer_fifo_data = 0
Gnuplot.GnuplotOpts.prefer_inline_data = 1

It sounds like Paul had the same problem. I'll also try the "binary =
1" before I upgrade.

Fernando, that's a creative solution, I'll try it as well...
while 1:
if os.path.isfile(your_plot_filename):
break
time.sleep(1)

Thanks guys!

[Fernando Perez]

syd wrote:

Thanks for all the help, guys! Fernando, that's a creative solution, I'll try it as well...
while 1:
if os.path.isfile(your_plot_filename):
break
time.sleep(1)

More like a desperate brute force one, but it gets the job done :) You
mentioned having ipython, so you can look in the Gnuplot2 module there, around
line 630. You'll see the lengths ipython goes to to ensure that .eps files are
properly created if you call plot() with a filename argument. That solution
has a cutoff, it tries 20 times and eventually gives up if it still doesn't
work. You may want to implement something similar, to avoid endlessly waiting
for your plot in case there is a more serious problem preventing the file from
being created at all.

On a different note, you might want to check out matplotlib. I held out as a
diehard gnuplot user for a long time, waiting for matplotlib to become
production ready (I've been using gnuplot since 1991!). But mpl has really
matured recently, and it offers a level of sophistication and features that
gnuplot simply can't match. Give it a try, you may like it. I've moved my
production code to it, and couldn't be happier. The gnuplot support will
remain in ipython for those who like it, but I won't be adding further features
to it (not that it has changed much in a while, as it's pretty solid).

best,

f

[syd]

Ok yall, here's my summary. Thanks again for all the help.

Most importantly, upgrading to gnuplot-py 1.7 fixes this problem.

In gnuplot-py 1.6, this is the bottom line...

Does not fix:
Gnuplot.GnuplotOpts.prefer_fifo_data = 0

Fixes:
Gnuplot.GnuplotOpts.prefer_inline_data = 1

Fixes: (placed between setting output and plot)
while 1:
if os.path.isfile("myGraph.png"):
break
time.sleep(1)

As for matplotlib, I checked it out. Looks amazing! I really, really
like what demos I tried.

HOWEVER, I could not find a good way to do smoothing. I like the
gnuplot bezier smoothing. This wouldn't be the hardest thing in the
world to code, but I was a little disappointed. If yall have any
ideas, bring em on! :)

[John Hunter]

>>>>> "syd" == syd <sy*********@gmail.com> writes:

syd> As for matplotlib, I checked it out. Looks amazing! I
syd> really, really like what demos I tried.

syd> HOWEVER, I could not find a good way to do smoothing. I like
syd> the gnuplot bezier smoothing. This wouldn't be the hardest
syd> thing in the world to code, but I was a little disappointed.
syd> If yall have any ideas, bring em on! :)

What is the gnuplot interface to use smoothing?

When I want to do something like this, I use scipy's interpolate
module and pass the reults to matplotlib for plotting.
from scipy import arange, sin, pi, interpolate
from pylab import plot, show

t = arange(0, 2.0, 0.1)
y = sin(2*pi*t)
tck = interpolate.splrep(t, y, s=0)
tnew = arange(0, 2.0, 0.01)
ynew = interpolate.splev(tnew, tck, der=0)
plot(t, y, 'o', tnew, ynew)

show()