Psycho question

[David C. Ullrich]

Just heard about Psycho. I've often wondered why someone
doesn't make something that does exactly what Psycho does - keen.

Silly question: It's correct, is it not, that Psycho doesn't
actually modify the Python installation, except by adding a
module or two (so that code not using Psycho is absolutely
unaffected)?

Thanks,

--
David C. Ullrich

[Erik Max Francis]

David C. Ullrich wrote:

Just heard about Psycho. I've often wondered why someone
doesn't make something that does exactly what Psycho does - keen.

Silly question: It's correct, is it not, that Psycho doesn't
actually modify the Python installation, except by adding a
module or two (so that code not using Psycho is absolutely
unaffected)?

That's correct. Hi, David!

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM, Y!M erikmaxfrancis
Longevity has its place. But I'm not concerned about that now.
-- Dr. Martin Luther King, Jr.

[David C. Ullrich]

In article <w_******************************@speakeasy.net> ,
Erik Max Francis <ma*@alcyone.comwrote:

David C. Ullrich wrote:

Just heard about Psycho. I've often wondered why someone
doesn't make something that does exactly what Psycho does - keen.

Silly question: It's correct, is it not, that Psycho doesn't
actually modify the Python installation, except by adding a
module or two (so that code not using Psycho is absolutely
unaffected)?

That's correct. Hi, David!

Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just "accelerate" a few Python routines that'll
be great.

Tentatively a very happy camper. See ya.

--
David C. Ullrich

[bearophileHUGS]

David C. Ullrich:

Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just "accelerate" a few Python routines that'll
be great.

To have better performance with Psyco you need low-level style code,
generally not lazy, etc, and adopt some programming conventions, so
you may have to rewrite your routines for max speed.

If some of your routines are too much slow there are many ways in
Python to write faster modules, like Cython, Weave, Inline, Swig, SIP,
ShedSkin, etc. For bioinformatics purposes I have found that Pyd + D
language is good for me (I have tried Pyrex too few times, but I have
lost my patience trying to track down in a jungle of ugly auto-
generated C code where some reference count updates happen. Writing D
code is hugely faster/better for me. Even writing a C extension for
Python from scratch may be better for me because there aren't hidden
things happening everywhere. I presume other people don't share this
problems of mine because there are lot of people using Cython now).

Bye,
bearophile

[David C. Ullrich]

In article
<8f**********************************@e53g2000hsa. googlegroups.com>,
be************@lycos.com wrote:

David C. Ullrich:

Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just "accelerate" a few Python routines that'll
be great.

To have better performance with Psyco you need low-level style code,
generally not lazy, etc, and adopt some programming conventions, so
you may have to rewrite your routines for max speed.

Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like

for j in range(len(bytes)/3):
g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g

then convert back to a list or string or whatever outside
the accelerated function.

Surely something like _that_ is exactly what Psyco is going
to do well with, yes? (Ok, we're talking about image processing,
in cases where I can't figure out how to get PIL to do whatever
directly. So sometimes there will be double loops

for row in range(width):
for col in range(height):
do_something[row*width + col]

but at least for the things I can think of right now it
shouldn't get much worse than that.)

The things you mention below sound very interesting - I'm
going to try Psyco first because unless I'm missing something
I won't have to learn how to use it. Someday when it turns out
to be not good enough I'll be in touch...

If some of your routines are too much slow there are many ways in
Python to write faster modules, like Cython, Weave, Inline, Swig, SIP,
ShedSkin, etc. For bioinformatics purposes I have found that Pyd + D
language is good for me (I have tried Pyrex too few times, but I have
lost my patience trying to track down in a jungle of ugly auto-
generated C code where some reference count updates happen. Writing D
code is hugely faster/better for me. Even writing a C extension for
Python from scratch may be better for me because there aren't hidden
things happening everywhere. I presume other people don't share this
problems of mine because there are lot of people using Cython now).

Bye,
bearophile

--
David C. Ullrich

[David C. Ullrich]

In article <du****************************@text.giganews.com> ,
"David C. Ullrich" <du******@sprynet.comwrote:

In article
<8f**********************************@e53g2000hsa. googlegroups.com>,
be************@lycos.com wrote:

David C. Ullrich:

Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just "accelerate" a few Python routines that'll
be great.

To have better performance with Psyco you need low-level style code,
generally not lazy, etc, and adopt some programming conventions, so
you may have to rewrite your routines for max speed.

Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like

for j in range(len(bytes)/3):
g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g

then convert back to a list or string or whatever outside
the accelerated function.

Surely something like _that_ is exactly what Psyco is going
to do well with, yes?

teehee. Downloaded Psyco. The install actually worked.
Tried exactly what's above with a list of 3 million ints.
Didn't time it carefully, seemed to take about two seconds.
Ran it again, in case the second run would be faster for some reason.
Second was about the same.

Said "import psyco", etc. Ran the routine again, it returned
in _no_ time, perceptually.

This is so cool. Gonna find out whether a decorator that
returns the accelerated function works, just for the fun
of deciding what the name should be: @cool? @wheee?
@wow? @dontblinkyoullmissit?

>(Ok, we're talking about image processing,
in cases where I can't figure out how to get PIL to do whatever
directly. So sometimes there will be double loops

for row in range(width):
for col in range(height):
do_something[row*width + col]

but at least for the things I can think of right now it
shouldn't get much worse than that.)

The things you mention below sound very interesting - I'm
going to try Psyco first because unless I'm missing something
I won't have to learn how to use it. Someday when it turns out
to be not good enough I'll be in touch...

If some of your routines are too much slow there are many ways in
Python to write faster modules, like Cython, Weave, Inline, Swig, SIP,
ShedSkin, etc. For bioinformatics purposes I have found that Pyd + D
language is good for me (I have tried Pyrex too few times, but I have
lost my patience trying to track down in a jungle of ugly auto-
generated C code where some reference count updates happen. Writing D
code is hugely faster/better for me. Even writing a C extension for
Python from scratch may be better for me because there aren't hidden
things happening everywhere. I presume other people don't share this
problems of mine because there are lot of people using Cython now).

Bye,
bearophile

--
David C. Ullrich

[Erik Max Francis]

David C. Ullrich wrote:

Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like

for j in range(len(bytes)/3):
g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g

If len(bytes) is large, you might want to use `xrange`, too. `range`
creates a list which is not really what you need.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM, Y!M erikmaxfrancis
You and I / We've seen it all / Chasing our hearts' desire
-- The Russian and Florence, _Chess_

[bearophileHUGS]

Erik Max Francis:

If len(bytes) is large, you might want to use `xrange`, too. `range`
creates a list which is not really what you need.

That's right for Python, but Psyco uses normal loops in both cases,
you can time this code in the two situations:

def foo1(n):
count = 0
for i in range(n):
count += 1
print count

def foo2(n):
count = 0
for i in xrange(n):
count += 1
print count

import psyco; psyco.full()
N = 100000000
#foo1(N)
foo2(N)

Bye,
bearophile

[David C. Ullrich]

In article <68******************************@speakeasy.net> ,
Erik Max Francis <ma*@alcyone.comwrote:

David C. Ullrich wrote:

Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like

for j in range(len(bytes)/3):
g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g

If len(bytes) is large, you might want to use `xrange`, too. `range`
creates a list which is not really what you need.

I didn't follow the explanation, but I read in the docs
that xrange can actually be slower under Psyco.

This morning I learned that my guess that array.array was
a good idea was correct: When I pass a list of ints to the
routine above it gets accelerated by a factor of between
10 and 15, while if I pass an array it's closer to 50.

This is so cool. Maybe I already said that.

--
David C. Ullrich

[MRAB]

On Aug 6, 8:52*pm, "David C. Ullrich" <[e-mail]:

In article
<[e-mail]>,

*[e-mail] wrote:

David C. Ullrich:

Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just "accelerate" a few Python routines that'll
be great.

To have better performance with Psyco you need low-level style code,
generally not lazy, etc, and adopt some programming conventions, so
you may have to rewrite your routines for max speed.

Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like

for j in range(len(bytes)/3):
* g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
* bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g

then convert back to a list or string or whatever outside
the accelerated function.

[snip]
A couple of points:

1. '/' with ints in Python 2.x returns an int, but from Python 3.x
it'll return a float. You're recommended to use '//' for int division.

2. 'range' can accept a step value, so you can rewrite that as:

for j in range(0, len(bytes), 3):
g = (bytes[j] + bytes[j+1] + bytes[j+2])//3 # I think you also
want // here
bytes[j] = bytes[j+1] = bytes[j+2] = g

[David C. Ullrich]

In article
<de**********************************@p25g2000hsf. googlegroups.com>,
MRAB <go****@mrabarnett.plus.comwrote:

On Aug 6, 8:52*pm, "David C. Ullrich" <[e-mail]:

In article
<[e-mail]>,

*[e-mail] wrote:

David C. Ullrich:
Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just "accelerate" a few Python routines that'll
be great.

To have better performance with Psyco you need low-level style code,
generally not lazy, etc, and adopt some programming conventions, so
you may have to rewrite your routines for max speed.

Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like

for j in range(len(bytes)/3):
* g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
* bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g

then convert back to a list or string or whatever outside
the accelerated function.

[snip]
A couple of points:

1. '/' with ints in Python 2.x returns an int, but from Python 3.x
it'll return a float. You're recommended to use '//' for int division.

2. 'range' can accept a step value, so you can rewrite that as:

for j in range(0, len(bytes), 3):
g = (bytes[j] + bytes[j+1] + bytes[j+2])//3 # I think you also
want // here
bytes[j] = bytes[j+1] = bytes[j+2] = g

Not the issues I expected to be worrying about here, but thanks.

Of course the range(0, len(bytes), 3) is more elegant, and
it's probably faster in Python, but curiously it's much
slower under Psyco! Otoh xrange(0, len(bytes), 3) becomes
pretty fast again. So I conjecture that Psyco compiles
"for j in range(l)" just as a loop but actually constructs
an array for range(0, l, step).

Also very curiously, // inside the loop is much slower than
/ here (under Psyco). This one I'm not going to guess why...

Honest:

"""Ah - psyco does work with exec if the import psyco,
etc is inside the code being executed (right now it's
in its own namespace, hence a fresh import each time
- check whether this works with exec in default
namespaces).

Ie, this script works fine in DUShell:"""

from psyco import proxy, bind

def f(b):
for j in range(len(b)/3):
i = 3*j
g = (b[i] + b[i+1] + b[i+2])/3
b[i] = b[i+1] = b[i+2] = g

g = proxy(f)

def h(b):
for j in range(0,len(b),3):
#for i in range(len(b)/3):
#j = 3*i
g = (b[j] + b[j+1] + b[j+2])//3
b[j] = b[j+1] = b[j+2] = g
bind(h)
from time import time

fs = {'f':f, 'g':g, 'h':h}

def t(f,b):
F = fs[f]
st = time()
F(b)
et = time()
print "%s: %s" % (f, et-st)

b = range(30000)

from array import array
c = array('i',b)
t('f',c)
t('g',c)
t('h',c)
t('f',c)
t('g',c)
t('h',c)

outputs

f: 0.0158488750458
g: 0.000610113143921
h: 0.00200295448303
f: 0.0184948444366
g: 0.000257015228271
h: 0.00116610527039

--
David C. Ullrich

[Terry Reedy]

For the benefit of those trying to find, download, and import the
module, its name is psyco (no 'h'), not psycho .
http://psyco.sourceforge.net/