Newbie question about tuples and list comprehensions

[idiolect]

Hi all - Sorry to plague you with another newbie question from a
lurker. Hopefully, this will be simple.

I have a list full of RGB pixel values read from an image. I want to
test each RGB band value per pixel, and set it to something else if it
meets or falls below a certain threshold - i.e., a Red value of 0
would be changed to 50.

I've built my list by using a Python Image Library statement akin to
the following:

data = list(image.getdata())

Which produces a very long list that looks like [(0,150,175),
(50,175,225),...]. I'm trying to figure out a fast and pythonic way
to perform my operation. The closest I've come so far to a succinct
statement is a list comprehension along the syntax of:

source = [((x,y,z),(x+50,y+50,z+50))[bool(x or y or z < 50)] for
(x,y,z) in source]

....which kind of approaches the effect I'm looking for, but it doesn't
really test and change each value in the tuple individually. My
understanding of the things you can do with lists and python in
general is rather naive, so I would appreciate any insight anyone can
offer since I am not sure if I'm even headed down the correct path
with list comprehensions.

Much obliged!

[Matimus]

On Jun 25, 2:37*pm, idiolect <[e-mail]:

Hi all - Sorry to plague you with another newbie question from a
lurker. *Hopefully, this will be simple.

I have a list full of RGB pixel values read from an image. *I want to
test each RGB band value per pixel, and set it to something else if it
meets or falls below a certain threshold - i.e., a Red value of 0
would be changed to 50.

I've built my list by using a Python Image Library statement akin to
the following:

data = list(image.getdata())

Which produces a very long list that looks like [(0,150,175),
(50,175,225),...]. *I'm trying to figure out a fast and pythonic way
to perform my operation. *The closest I've come so far to a succinct
statement is a list comprehension along the syntax of:

source = [((x,y,z),(x+50,y+50,z+50))[bool(x or y or z < 50)] for
(x,y,z) in source]

...which kind of approaches the effect I'm looking for, but it doesn't
really test and change each value in the tuple individually. *My
understanding of the things you can do with lists and python in
general is rather naive, so I would appreciate any insight anyone can
offer since I am not sure if I'm even headed down the correct path
with list comprehensions.

Much obliged!

List comprehension is just a tool. If it looks more complicated than
it would if you used a regular for loop, then use a regular for loop.

It seems like you are evaluating a lot of data, so I would take
advantage of pythons (lazy) generators to save a lot of memory:

def brighten(seq, threshold=50): # I don't know what the generic name
for this operation is
for pixel in seq:
if any(channel < threshold for channel in pixel):
r, g, b = pixel
yield r + threshold, g + threshold, b + threshold

then...

source = brighten(image.getdata())

Source is going to be a lazy generator, so you can cast it to list, or
just pass it to something else that takes a sequence. The latter is
preferred.

Matt

[John Machin]

On Jun 26, 7:37 am, idiolect <[e-mail]:

Hi all - Sorry to plague you with another newbie question from a
lurker. Hopefully, this will be simple.

I have a list full of RGB pixel values read from an image. I want to
test each RGB band value per pixel, and set it to something else if it
meets or falls below a certain threshold - i.e., a Red value of 0
would be changed to 50.

I've built my list by using a Python Image Library statement akin to
the following:

data = list(image.getdata())

Which produces a very long list that looks like [(0,150,175),
(50,175,225),...]. I'm trying to figure out a fast and pythonic way
to perform my operation. The closest I've come so far to a succinct
statement is a list comprehension along the syntax of:

source = [((x,y,z),(x+50,y+50,z+50))[bool(x or y or z < 50)] for
(x,y,z) in source]

...which kind of approaches the effect I'm looking for, but it doesn't
really test and change each value in the tuple individually. My
understanding of the things you can do with lists and python in
general is rather naive, so I would appreciate any insight anyone can
offer since I am not sure if I'm even headed down the correct path
with list comprehensions.

"x or y or z < 50" doesn't do what you think it does:

>>x, y, z = 120, 130, 140
x or y or z < 50

120

>>((x or y) or z) < 50

False

>>x or (y or (z < 50))

120

>>>

Here's one approach (requires Python 2.5 or later):
[(50 if x < 50 else x, 50 if y < 50 else y, 50 if z < 50 else z) for
(x, y, z) in source]

[Roger Miller]

First, I second Matt's comment about using a boring old for loop when
it is the simplest way to express what you want to do. Being Pythonic
is not about using exotic features to scrunch your code down to a cool
one-liner. It is about expressing your intentions in a simple, direct
way. Sometimes comprehensions, generators, etc. help. Sometimes they
only obfuscate.

That said, the *most* Pythonic way to do something is to find the
existing library that already does it. Check out the Image.point()
function in the PIL documentation. You may not need to convert the
image to a list at all.

- Roger

[Terry Reedy]

idiolect wrote:

Hi all - Sorry to plague you with another newbie question from a
lurker. Hopefully, this will be simple.

I have a list full of RGB pixel values read from an image. I want to
test each RGB band value per pixel, and set it to something else if it
meets or falls below a certain threshold - i.e., a Red value of 0
would be changed to 50.

I've built my list by using a Python Image Library statement akin to
the following:

data = list(image.getdata())

Which produces a very long list that looks like [(0,150,175),
(50,175,225),...]. I'm trying to figure out a fast and pythonic way
to perform my operation. The closest I've come so far to a succinct
statement is a list comprehension along the syntax of:

Why are you trying to do this with a list comprehension? Learn the
basics first. Perhaps you have read too many of the recent threads
presenting diverting challenges for bored experienced programmers. Some
of these were definitely not Pythonic code for real use.

First question: do you really want to create a new 'very long list' or
modify list 'data' in place. Let's assume the latter.

for i,tup in enumerate(data):
data[i] = replace(tup)

where replace(tup) is an expression or function that produces a tuple
meeting your criteria. Simplest is
(max(tup[0],Rthresh), max(tup[1],Gthresh), max(tup[2],Bthresh)).

If nearly all your pixels are ok, add the following before the
assignment so you only make replacements when actually needed:
if tup[0] < Rthresh or tup[1] < Gthresh or tup[2] < Bthresh:

Terry Jan Reedy

[idiolect]

On Jun 25, 7:26 pm, Terry Reedy <[e-mail]:

idiolect wrote:

Hi all - Sorry to plague you with another newbie question from a
lurker. Hopefully, this will be simple.

I have a list full of RGB pixel values read from an image. I want to
test each RGB band value per pixel, and set it to something else if it
meets or falls below a certain threshold - i.e., a Red value of 0
would be changed to 50.

I've built my list by using a Python Image Library statement akin to
the following:

data = list(image.getdata())

Which produces a very long list that looks like [(0,150,175),
(50,175,225),...]. I'm trying to figure out a fast and pythonic way
to perform my operation. The closest I've come so far to a succinct
statement is a list comprehension along the syntax of:

Why are you trying to do this with a list comprehension? Learn the
basics first. Perhaps you have read too many of the recent threads
presenting diverting challenges for bored experienced programmers. Some
of these were definitely not Pythonic code for real use.

First question: do you really want to create a new 'very long list' or
modify list 'data' in place. Let's assume the latter.

for i,tup in enumerate(data):
data[i] = replace(tup)

where replace(tup) is an expression or function that produces a tuple
meeting your criteria. Simplest is
(max(tup[0],Rthresh), max(tup[1],Gthresh), max(tup[2],Bthresh)).

If nearly all your pixels are ok, add the following before the
assignment so you only make replacements when actually needed:
if tup[0] < Rthresh or tup[1] < Gthresh or tup[2] < Bthresh:

Terry Jan Reedy

A giant thank-you to all who've posted in response to my query - these
are all much better approaches to my problem. I think I got hooked on
using a list comprehension as it seemed the most concise approach vs.
other techniques after a bunch of Google searches, but all of you have
pointed out more efficient methods. I appreciate your willingness to
indulge a n00b who hasn't thought his problem through, apparently.
I'll try all of these approaches out over the next day and see what
works best, although I suspect you've all posted sufficient solutions.

Can't wait to try these suggestions out - cheers, idiolect

[William McBrine]

On Wed, 25 Jun 2008 16:02:52 -0700, John Machin wrote:

Here's one approach (requires Python 2.5 or later):
[(50 if x < 50 else x, 50 if y < 50 else y, 50 if z < 50 else z) for
(x, y, z) in source]

[(max(x, 50), max(y, 50), max(z, 50)) for (x, y, z) in source]

--
09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 -- pass it on

[Peter Otten]

idiolect wrote:

On Jun 25, 7:26 pm, Terry Reedy <[e-mail]:

>idiolect wrote:

Hi all - Sorry to plague you with another newbie question from a
lurker. Hopefully, this will be simple.

I have a list full of RGB pixel values read from an image. I want to
test each RGB band value per pixel, and set it to something else if it
meets or falls below a certain threshold - i.e., a Red value of 0
would be changed to 50.

I've built my list by using a Python Image Library statement akin to
the following:

data = list(image.getdata())

Which produces a very long list that looks like [(0,150,175),
(50,175,225),...]. I'm trying to figure out a fast and pythonic way
to perform my operation. The closest I've come so far to a succinct
statement is a list comprehension along the syntax of:

Why are you trying to do this with a list comprehension? Learn the
basics first. Perhaps you have read too many of the recent threads
presenting diverting challenges for bored experienced programmers. Some
of these were definitely not Pythonic code for real use.

First question: do you really want to create a new 'very long list' or
modify list 'data' in place. Let's assume the latter.

for i,tup in enumerate(data):
data[i] = replace(tup)

where replace(tup) is an expression or function that produces a tuple
meeting your criteria. Simplest is
(max(tup[0],Rthresh), max(tup[1],Gthresh), max(tup[2],Bthresh)).

If nearly all your pixels are ok, add the following before the
assignment so you only make replacements when actually needed:
if tup[0] < Rthresh or tup[1] < Gthresh or tup[2] < Bthresh:

Terry Jan Reedy

A giant thank-you to all who've posted in response to my query - these
are all much better approaches to my problem. I think I got hooked on
using a list comprehension as it seemed the most concise approach vs.
other techniques after a bunch of Google searches, but all of you have
pointed out more efficient methods. I appreciate your willingness to
indulge a n00b who hasn't thought his problem through, apparently.
I'll try all of these approaches out over the next day and see what
works best, although I suspect you've all posted sufficient solutions.

Can't wait to try these suggestions out - cheers, idiolect

I'd like to emphasize Roger Miller's point with a small example script:

#!/usr/bin/env python
import sys
import Image

def transform_image_getdata(old, new):
image = Image.open(old)

image.putdata([(max(50, r), min(g*2, 255), 0) for r, g, b in
image.getdata()])

image.save(new)

def transform_image_point(old, new):
image = Image.open(old)

transform_red = [max(50, i) for i in range(256)]
transform_green = [min(i*2, 255) for i in range(256)]
transform_blue = [0 for i in range(256)]

image.point(transform_red + transform_green + transform_blue).save(new)
if __name__ == "__main__":
import os, time
def measure(f):
def g(*args, **kw):
start = time.time()
try:
return f(*args, **kw)
finally:
print f.__name__, time.time() - start
return g

for name in sys.argv[1:]:
a, b = os.path.splitext(name)
measure(transform_image_getdata)(name, a + ".getdata" + b)
measure(transform_image_point)(name, a + ".point" + b)

Run it:

$ ./transform_image.py tmp.jpg
transform_image_getdata 27.5557489395
transform_image_point 0.458500862122

For the example Image.point() is about 60 times faster than getdata(). It's
the way to go if you can handle the color channels individually.

Peter