Perl and Python, a practical side-by-side example.

[Shawn Milo]

I'm new to Python and fairly experienced in Perl, although that
experience is limited to the things I use daily.

I wrote the same script in both Perl and Python, and the output is
identical. The run speed is similar (very fast) and the line count is
similar.

Now that they're both working, I was looking at the code and wondering
what Perl-specific and Python-specific improvements to the code would
look like, as judged by others more knowledgeable in the individual
languages.

I am not looking for the smallest number of lines, or anything else
that would make the code more difficult to read in six months. Just
any instances where I'm doing something inefficiently or in a "bad"
way.

I'm attaching both the Perl and Python versions, and I'm open to
comments on either. The script reads a file from standard input and
finds the best record for each unique ID (piid). The best is defined
as follows: The newest expiration date (field 5) for the record with
the state (field 1) which matches the desired state (field 6). If
there is no record matching the desired state, then just take the
newest expiration date.

Thanks for taking the time to look at these.

Shawn

############### ############### ############### ############### ##############
Perl code:
############### ############### ############### ############### ##############
#! /usr/bin/env perl

use warnings;
use strict;

my $piid;
my $row;
my %input;
my $best;
my $curr;

foreach $row (<>){

chomp($row);
$piid = (split(/\t/, $row))[0];

push ( @{$input{$piid} }, $row );
}

for $piid (keys(%input)){

$best = "";

for $curr (@{$input{$piid }}){
if ($best eq ""){
$best = $curr;
}else{
#If the current record is the correct state

if ((split(/\t/, $curr))[1] eq (split(/\t/, $curr))[6]){
#If existing record is the correct state
if ((split(/\t/, $best))[1] eq (split(/\t/, $curr))[6]){
if ((split(/\t/, $curr))[5] gt (split(/\t/, $best))[5]){
$best = $curr;
}
}else{
$best = $curr;
}
}else{
#if the existing record does not have the correct state
#and the new one has a newer expiration date
if (((split(/\t/, $best))[1] ne (split(/\t/, $curr))[6]) and
((split(/\t/, $curr))[5] gt (split(/\t/, $best))[5])){
$best = $curr;
}
}
}
}
print "$best\n";
}

############### ############### ############### ############### ##############
End Perl code
############### ############### ############### ############### ##############


############### ############### ############### ############### ##############
Python code
############### ############### ############### ############### ##############

#! /usr/bin/env python

import sys

input = sys.stdin

recs = {}

for row in input:
row = row.rstrip('\n' )
piid = row.split('\t')[0]
if recs.has_key(pi id) is False:
recs[piid] = []
recs[piid].append(row)

for piid in recs.keys():
best = ""
for current in recs[piid]:
if best == "":
best = current;
else:
#If the current record is the correct state
if current.split(" \t")[1] == current.split(" \t")[6]:
#If the existing record is the correct state
if best.split("\t" )[1] == best.split("\t" )[6]:
#If the new record has a newer exp. date
if current.split(" \t")[5] best.split("\t" )[5]:
best = current
else:
best = current
else:
#If the existing record does not have the correct state
#and the new record has a newer exp. date
if best.split("\t" )[1] != best.split("\t" )[6] and
current.split(" \t")[5] best.split("\t" )[5]:
best = current

print best
############### ############### ############### ############### ##############
End Python code
############### ############### ############### ############### ##############

[Marc 'BlackJack' Rintsch]

In <54************ *@mid.individua l.net>, Bjoern Schliessmann wrote:

Bruno Desthuilliers wrote:

>Shawn Milo a écrit :

>> if recs.has_key(pi id) is False:

'is' is the identity operator - practically, in CPython, it
compares memory addresses. You *dont* want to use it here.

It's recommended to use "is None"; why not "is False"? Are there
multiple False instances or is False generated somehow?

Before `True` and `False` existed many people defined them as aliases to 1
and 0. And of course there are *many* other objects that can be used in a
boolean context of an ``if`` statement for testing "trueness" and
"falseness" .

Ciao,
Marc 'BlackJack' Rintsch

[John Machin]

On Mar 3, 7:08 pm, attn.steven.... @gmail.com wrote:

On Mar 2, 2:44 pm, "Shawn Milo" <S...@Milochik. comwrote:

(snipped)

I'm attaching both the Perl and Python versions, and I'm open to
comments on either. The script reads a file from standard input and
finds the best record for each unique ID (piid). The best is defined
as follows: The newest expiration date (field 5) for the record with
the state (field 1) which matches the desired state (field 6). If
there is no record matching the desired state, then just take the
newest expiration date.

Thanks for taking the time to look at these.

My attempts:
### Python (re-working John's code) ###

import sys

def keep_best(best, current):

ACTUAL_STATE = 1
# John had these swapped
DESIRED_STATE = 5
EXPIRY_DATE = 6

*Bullshit* -- You are confusing me with Bruno; try (re)?reading what
the OP wrote (and which you quoted above):
"""
The newest expiration date (field 5) for the record with
the state (field 1) which matches the desired state (field 6).
"""

and his code (indented a little less boisterously):

"""
#If the current record is the correct state
if current.split(" \t")[1] == current.split(" \t")[6]:
#If the existing record is the correct state
if best.split("\t" )[1] == best.split("\t" )[6]:
#If the new record has a newer exp. date
if current.split(" \t")[5] best.split("\t" )[5]:
"""

[William Heymann]

On Saturday 03 March 2007, Ben Finney wrote:

Bjoern Schliessmann <us************ **************@ spamgourmet.com writes:

if not recs.has_key(pi id): # [1]

Why not

if piid not in recs:

That is shorter, simpler, easier to read and very slightly faster. Plus you
can change the data structure of recs later without changing that line so
long as it implements containment testing.

[Ben Finney]

William Heymann <ko**@aesaeion. comwrites:

On Saturday 03 March 2007, Ben Finney wrote:

Bjoern Schliessmann <us************ **************@ spamgourmet.com writes:

if not recs.has_key(pi id): # [1]

Why not

if piid not in recs:

That is shorter, simpler, easier to read and very slightly faster.

Perhaps if I'd made my posting shorter, simpler, easier to read and
slightly faster, you might have read the footnote to which the '[1]'
referred.

--
\ "Choose mnemonic identifiers. If you can't remember what |
`\ mnemonic means, you've got a problem." -- Larry Wall |
_o__) |
Ben Finney

[Paddy]

On Mar 2, 10:44 pm, "Shawn Milo" <S...@Milochik. comwrote:

I'm new to Python and fairly experienced in Perl, although that
experience is limited to the things I use daily.

I wrote the same script in both Perl and Python, and the output is
identical. The run speed is similar (very fast) and the line count is
similar.

Now that they're both working, I was looking at the code and wondering
what Perl-specific and Python-specific improvements to the code would
look like, as judged by others more knowledgeable in the individual
languages.

Hi Shawn, there is a web page that gives examples from Perl's
Datastructures Cookbook re-implemented in Python. It might be of help
for future Python projects:
http://wiki.python.org/moin/PerlPhrasebook

- Paddy.

[Jussi Salmela]

Shawn Milo kirjoitti:

<snip>
I am not looking for the smallest number of lines, or anything else
that would make the code more difficult to read in six months. Just
any instances where I'm doing something inefficiently or in a "bad"
way.

I'm attaching both the Perl and Python versions, and I'm open to
comments on either. The script reads a file from standard input and
finds the best record for each unique ID (piid). The best is defined
as follows: The newest expiration date (field 5) for the record with
the state (field 1) which matches the desired state (field 6). If
there is no record matching the desired state, then just take the
newest expiration date.

I don't know if this attempt satisfies your criteria but here goes!

This is not a rewrite of your program but was created using your problem
description above. I've not included the reading of the data because it
has not much to do with the problem per se.

#============== =============== =============== =============== =
input = [
"aaa\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
"aaa\tAAA\t...\ t...\t...\t2007 0120\tAAA\n",
"aaa\tAAA\t...\ t...\t...\t2007 0101\tAAA\n",
"aaa\tAAA\t...\ t...\t...\t2007 1010\tBBB\n",
"aaa\tAAA\t...\ t...\t...\t2007 1111\tBBB\n",
"ccc\tAAA\t...\ t...\t...\t2007 1201\tBBB\n",
"ccc\tAAA\t...\ t...\t...\t2007 0101\tAAA\n",
"ccc\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
"ccc\tAAA\t...\ t...\t...\t2007 1212\tAAA\n",
"bbb\tAAA\t...\ t...\t...\t2007 0101\tAAA\n",
"bbb\tAAA\t...\ t...\t...\t2007 0101\tAAA\n",
"bbb\tAAA\t...\ t...\t...\t2007 1212\tAAA\n",
"bbb\tAAA\t...\ t...\t...\t2007 0612\tAAA\n",
"bbb\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
]

input = [x[:-1].split('\t') for x in input]
recs = {}
for row in input:
recs.setdefault (row[0], []).append(row)

for key in recs:
rows = recs[key]
rows.sort(key=l ambda x:x[5], reverse=True)
for current in rows:
if current[1] == current[6]:
break
else:
current = rows[0]
print '\t'.join(curre nt)
#============== =============== =============== =============== =
The output is:

aaa AAA ... ... ... 20070120 AAA
bbb AAA ... ... ... 20071212 AAA
ccc AAA ... ... ... 20071212 AAA

and it is the same as the output of your original code on this data.
Further testing would naturally be beneficial.

Cheers,
Jussi

[Bruno Desthuilliers]

John Machin a écrit :

On Mar 3, 12:36 pm, Bruno Desthuilliers >
[snip]

> DATE = 5
TARGET = 6

[snip]

>>Now for the bad news: I'm afraid your algorithm is broken : here are my
test data and results:

input = [
#ID STATE ... ... ... TARG DATE
"aaa\tAAA\t...\ t...\t...\tBBB\ t20071212\n",

[snip]

Bruno, The worse news is that your test data is broken.

Re-reading the OP's specs, the bad news is that my only neuron left is
broken. Shouldn't code at 2 o'clock in the morning :(

[Bruno Desthuilliers]

Bjoern Schliessmann a écrit :

Bruno Desthuilliers wrote:

>>Shawn Milo a écrit :

>> if recs.has_key(pi id) is False:

'is' is the identity operator - practically, in CPython, it
compares memory addresses. You *dont* want to use it here.

It's recommended to use "is None"; why not "is False"? Are there
multiple False instances or is False generated somehow?

Once upon a time, Python didn't have a "proper" boolean type. It only
had rules for boolean evaluation of a given object. According to these
rules - that of course still apply -, empty strings, lists, tuples or
dicts, numeric zeros and None are false in a boolean context. IOW, an
expression can eval to false without actually being the False object
itself. So the result of using the identity operator to test against
such an expression, while being clearly defined, may not be exactly what
you'd think.

To make a long story short:

if not []:
print "the empty list evals to false in a boolean context"

if [] is False:
print "this python interpreter is broken"

HTH

[Bruno Desthuilliers]

Shawn Milo a écrit :
(snip)

The script reads a file from standard input and
finds the best record for each unique ID (piid). The best is defined
as follows: The newest expiration date (field 5) for the record with
the state (field 1) which matches the desired state (field 6). If
there is no record matching the desired state, then just take the
newest expiration date.

Here's a fixed (wrt/ test data) version with a somewhat better (and
faster) algorithm using Decorate/Sort/Undecorate (aka schwarzian transform):

import sys
output = sys.stdout

input = [
#ID STATE ... ... ... DATE TARGET
"aaa\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
"aaa\tAAA\t...\ t...\t...\t2007 0120\tAAA\n",
"aaa\tAAA\t...\ t...\t...\t2007 0101\tAAA\n",
"aaa\tAAA\t...\ t...\t...\t2007 1010\tBBB\n",
"aaa\tAAA\t...\ t...\t...\t2007 1111\tBBB\n",
"ccc\tAAA\t...\ t...\t...\t2007 1201\tBBB\n",
"ccc\tAAA\t...\ t...\t...\t2007 0101\tAAA\n",
"ccc\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
"ccc\tAAA\t...\ t...\t...\t2007 1212\tAAA\n",
"bbb\tAAA\t...\ t...\t...\t2007 0101\tBBB\n",
"bbb\tAAA\t...\ t...\t...\t2007 0101\tBBB\n",
"bbb\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
"bbb\tAAA\t...\ t...\t...\t2007 0612\tBBB\n",
"bbb\tAAA\t...\ t...\t...\t2007 1212\tBBB\n",
]

def find_best_match (input=input, output=output):
PIID = 0
STATE = 1
EXP_DATE = 5
DESIRED_STATE = 6

recs = {}
for line in input:
line = line.rstrip('\n ')
row = line.split('\t' )
sort_key = (row[STATE] == row[DESIRED_STATE], row[EXP_DATE])
recs.setdefault (row[PIID], []).append((sort_ key, line))

for decorated_lines in recs.itervalues ():
print >output, sorted(decorate d_lines, reverse=True)[0][1]

Lines are sorted first on whether the state matches the desired state,
then on the expiration date. Since it's a reverse sort, we first have
lines that match (if any) sorted by date descending, then the lines that
dont match sorted by date descending. So in both cases, the 'best match'
is the first item in the list. Then we just have to get rid of the sort
key, et voilà !-)

HTH

[Peter Otten]

Bruno Desthuilliers wrote:

print >output, sorted(decorate d_lines, reverse=True)[0][1]

Or just
print >output, max(decorated_l ines)[1]

Peter