Just my curiosity.
Can python beats perl at speed of grep-like processing?
$ wget http://www.gutenberg.org/files/7999/7999-h.zip
$ unzip 7999-h.zip
$ cd 7999-h
$ cat *.htm bigfile
$ du -h bigfile
du -h bigfile
8.2M bigfile
---------- grep.pl ----------
#!/usr/local/bin/perl
open(F, 'bigfile') or die;
while(<F>) {
s/[\n\r]+$//;
print "$_\n" if m/destroy/oi;
}
---------- END ----------
---------- grep.py ----------
#!/usr/bin/env python
import re
r = re.compile(r'de stroy', re.IGNORECASE)
for s in file('bigfile') :
if r.search(s): print s.rstrip("\r\n" )
---------- END ----------
$ time perl grep.pl pl.out; time python grep.py py.out
real 0m0.168s
user 0m0.149s
sys 0m0.015s
real 0m0.450s
user 0m0.374s
sys 0m0.068s
# I used python2.5 and perl 5.8.6 4 4864
js wrote:
Just my curiosity.
Can python beats perl at speed of grep-like processing?
$ wget http://www.gutenberg.org/files/7999/7999-h.zip
$ unzip 7999-h.zip
$ cd 7999-h
$ cat *.htm bigfile
$ du -h bigfile
du -h bigfile
8.2M bigfile
---------- grep.pl ----------
#!/usr/local/bin/perl
open(F, 'bigfile') or die;
while(<F>) {
s/[\n\r]+$//;
print "$_\n" if m/destroy/oi;
}
---------- END ----------
---------- grep.py ----------
#!/usr/bin/env python
import re
r = re.compile(r'de stroy', re.IGNORECASE)
for s in file('bigfile') :
if r.search(s): print s.rstrip("\r\n" )
---------- END ----------
$ time perl grep.pl pl.out; time python grep.py py.out
real 0m0.168s
user 0m0.149s
sys 0m0.015s
real 0m0.450s
user 0m0.374s
sys 0m0.068s
# I used python2.5 and perl 5.8.6
I'm thankful for the Python version or else, I'd never have guessed what
that code was supposed to do!
Try that :
---------- grep.py ----------
#!/usr/bin/env python
import re
def main():
search = re.compile(r'de stroy', re.IGNORECASE). search
for s in file('bigfile') :
if search(s): print s.rstrip("\r\n" )
main()
---------- END ----------
js <eb*****@gmail. comwrote:
Just my curiosity.
Can python beats perl at speed of grep-like processing?
$ wget http://www.gutenberg.org/files/7999/7999-h.zip
$ unzip 7999-h.zip
$ cd 7999-h
$ cat *.htm bigfile
$ du -h bigfile
du -h bigfile
8.2M bigfile
#!/usr/local/bin/perl
open(F, 'bigfile') or die;
while(<F>) {
s/[\n\r]+$//;
print "$_\n" if m/destroy/oi;
}
#!/usr/bin/env python
import re
r = re.compile(r'de stroy', re.IGNORECASE)
for s in file('bigfile') :
if r.search(s): print s.rstrip("\r\n" )
$ time perl grep.pl pl.out; time python grep.py py.out
real 0m0.168s
user 0m0.149s
sys 0m0.015s
real 0m0.450s
user 0m0.374s
sys 0m0.068s
# I used python2.5 and perl 5.8.6
Playing for the other side temporarily, this is nearly twice as fast...
$ time perl -lne 'print if m/destroy/oi' bigfile >pl.out
real 0m0.133s
user 0m0.120s
sys 0m0.012s
vs
$ time ./z.pl >pl.out.orig
real 0m0.223s
user 0m0.208s
sys 0m0.016s
Which gives the same output modulo a few \r
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick
js a écrit :
Just my curiosity.
Can python beats perl at speed of grep-like processing?
Probably not.
>
$ wget http://www.gutenberg.org/files/7999/7999-h.zip
$ unzip 7999-h.zip
$ cd 7999-h
$ cat *.htm bigfile
$ du -h bigfile
du -h bigfile
8.2M bigfile
---------- grep.pl ----------
#!/usr/local/bin/perl
open(F, 'bigfile') or die;
while(<F>) {
s/[\n\r]+$//;
print "$_\n" if m/destroy/oi;
}
---------- END ----------
---------- grep.py ----------
#!/usr/bin/env python
import re
r = re.compile(r'de stroy', re.IGNORECASE)
for s in file('bigfile') :
if r.search(s): print s.rstrip("\r\n" )
---------- END ----------
Please notice that you're also benchmarking IO here - and perl seems to
use a custom, highly optimized IO lib, that is much much faster than the
system's one. I once made a Q&D cat-like comparison of perl, Python and
C on my gentoo-linux box, and the perl version was insanely faster than
the C one.
Now the real question is IMHO: is the Python version fast enough ?
My 2 cents..
Nick Craig-Wood wrote:
> #!/usr/bin/env python import re r = re.compile(r'de stroy', re.IGNORECASE)
for s in file('bigfile') : if r.search(s): print s.rstrip("\r\n" )
footnote: if you're searching for literal strings with Python 2.5, using "in" is a
lot faster than using re.search.
</F> This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Xah Lee |
last post by:
© # -*- coding: utf-8 -*-
© # Python
©
© # the "filter" function can be used to
© # reduce a list such that unwanted
© # elements are removed.
© # example:
©
© def even(n): return n % 2 == 0
© print filter( even, range(11))
|
by: Studentmadhura05 |
last post by:
Hi,
I recently started using bash and wrote some code to find certain IPs and related stuff in the leases files by using grep
I am trying to switch to perl. I am a bigginer and trying to learn perl by reading stuff on net.
I am getting stuck at following point
I wrote in bash
grep $IP data061009 -A 6
Now in perl I am writing
@result= grep (/$IP/, $_) in a while loop.
How do I incorporate '-A 6' part in perl
|
by: None |
last post by:
Hi,
I was trying to use SWIG to expose some C++ functions to perl. But, I
ran into some issues here. I don't know why, but I see that a macro in
SWIG replaces "NORMAL" with "PL_op->op_next"...
I ran the following test:
Files:
|
by: fox |
last post by:
I need to extract patterns from a line in a web page and these patterns sometimes show up twice in the same line so using grep with the pattern only grabs one.
Exaple is I need
<td width="30%" class="navtext">Sample1</td><width="7%">20</td>
<td width="30%" class="navtext">Sample2</td><td width="7%">18</td>
extracted from the bottom code line.
My final result wants to be
Sample1 20
|
by: gyandwivedi |
last post by:
what should I use in perl /in place of grep -v in unix.
| |
by: jonniethecodeprince |
last post by:
I've found a program and interpreter for PERL 5.8 so to everyone who tried to help me with that thank you.
Now my attention turns to the GREP function in PERL. From what I understand GREP is a simple pattern matching function that searches for either the full element in a string or a first letter of an element, but if anyone can clarify this that would be great :)
@lastNames = ('Sutherland','Summers','Sunley','Duran','Grieve')...
|
by: ravir |
last post by:
Hi,
I am new to this group. I am working in Perl and shellscripts.
I have a clarification regarding perl grep and pattern matching.
I am writing a perl script to automate the process of code review upto some level so that the reviewer and developer's time can be reduced during the code review. In my script, I am using pattern matching and grep in many places.
So, my clarification is : Is it fine to make use of grep to find a pattern in a...
|
by: seberino |
last post by:
How similar is Python's re module (regular expressions) compared
to Perl's and grep's regular expression syntaxes?
I really hope regular expression syntax is sufficiently standardized
that
we don't have to learn new dialects everytime we move from one
language or shell command to another.
chris
|
by: hassan470 |
last post by:
I work in an investment bank. We have a huge log file which contains FIX messages. I have a problem to grep FIX messages within a time rage (i.e. between 9:30 am to 11 pm, those were sent by CITIBANK for ticker symbol MSFT). I tried different ways but it did not work. Could anyone please help me?
Here is one of the codes in perl:
#!/usr/bin/perl –w
$myfile = “/tmp/fixlog.txt”;
open(FH, $myfile) || die “Can not open”;
while<FH>{...
|
by: pradeeps |
last post by:
when i run my code it's occurred an error
my code...
sub starting {
$out1=`perl macswitch.pl`;
foreach ($out1){
chomp;
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |