473,320 Members | 2,158 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

forkmanager takes too long when I set children

Hi all,

I've made the script below to calculate the coverage per position of small pieces of strings. It works fine but when I tried to speed thing up for large files using forkmanager the script takes an awful lot of time when I set 2 parallel forks. Setting it to 0 works fine. Does anybody have an idea what is going wrong?

Kind Regards
Jaap


Expand|Select|Wrap|Line Numbers
  1. #!/usr/local/bin/perl -w
  2. use Parallel::ForkManager;
  3. use strict;
  4.  
  5. if(scalar(@ARGV) != 3){
  6.     print "Please use the correct parameters\nUsage: bowtie2wiggle.pl -infile -outfile- -processes-\n";
  7.     exit();
  8. }
  9.  
  10. my $pm = new Parallel::ForkManager($ARGV[2]); 
  11. my %hash=();
  12. open (IN, $ARGV[0]);
  13. while (<IN>){
  14.     $pm->start and next;
  15.     $_ =~ s/\n|\r//;
  16.     my @element=split(/\t/,$_);
  17.     $element[2] =~ s/T//;
  18.     for (my $i=1;$i<=length($element[4]);$i++){
  19.         $hash{$element[2]}{($element[3]+$i)}{coverage}++;
  20.     }
  21.     $pm->finish;
  22. }
  23. $pm->wait_all_children;
  24. close IN;
  25.  
  26. open (OUT, ">$ARGV[1]");
  27. print OUT 'track type=wiggle_0 name="'.$ARGV[0].'" description="theoretical coverage from '.$ARGV[0].'" visibility=full autoScale=on color=50,150,255'."\n";
  28.  
  29. my $old="";
  30. foreach my $chromosome (sort keys %hash){
  31.     print OUT "variableStep chrom=chr$chromosome span=1\n";
  32.     foreach my $position (sort {$a<=>$b} keys %{$hash{$chromosome}}){
  33.         if ($old ne "" && $old != ($position-1)){
  34.             print OUT "variableStep chrom=chr$chromosome span=1\n";
  35.         }
  36.         print OUT "$position $hash{$chromosome}{$position}{coverage}\n";
  37.         $old=$position;
  38.     }
  39. }
  40. close OUT;
Sep 23 '10 #1
1 2152
numberwhun
3,509 Expert Mod 2GB
I would help you if I had experience with forkmanager, but I have not used it. Hopefully someone will be able to help you.
Oct 8 '10 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

0
by: ptouch | last post by:
Hi, I am using Python 2.3.4 with Windows 2000. It is a Pentium Xeon 2GHz with 1GB of RAM. As the title says, importing most modules takes about 2 to 3 seconds. Also, starting the interpreter...
4
by: Karaoke Prince | last post by:
Hi There, I have an update statement to update a field of a table (~15,000,000 records). It took me around 3 hours to finish 2 weeks ago. After that no one touched the server and no...
1
by: Lee Benfield | last post by:
Anyone seen the following odd behaviour? (not found it in FAQs etc) Create a simple project, two forms, first is MDI parent with a MenuStrip, (File->New and Windows menu items). Set...
0
by: JJ | last post by:
Hi, I'm having a little problem. I use the following code to open a .pdf file from one app. : System.Diagnostics.ProcessStartInfo psi = new System.Diagnostics.ProcessStartInfo(); string path...
0
by: Erwin Heimgartner | last post by:
Adding a member variable in a Dialog class may take a long time in Visual Studio .NET 2003 (MFC application). An interesting finding is that when I disable the LAN connection, the "Add member...
4
by: Brian | last post by:
Hello, I have a vb.net web app that I am doing reads and adds to a database. Everything works fine and happens instantly. I recently added an update and it takes a long time to finish. I was...
1
by: William Sullivan | last post by:
I've got a website that may, on occasion, display a large list of items in a bulletedlist control. On the client side, it takes about 4 seconds to get a response that weighs in at over 1mb. It...
0
by: John Smith | last post by:
Hello, I have 7 different crystal reports that need to be collated. Since I want to end up with a page of each (which all together make a single report), I created a blank main report and then...
2
by: Jason Huang | last post by:
Hi, I am wondering why it takes a long time to open an .aspx file at the first time in a browser. Are there some methods to improve that speed? Given that we are using the IIS5+VS2005 C#....
4
by: parag_paul | last post by:
hi All I understand the need for long long , but what is the purpose of long as a data type separately. Just makes the language intimidating to start with, when you have to deal with so many data...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.