By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,848 Members | 1,319 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,848 IT Pros & Developers. It's quick & easy.

perl memory usage

P: 1
Hello,

This is really a reply to
http://www.thescripts.com/forum/thread50187.html

I have a script that reads in a bunch of data from txt files, then does a bunch of calculations, and outputs other txt files. I've got about 7 input txt files, each of which is 14 - 17 MB. If I open Windows Task Manager's process list I can watch Perl using increasing amounts of memory as it reads in the txt files until it chokes, at about 1.7GB (I have 2GB on my machine), which is about the 6th input txt file.

Below is a typical code block showing how I'm reading in data. Can anybody suggest a way to do the same thing without using as much memory? If not, what's the easiest way for me to tell what in my script is taking all the memory? Any help is greatly appreciated.

Expand|Select|Wrap|Line Numbers
  1. open TRAIL_FILE, "perl-inputs/fire_trails.asc.txt";
  2. my $counter = 0;
  3. while (<TRAIL_FILE>) {
  4.    chomp;
  5.    @{$trail_lines{$counter}} = split /\s+/,$_;
  6.    $counter++;
  7. }
  8. close TRAIL_FILE;
  9.  
  10. for (my $row = 6; $row < $nrows+6; $row++) {
  11.     for (my $col = 0; $col < $ncols; $col++) {
  12.         $trails{$row}{$col} = @{$trail_lines{$row}}[$col];
  13.     } # close for on $col
  14. } # close for on $row
  15.  
Feb 7 '08 #1
Share this Question
Share on Google+
2 Replies


P: 58
Can anybody suggest a way to do the same thing without using as much memory? If not, what's the easiest way for me to tell what in my script is taking all the memory?
All your memory is being used up because you're storing all the information you're reading. What does your data look like?
Feb 7 '08 #2

KevinADC
Expert 2.5K+
P: 4,059
Its the data you read into %trails (or whatever your data structure is) that is taking up memory. Maybe if you explain what you are doing someone can suggest a different way that might save memory.
Feb 7 '08 #3

Post your reply

Sign in to post your reply or Sign up for a free account.