By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
437,661 Members | 1,330 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 437,661 IT Pros & Developers. It's quick & easy.

to extract certain column of data from a number to files.

P: 3
I am new to perl scripting. I am having some problem to write a program.
I have a number of files containing same type of data with same header.

File is like this.

SN. Cities temperature Humidity rainfall
1 abc 33 66 23
2 ghi 36 83 12
3 xyz 23 78 11
......

I want to extract temperature and humidity for all cities from each file to a new file.
File names are like 1.txt, 2.txt, 3.txt, 4.txt.........
Please help me to solve this problem.
Thanks in advance.
~NaraN
Oct 30 '08 #1
Share this Question
Share on Google+
8 Replies


nithinpes
Expert 100+
P: 410
I am new to perl scripting. I am having some problem to write a program.
I have a number of files containing same type of data with same header.

File is like this.

SN. Cities temperature Humidity rainfall
1 abc 33 66 23
2 ghi 36 83 12
3 xyz 23 78 11
......

I want to extract temperature and humidity for all cities from each file to a new file.
File names are like 1.txt, 2.txt, 3.txt, 4.txt.........
Please help me to solve this problem.
Thanks in advance.
~NaraN
What have you tried so far?
If the source data is in excel file, you can make use of Spreadsheet::ParseExcel and Spreadsheet::WriteExcel. If it is text file, then you should be able to do it using split() function.

However, we will not be able to help you unless you post your code.
Oct 30 '08 #2

P: 65
Hi there,

did u see this post:

http://bytes.com/forum/thread845672.html

i guess, your problem will get resolve if you try it out..

But as said by nitinpes, you should try scripting yourself.if get struck then post the code here..people will help you out.

-Vijayarl
Oct 30 '08 #3

crazy4perl
P: 20
Hi !!!
don't know whether you got ur solution or not. just got some free time so tried to solve your problem. As i m also a beginner, probably this is not the best way, there must be much better ways to do the same thing which someone more experienced can suggest.
Here is my peice of code:-

Expand|Select|Wrap|Line Numbers
  1. use strict;
  2. use warnings;
  3. use FileHandle;
  4. use Fcntl qw(:flock);
  5.  
  6. my $file = shift;
  7.  
  8. my $openmode = '+<';
  9. my ($fh, $eof,$line, $linenr, $header, $fields);
  10. my (@fields, @header);
  11.  
  12. if (!open($fh, $openmode, $file)) {
  13.     die("Unable to open file '$file': $!");
  14. }
  15.  
  16. if (!flock($fh, LOCK_EX | LOCK_NB)) { # Get an exclusive non-blocking lock
  17.     # We didn't get the lock
  18.     die("File $file locked by another process. Skipping it");
  19.     close($fh);
  20. }
  21.  
  22. print("Processing $file ");
  23.  
  24. $eof = 0;
  25.  
  26. while(!$eof) {
  27.  
  28.     # Read another line of input
  29.     $line = $fh->getline();
  30.     if (!defined($line)) {
  31.         # Reached EOF
  32.         $eof = 1;
  33.  
  34.         last;
  35.     }
  36.  
  37.     $linenr = $fh->input_line_number();
  38.  
  39.     # Stript EOL chars from line
  40.     $line =~ s/\r?\n$//s;
  41. #    $line =~ s/ //g;
  42.     if ($line =~ /^\s*$/) {
  43.         # Blank line
  44.         print "Skipping blank line $linenr\n";
  45.         next;
  46.     } elsif ($line =~ /^#/) {
  47.         # Comment line
  48.         print "Skipping comment line $linenr\n";
  49.         next;
  50.     } elsif ($linenr == 1) {
  51.         # Header line
  52.         print "Processing header line #$linenr\n";
  53.         @header = split / /,$line;
  54.         print("\n @header \n");
  55.         next;
  56.     }
  57.  
  58.     @fields = split / /,$line;
  59.     print("\n  @fields  \n");
  60.  
  61. }
  62.  
  63.  
Oct 30 '08 #4

P: 3
hey friends,

Thanks a lot for ur replies. I will try according to you people..
I will post the result as soon as i get some result.

presently, i just can filter the certain column of data using split function but they have to be concatenated similar datas( just like adding columns in excel) from datas from multiple files.

Result shuld be like:
SN Places Temp humidy date temp humidity date temp humidiy date.........
1 abc 33 97 Oct1 34 98 Oct2 32 97 Oct3
2 xyz 23 88 Oct1 22 89 Oct2 21 86 Oct3
.......
Thanks again,
Regards
~NaraN
Oct 31 '08 #5

nithinpes
Expert 100+
P: 410
presently, i just can filter the certain column of data using split function but they have to be concatenated similar datas( just like adding columns in excel) from datas from multiple files.

Result shuld be like:
SN Places Temp humidy date temp humidity date temp humidiy date.........
1 abc 33 97 Oct1 34 98 Oct2 32 97 Oct3
2 xyz 23 88 Oct1 22 89 Oct2 21 86 Oct3
.......
Thanks again,
Regards
~NaraN
Post your code here, so that someone can help you out.
Oct 31 '08 #6

numberwhun
Expert Mod 2.5K+
P: 3,503
crazy4perl,

While I understand that you are probably anxious to help, please be sure and read the posts in any thread you visit. One of our experts (Nithinpes) had asked the OP what they have tried and to post their code. This is a learning forum and just providing someone answers as you have does not help them learn. Also, in doing so you may be giving someone the answers to their homework, which is against site policy.

I am not saying don't help, but next time please wait until they show their code and see if you can help with that first.

Regards,

Moderator
Oct 31 '08 #7

P: 3
Thanks nithinpes and crazy4perl.
I have written a simple code that can extract required columns from a simple file. I am just trying to get some output and this works for a file.

$myfile=$ARGV[0];
$myfile1=$ARGV[1];

open (DATA, "<$myfile") || die "Can't open $myfile $!";
while (<DATA> )
{
@x= split('\t');

if ($x[0] >= 1) {

print STDOUT "$x[0]\t$x[1]\t$x[2]\n" ;
}}

close DATA;

exit(0);

And i execute this ...perl perlscript.pl 1.txt >out.txt

Now my problem is: Can i open another file and append the output columns (x[1] and [2] from another file 2.txt) in out.txt in each line ...? Or other good ways to do it efficiently? Should i open all the files simultaneously and do my work. Can you suggest me please.

Thanks in advance
regards,
~NaraN
Nov 1 '08 #8

KevinADC
Expert 2.5K+
P: 4,059
Tie::File allows you to easily edit a file. Perls builtin editor (-i command line or $^I in a script) can also be used to edit files inplace. You can also open two files for reading/input and a third file to output the new results to.
Nov 1 '08 #9

Post your reply

Sign in to post your reply or Sign up for a free account.