473,748 Members | 8,376 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

to extract certain column of data from a number to files.

3 New Member
I am new to perl scripting. I am having some problem to write a program.
I have a number of files containing same type of data with same header.

File is like this.

SN. Cities temperature Humidity rainfall
1 abc 33 66 23
2 ghi 36 83 12
3 xyz 23 78 11
......

I want to extract temperature and humidity for all cities from each file to a new file.
File names are like 1.txt, 2.txt, 3.txt, 4.txt.........
Please help me to solve this problem.
Thanks in advance.
~NaraN
Oct 30 '08 #1
8 4670
nithinpes
410 Recognized Expert Contributor
I am new to perl scripting. I am having some problem to write a program.
I have a number of files containing same type of data with same header.

File is like this.

SN. Cities temperature Humidity rainfall
1 abc 33 66 23
2 ghi 36 83 12
3 xyz 23 78 11
......

I want to extract temperature and humidity for all cities from each file to a new file.
File names are like 1.txt, 2.txt, 3.txt, 4.txt.........
Please help me to solve this problem.
Thanks in advance.
~NaraN
What have you tried so far?
If the source data is in excel file, you can make use of Spreadsheet::Pa rseExcel and Spreadsheet::Wr iteExcel. If it is text file, then you should be able to do it using split() function.

However, we will not be able to help you unless you post your code.
Oct 30 '08 #2
vijayarl
65 New Member
Hi there,

did u see this post:

http://bytes.com/forum/thread845672.html

i guess, your problem will get resolve if you try it out..

But as said by nitinpes, you should try scripting yourself.if get struck then post the code here..people will help you out.

-Vijayarl
Oct 30 '08 #3
crazy4perl
20 New Member
Hi !!!
don't know whether you got ur solution or not. just got some free time so tried to solve your problem. As i m also a beginner, probably this is not the best way, there must be much better ways to do the same thing which someone more experienced can suggest.
Here is my peice of code:-

Expand|Select|Wrap|Line Numbers
  1. use strict;
  2. use warnings;
  3. use FileHandle;
  4. use Fcntl qw(:flock);
  5.  
  6. my $file = shift;
  7.  
  8. my $openmode = '+<';
  9. my ($fh, $eof,$line, $linenr, $header, $fields);
  10. my (@fields, @header);
  11.  
  12. if (!open($fh, $openmode, $file)) {
  13.     die("Unable to open file '$file': $!");
  14. }
  15.  
  16. if (!flock($fh, LOCK_EX | LOCK_NB)) { # Get an exclusive non-blocking lock
  17.     # We didn't get the lock
  18.     die("File $file locked by another process. Skipping it");
  19.     close($fh);
  20. }
  21.  
  22. print("Processing $file ");
  23.  
  24. $eof = 0;
  25.  
  26. while(!$eof) {
  27.  
  28.     # Read another line of input
  29.     $line = $fh->getline();
  30.     if (!defined($line)) {
  31.         # Reached EOF
  32.         $eof = 1;
  33.  
  34.         last;
  35.     }
  36.  
  37.     $linenr = $fh->input_line_number();
  38.  
  39.     # Stript EOL chars from line
  40.     $line =~ s/\r?\n$//s;
  41. #    $line =~ s/ //g;
  42.     if ($line =~ /^\s*$/) {
  43.         # Blank line
  44.         print "Skipping blank line $linenr\n";
  45.         next;
  46.     } elsif ($line =~ /^#/) {
  47.         # Comment line
  48.         print "Skipping comment line $linenr\n";
  49.         next;
  50.     } elsif ($linenr == 1) {
  51.         # Header line
  52.         print "Processing header line #$linenr\n";
  53.         @header = split / /,$line;
  54.         print("\n @header \n");
  55.         next;
  56.     }
  57.  
  58.     @fields = split / /,$line;
  59.     print("\n  @fields  \n");
  60.  
  61. }
  62.  
  63.  
Oct 30 '08 #4
NaraN
3 New Member
hey friends,

Thanks a lot for ur replies. I will try according to you people..
I will post the result as soon as i get some result.

presently, i just can filter the certain column of data using split function but they have to be concatenated similar datas( just like adding columns in excel) from datas from multiple files.

Result shuld be like:
SN Places Temp humidy date temp humidity date temp humidiy date.........
1 abc 33 97 Oct1 34 98 Oct2 32 97 Oct3
2 xyz 23 88 Oct1 22 89 Oct2 21 86 Oct3
.......
Thanks again,
Regards
~NaraN
Oct 31 '08 #5
nithinpes
410 Recognized Expert Contributor
presently, i just can filter the certain column of data using split function but they have to be concatenated similar datas( just like adding columns in excel) from datas from multiple files.

Result shuld be like:
SN Places Temp humidy date temp humidity date temp humidiy date.........
1 abc 33 97 Oct1 34 98 Oct2 32 97 Oct3
2 xyz 23 88 Oct1 22 89 Oct2 21 86 Oct3
.......
Thanks again,
Regards
~NaraN
Post your code here, so that someone can help you out.
Oct 31 '08 #6
numberwhun
3,509 Recognized Expert Moderator Specialist
crazy4perl,

While I understand that you are probably anxious to help, please be sure and read the posts in any thread you visit. One of our experts (Nithinpes) had asked the OP what they have tried and to post their code. This is a learning forum and just providing someone answers as you have does not help them learn. Also, in doing so you may be giving someone the answers to their homework, which is against site policy.

I am not saying don't help, but next time please wait until they show their code and see if you can help with that first.

Regards,

Moderator
Oct 31 '08 #7
NaraN
3 New Member
Thanks nithinpes and crazy4perl.
I have written a simple code that can extract required columns from a simple file. I am just trying to get some output and this works for a file.

$myfile=$ARGV[0];
$myfile1=$ARGV[1];

open (DATA, "<$myfile") || die "Can't open $myfile $!";
while (<DATA> )
{
@x= split('\t');

if ($x[0] >= 1) {

print STDOUT "$x[0]\t$x[1]\t$x[2]\n" ;
}}

close DATA;

exit(0);

And i execute this ...perl perlscript.pl 1.txt >out.txt

Now my problem is: Can i open another file and append the output columns (x[1] and [2] from another file 2.txt) in out.txt in each line ...? Or other good ways to do it efficiently? Should i open all the files simultaneously and do my work. Can you suggest me please.

Thanks in advance
regards,
~NaraN
Nov 1 '08 #8
KevinADC
4,059 Recognized Expert Specialist
Tie::File allows you to easily edit a file. Perls builtin editor (-i command line or $^I in a script) can also be used to edit files inplace. You can also open two files for reading/input and a third file to output the new results to.
Nov 1 '08 #9

Sign in to post your reply or Sign up for a free account.

Similar topics

7
3836
by: Greg Brunet | last post by:
I'm writing some routines for handling dBASE files. I've got a table (DBF file) object & field object already defined, and after opening the file, I can get the field info like this: >>> tbl.Fields() What I would like to do is be able to extract the field names into a single, separate list. It should look like:
1
3453
by: Ori | last post by:
Hi, I have a HTML text which I need to parse in order to extract data from it. My html contain a table contains few rows and two columns. I want to extract the data from the 2nd column in the most efficient way (using Reg Ex.) either than using the "indexOf" function of String. Thanks,
9
3391
by: Brian Hanson | last post by:
Hi, I have an unusual problem that just showed its ugly head at a pretty bad time. I have an asp.net (VB) app that takes data from an Excel sheet and puts it into SQL Server. I get the data out of Excel using OleDB, and suddenly, some of the data was not being extracted from Excel. I use OleDb for the extract into a DataTable and from there an SqlClient.SqlCommand to put it into SQL Server.
1
11893
by: vkrishn | last post by:
Hello i am extememly new to PERL and have a very basic question. Say i have two columns in a text file something like this... 5330 1 5380 5 5390 6 5400 9 5410 11 5420 15
8
2840
by: Fabian Braennstroem | last post by:
Hi, I would like to remove certain lines from a log files. I had some sed/awk scripts for this, but now, I want to use python with its re module for this task. Actually, I have two different log files. The first file looks like: ...
10
2441
by: AA Arens | last post by:
I do have a database with customer info in it. To avoid it will be taken out of our office, is it possible to make it not-readable after a certain period? then every let say seven days, I needs to "extend the license", so it will last another week. It consists of queries, forms, and tables, format Access 2003. Bart
5
2849
by: =?Utf-8?B?aWxy?= | last post by:
Hi This is probably fairly simple but I am newish at programming and was wondering if someone can give me some advice on handling the following. I have an array with a large number of elements in it. 0-9 are related data, 10-19, 20-29 are related and so on. What is the best way of extracting groups of elements from the array into another array where each element is the related data or to extract say elements 0,1,5 from the first...
7
2113
by: JoeC | last post by:
I am trying to create a windows program that reads binary graphics as a resource. This has nothing to do with win32 but conversion of data with memcpy. graphic::graphic(UINT uiResID, HINSTANCE hinstance){ size = 32; bitData.clear(); void * p = NULL; // point to the data int end; BYTE data;
2
4686
by: MaryJolly | last post by:
I want to extract number part from a string. I am doing project in VB6 and my MSFlexGrid control contains certain codes in the first column.eg. EC101,CS104,etc. I want to extract the number part from the text.
0
8991
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8830
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9541
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9370
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8242
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6796
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6074
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4874
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
3
2215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.