473,239 Members | 1,659 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,239 software developers and data experts.

Erasing or Skipping lines in a data file

3
Hi there,
I just started programming with PERL and am trying to put together my first little data manipulation program. I am working on a MAC with OSX.

I have a data file with the following header that has been created on a Windows XP machine:

------ Begin Next Fly'm ------


"Begin Fly'm (Thu Jul 24 01:22:02 2008)"
"Ions Flown Separately, Comp Quality(100)"
"Number of Ions to Fly = 200000"
"Changes:","Mass","Charge","X","Y","Z","KE","Azm", "Elv","Time of Birth"
" ","YES","NO ","NO ","NO ","NO ","YES","YES","YES","NO "


"Ion N","Events","TOF","Mass","X","KE","KE Error"

1,1,0,2.01402,9.53,1,1.28051e-009
1,4,0.725546,2.01402,178,3181.59,0.542952


My goal is to get rid of the header and the empty lines to finally have an output file with only the number entries.

I've put together some lines that -IMO- should work, but they don't and I don't really get it why they don't do their work.


Expand|Select|Wrap|Line Numbers
  1. # read file that is given in STDIN
  2. $dat = shift or die "Need input file with data. \n";
  3.  
  4. # open files for read and for write
  5. open DAT,"< $dat" or die "Cannot read $dat\n";
  6. open OUT, "> output.txt" or die "Cannot open file!\n";
  7.  
  8.  
  9. # loop for reading input file
  10. while($dat = <DAT>){
  11.  
  12. # skip lines beginning with ", - and empty
  13.      next if $dat =~ /^\s*$|\"|-/;
  14.  
  15.     # next if $dat =~ /^\"/; 
  16.     # s/^-//;
  17.     # s/^"//;
  18.     #s/^\s*//;
  19.      chomp($dat);   
  20.  
  21. print OUT <DAT>; #print the contents of the input file into the ouput file
  22. }
  23.  
  24. # close files for reading and writing
  25. close DAT;
  26. close OUT;

The bold part is the line where the skipping /erasing should take place. I tried several different combinations of the command, put m/ before the expression and /i after and even had it split into three seperate commands that should skip ", - and whitespace:

Expand|Select|Wrap|Line Numbers
  1. next if $dat =~ /^\"/; 
for just erasing the " and the same for - and whitespace.

My output file looks like this when I run the program:



"Begin Fly'm (Thu Jul 24 01:22:02 2008)"
"Ions Flown Separately, Comp Quality(100)"
"Number of Ions to Fly = 200000"
"Changes:","Mass","Charge","X","Y","Z","KE","Azm", "Elv","Time of Birth"
" ","YES","NO ","NO ","NO ","NO ","YES","YES","YES","NO "


"Ion N","Events","TOF","Mass","X","KE","KE Error"

1,1,0,2.01402,9.53,1,1.28051e-009
1,4,0.725546,2.01402,178,3181.59,0.542952
There are some empty lines in the beginning and then the usual header minus the very first line.

I even tried to completely erase the header using the commands in line 16 and 17 of my code, but usually I get the following error for that attempt:

Use of uninitialized value in substitution (s///) at ./foo.pl line 16, <DAT> line 1.
Use of uninitialized value in substitution (s///) at ./foo.pl line 17, <DAT> line 1.


The skipping of lines worked to some extent with a simple little textfile I made up (except for getting rid of the empty lines), but it more or less fails when I try the same on my datafile.

Hopefully someone can help me with this problem.

Thanks a lot.
Aug 5 '08 #1
4 7178
BibI
3
I just saw that the very first line of my data file gets erased whenever I run the program. No idea how this can happen.
Aug 5 '08 #2
nithinpes
410 Expert 256MB
I just saw that the very first line of my data file gets erased whenever I run the program. No idea how this can happen.

Modify the while() loop as below:
Expand|Select|Wrap|Line Numbers
  1. while($dat = <DAT>){
  2.  # skip lines beginning with ", - and empty
  3.      next if $dat =~ /(^\s*$)|(^\")|(^-)/;
  4.      print OUT $dat; #print the contents of the input file into the ouput file
  5. }
  6.  
The regex is changed to suit your need (skip lines beginning with ", - and empty), the regex you used would skip blank lines, lines containing " and - (not just the lines beginning with).
The chomp() line was removed to obtain lines of output. If you want all the lines having numbers in a single line output, you can include that.
The first line of data was getting removed because of this:

Expand|Select|Wrap|Line Numbers
  1. print OUT <DAT>;
  2.  
When you have already assigned <DAT> to $dat, using <DAT> again will read the next line and print it. You should be using $dat in this line.
Aug 6 '08 #3
KevinADC
4,059 Expert 2GB
Expand|Select|Wrap|Line Numbers
  1. while(my $dat = <DAT>) {
  2.    next if ($dat =~ /^([-"])|^\s*$/);
  3.    print OUT $dat;
  4. }    
Aug 6 '08 #4
BibI
3
Hey thanks very much.

That totally solved my problem. :)
Aug 6 '08 #5

Sign in to post your reply or Sign up for a free account.

Similar topics

5
by: Sugapablo | last post by:
I know how to read a text file in ASP, I know how to append lines to a text file. How do I simply erase all lines from a text file? -- http://www.sugapablo.com <--music ]
11
by: eeykay | last post by:
Hello, I am facing a starnge problem while erasing the last member in a vector. I am using VC++ .NET 2002 complier. I have vector of CComPtr<..> (irrelevant here), and then I iterate over the...
4
by: darrel | last post by:
I'm backtracking to a problem I had a month or so ago. I need to write XML files quite a bit. I'm finding that the way I'm doing it doesn't write a new, clean XML file each time, but just dumps...
3
by: Ivan Liu | last post by:
I would like know how I can skip a line while reading a set of input data (from a text file) if the first character of the line is "#". My original code reads: ifstream Infile("data.dat"); ...
6
by: lisa.engblom | last post by:
Hi, I've just started programming in python, and have run into an unexpected problem. I am using python to pull text data from some csv files. I have one file that has the important...
3
by: =?Utf-8?B?UmF5IE1pdGNoZWxs?= | last post by:
I'm drawing text using the DrawString method. I need a way to go back and erase one or more characters. As a first thought it seemed that one way to do it would be to go back to the character(s)...
3
by: Anthony1312002 | last post by:
Hello. I have a scipt the reads and imports a text file into a database table. Below is an example of the text file I'm importing and the script I'm using to accomplish the import. You'll notice at...
7
by: Gustaf | last post by:
Hi all, Just for fun, I'm working on a script to count the number of lines in source files. Some lines are auto-generated (by the IDE) and shouldn't be counted. The auto-generated part of files...
0
by: Jerry Coffin | last post by:
In article <4fae62b0-6858-4e9e-830e-9eecf6691d4a@ 59g2000hsb.googlegroups.com>, friend.blah@googlemail.com says... Each time you read from the file, keep track of the file position after...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.