473,386 Members | 1,708 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

discarding new lines from my file

112 100+
I have a file that has lines that are deliminated with '^A', but some of the lines go for a few lines and I need those lines to be appended into one line.

All of the lines start with 'low debug' and end with ' " 0 '.

How can I read each line from start to finish without some of the data being screwed up because the middle of the line starts at a new line.

Here are some examples:
This is an example of a line that is correct from beginning to end:
Expand|Select|Wrap|Line Numbers
  1. low debug 2010/4/1 9:00:33.40 ICSNotificationAlarm Prodics01ics0001 ICS "1.0^AB^A7611^A1270130433400^A1965^A1963^A2^Am[0]=801^A10635^AProdfixcas11v2fix61^ACD
  2. 890/CM/ConsumerProxy@29002971^Am[1]=19002^A10635^AProdfixcas11v2fix61^ACD890/CM/ConsumerProxy@29002971^A" 0
  3.  
Notice how the line is deliminated with ^A. I use those fields in the data that I need to capture.

Here is an example of one that is broken up into a few new lines which does not help because when I try to search for $7(the 7th value), my data gets all screwed up because that value is on a new line and since the value number starts over each time a new line begins, everything gets messed up.

Expand|Select|Wrap|Line Numbers
  1. low debug 2010/4/1 9:00:37.38 ICSNotificationAlarm Prodics01ics0001 ICS "1.0^AB^A7612^A1270130437380^A31829^A31827^A1^Am[0]=31806^A[[(ProdBC10x1OHServerHybri
  2. dprdbc10b) ]] OhsSender(W_MAIN.ProdBC10x1HybridTradeServer4.Sender): : Exception while sending data to OHS.
  3. -------- Exception --------
  4. org.omg.CORBA.TIMEOUT: Request timed out after 10000065563 ns, falseNotifies(0). RequestId(2), typeId(IDL:orderHandlingServices/ManualOrderMaintenanceService
  5. :1.0), operation(getQuoteQueryDataForProducts), orbName(ProdBC10x1OHServerHybridprdbc10b), iiopHost(prdbc10b), iiopPort(18202), tiopHost(prdbc10b), tiopPort(
  6. 18701)  vmcid: 0x0  minor code: 0 completed: Maybe
  7. -------- Trace ------------
  8. com.cboe.ORBInfra.Transport.PendingRequestManager.waitForCompletion(PendingRequestManager.java:133)
  9. com.cboe.IIOPTransport.IIOPClientConnection.receiveReply(IIOPClientConnection.java:529)
  10. com.cboe.IIOPTransport.IIOPClientConnection.receiveReply(IIOPClientConnection.java:513)
  11. com.cboe.IIOPTransport.IIOPClientConnection.receiveReply(IIOPClientConnection.java:507)
  12. com.cboe.ORBInfra.ORB.GenericBindMediator.twoWaySendAndReceive(GenericBindMediator.java:259)
  13. com.cboe.ORBInfra.ORB.DelegateImpl.invoke(DelegateImpl.java:693)
  14. org.omg.CORBA.portable.ObjectImpl._invoke(ObjectImpl.java:457)
  15. com.cboe.idl.orderHandlingServices._ManualOrderMaintenanceServiceStub.getQuoteQueryDataForProducts(_ManualOrderMaintenanceServiceStub.java:519)
  16. com.cboe.ohsadapters.par.orderHandling.ParManualOrderMaintenanceServiceProxyImpl.getQuoteQueryDataForProducts(ParManualOrderMaintenanceServiceProxyImpl.java:
  17. 325)
  18. com.cboe.ohsadapters.par.orderHandling.formatters.outbound.TopOfTheSpreadBookRequestHandler.sendMessage(TopOfTheSpreadBookRequestHandler.java:52)
  19. com.cboe.ohsadapters.par.orderHandling.formatters.OhsOutboundHandler.send(OhsOutboundHandler.java:96)
  20. com.cboe.ohsadapters.par.orderHandling.OhsSender.doTask(OhsSender.java:120)
  21. com.cboe.lwt.thread.ThreadTask.execute(ThreadTask.java:365)
  22. com.cboe.lwt.thread.WorkerThread.run(WorkerThread.java:164)
  23. java.lang.Thread.run(Thread.java:619)
  24. ^AProdOHSA00ParAdapter9pprdaps04aA^AParAdapter^A" 0
  25.  
Notice here, the beginning of the first line starts with 'low debug', and the end of the last line ends with ' " 0 ' but every com.cboe is a new line along with a few others, like -------- Exception -------- and -------- Trace ------------ are new lines. I need this too all be one line.

Please help. Thanks!!!
Apr 7 '10 #1
1 1721
Hydaral
24
I presume you want to replace the newlines with the field delimiter: ^A?

Using AWK:
Expand|Select|Wrap|Line Numbers
  1. {
  2.     # If this is the first line of a new message, reset the string
  3.     if ($0 ~ /low debug/) {
  4.          string=$0
  5.     }
  6.  
  7.     # If it is not the first line, append
  8.     else {
  9.          string=string "^A" $0
  10.     }
  11.  
  12.     # When the last line of a message is found, print the string
  13.     if ($0 ~ /\" 0$/) {
  14.         print string
  15.     }
  16. }
Jun 25 '10 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Jesse Noller | last post by:
I am a relative newbie to python and I am having issues trying to iterate over the lines of a file. I have a text file - foo.bar inside of this file are lines of text: x-3411342 y-1324123...
22
by: Ling Lee | last post by:
Hi all. I'm trying to write a program that: 1) Ask me what file I want to count number of lines in, and then counts the lines and writes the answear out. 2) I made the first part like this: ...
1
by: j | last post by:
Hi, I've been trying to do line/character counts on documents that are being uploaded. As well as the "counting" I also have to remove certain sections from the file. So, firstly I was working...
6
by: ivan.perak | last post by:
Hello, im a beginner in VB.NET... The thing i would like to do is as it follows.... I have a text file (list of names, every name to the next line) which is about 350000 lines long. I would...
7
by: peraklo | last post by:
Hello, there is another problem i am facing. i have a text file which is about 15000 lines big. i have to cut the last 27 lines from that file and create a new text file that contans those 27...
8
by: Army1987 | last post by:
Is this a good way to discard unread data after scanf()? while (getchar() != '\n') ; According to the FAQ scanf always leaves the trailing newline on the input stream, so there's no risk of...
7
by: Gustaf | last post by:
Hi all, Just for fun, I'm working on a script to count the number of lines in source files. Some lines are auto-generated (by the IDE) and shouldn't be counted. The auto-generated part of files...
5
by: parthaspanda22 | last post by:
How can I get to discard the contents of a text file from a specified offset( say, obtained from ftell)? Sincerely.
3
by: Robert | last post by:
I would like to count lines in a file using the fileinput module and I am getting an unusual output. ------------------------------------------------------------------------------...
2
by: rka77 | last post by:
Hi, I am trying to make a Python2.6 script on a Win32 that will read all the text files stored in a directory and print only the lines containing actual data. A sample file - Set : 1 Date:...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.