471,349 Members | 1,592 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,349 software developers and data experts.

Parsing log file using C#

I need code that can parse data from log file.
The log is pipe delimited and in following format:
time=2006-11-03 13:13:56| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=24.141.197.139| dstname=| duration=| msg=AUT22670: Login succeeded for tjolen/EMEA.

time=2006-11-03 13:14:07| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=63.131.197.139| dstname=TUN-VPN| duration=| msg=JAV20021: Connected to EPN port 443

time=2006-11-03 13:16:10| fw=199.23.38.120| user=tjolen| realm=EMEA| roles=Network Connect| src=34.131.127.139| dstname=| duration=| msg=AUT22673: Logout from 24.131.127.139

Purpose is to read from text file and store in sql server database.
Please help!
Apr 8 '08 #1
7 6167
There are several good open-source implementations of CSVReader that you can
use for this. Just google "C#" CsvReader.
-- Peter
Site: http://www.eggheadcafe.com
UnBlog: http://petesbloggerama.blogspot.com
Short Urls & more: http://ittyurl.net
"Prem Parekh" wrote:
I need code that can parse data from log file.
The log is pipe delimited and in following format:
time=2006-11-03 13:13:56| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=24.141.197.139| dstname=| duration=| msg=AUT22670: Login succeeded for tjolen/EMEA.

time=2006-11-03 13:14:07| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=63.131.197.139| dstname=TUN-VPN| duration=| msg=JAV20021: Connected to EPN port 443

time=2006-11-03 13:16:10| fw=199.23.38.120| user=tjolen| realm=EMEA| roles=Network Connect| src=34.131.127.139| dstname=| duration=| msg=AUT22673: Logout from 24.131.127.139

Purpose is to read from text file and store in sql server database.
Please help!
Apr 8 '08 #2
On Tue, 08 Apr 2008 12:10:52 -0700, Prem Parekh wrote:
>I need code that can parse data from log file.
The log is pipe delimited and in following format:
time=2006-11-03 13:13:56| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=24.141.197.139| dstname=| duration=| msg=AUT22670: Login succeeded for tjolen/EMEA.

time=2006-11-03 13:14:07| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=63.131.197.139| dstname=TUN-VPN| duration=| msg=JAV20021: Connected to EPN port 443

time=2006-11-03 13:16:10| fw=199.23.38.120| user=tjolen| realm=EMEA| roles=Network Connect| src=34.131.127.139| dstname=| duration=| msg=AUT22673: Logout from 24.131.127.139

Purpose is to read from text file and store in sql server database.
Please help!
Have a look at String.Split()

rossum

Apr 8 '08 #3
On Apr 8, 3:10*pm, Prem Parekh wrote:
I need code that can parse data from log file.
The log is pipe delimited and in following format:
time=2006-11-03 13:13:56| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=24.141.197.139| dstname=| duration=| msg=AUT22670: Login succeeded for tjolen/EMEA. * * *

time=2006-11-03 13:14:07| fw=199.23.48.120| user=tjolen| realm=EMEA| roles=Network Connect| src=63.131.197.139| dstname=TUN-VPN| duration=| msg=JAV20021: Connected to EPN port 443 * * *

time=2006-11-03 13:16:10| fw=199.23.38.120| user=tjolen| realm=EMEA| roles=Network Connect| src=34.131.127.139| dstname=| duration=| msg=AUT22673: Logout from 24.131.127.139 *

Purpose is to read from text file and store in sql server database.
Please help!
Hi,

You can split the line using String.Split( new char[] {'|'}); and then
later split each piece by the '=' then you have a pair of key,value.
Apr 8 '08 #4
Hi,
try this....

string str = System.IO.File.ReadAllText("PathName");

now str hold the content of the file... you can get substring and split the values based on your criteria and use it...
Apr 9 '08 #5
Vasanth,

Thanks for your reply. I've done similar to what you suggested. Now how I can I put this parsed text into a table by removing column names (time, fw,etc) that are repeating in everyline and write it to Sql Server.
Should I use Dataset?
Please advise.

My code:
using (StreamReader sr = new StreamReader("c:\\textfiles\\modified.txt"))
{
String str;
// Read and display lines from the file until the end of
// the file is reached.
while ((str = sr.ReadLine()) != null)
{
string[] arInfo = new string[4];

// define which character is seperating fields
char[] splitter = { '|' };

arInfo = str.Split(splitter);

for (int x = 0; x < arInfo.Length; x++)
{
Console.Write(arInfo[x] + "<br>");
}

}
}
Apr 9 '08 #6
It depends on the length of the file.

The fastest way to get it into a SQL Server is using SqlBulkCopy -
WriteToServer.
If it is reasonable to hold it in memory, then DataTable is a
quick'n'dirty answer. If the file is very large, you will want to
stream, by impersonating an IDataReader. You could steal
SimpleDataReader from here:
http://groups.google.com/group/micro...c7a20056ffe8e1

and the rest of your code replaces XmlDataReader (from the same post);
in particular, your DoRead override would read the next line [return
false if EOF], call SetValues(arInfo) and return true.

Marc
Apr 9 '08 #7
For info - I've just (for fun) written a LINQ-based implementation of
this that makes it easy to chain an object stream to an IDataReader.
Let me know if you'd find it useful,
Apr 9 '08 #8

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by Willem Ligtenberg | last post: by
3 posts views Thread by Pir8 | last post: by
4 posts views Thread by Neil.Smith | last post: by
13 posts views Thread by Chris Carlen | last post: by
2 posts views Thread by Felipe De Bene | last post: by
reply views Thread by XIAOLAOHU | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.