469,269 Members | 1,006 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,269 developers. It's quick & easy.

Log file data mining

14
I am writing an application in .net 1.1 using c#.
I need to read and store data from a log file.
The log file has 3 types of information:
-the date
-the time
-the event
All 3 are on seperate lines.
I can open the file and read each line.

Any suggestions on the collection I should use to store this information?
I will need to report information from this data.
i.e. the of occurances of a specific event
i.e. times between certian events
Dec 5 '08 #1
19 1946
Frinavale
9,735 Expert Mod 8TB
Consider using a DataBase, XML file, or CSV file that you can easily parse or select from.
Dec 5 '08 #2
David R
14
So storing the log file to an xml file would be better than an arraylist?
Do you have an example?
Dec 5 '08 #3
Curtis Rutland
3,256 Expert 2GB
When you say store, do you mean write to disk, or store in your program in memory....there's a big difference.

Frinny's suggestions are aimed at persistent storage.
Dec 5 '08 #4
David R
14
Sorry, I mean store in my program. I do not need to store to a db.

I'm looking for the best way to read the log file and then do some calculations.
ie. number of occurances, times between certain events, etc
Dec 5 '08 #5
Curtis Rutland
3,256 Expert 2GB
Write a simple class with those properties. Then store your data in a List<YourClass>.
Dec 5 '08 #6
David R
14
can u give an example?
Dec 5 '08 #7
Curtis Rutland
3,256 Expert 2GB
Sure. Your class would look something like this:
Expand|Select|Wrap|Line Numbers
  1. public class LogEntry
  2. {
  3.     public DateTime DateAndTime;
  4.     public string Event;
  5.     LogEntry(DateTime dAndT, string e)
  6.     {
  7.         DateAndTime = dAndT;
  8.         Event = e;
  9.     }
  10. }
And your collection would look something like this:
Expand|Select|Wrap|Line Numbers
  1. System.Collections.ArrayList list = new System.Collections.ArrayList();
Add items to your collection using list.Add.
Dec 5 '08 #8
Frinavale
9,735 Expert Mod 8TB
@insertAlias
Or instead of using an ArrayList you can use a List(Of LogEntry)....this lets you be more specific about the types of objects in your list.
Dec 5 '08 #9
Curtis Rutland
3,256 Expert 2GB
I had thought of that, but I'm pretty sure that doesn't exist in 1.1 :(

In C# it's List<LogEntry>
Dec 5 '08 #10
Frinavale
9,735 Expert Mod 8TB
@insertAlias
1.1 was ages ago!
I can't even remember that far back!

You're right the List(T) Class doesn't exist in the 1.1 Framework: just 2.0 and forward (Generics are so helpful!)
Dec 5 '08 #11
David R
14
Thank You.
One last question -
Like I said, I'm using 1.1 for a stand alone windows app. No db.
Just reading/writing to a file.

Are there any benefits to upgrading to .Net 2 or even 3?
Dec 5 '08 #12
Frinavale
9,735 Expert Mod 8TB
@David R
Simply put: Yes.

There are many new features that will make your life (as a developer) much easier. Some new controls have been added as well that can also make the end user experience more enjoyable as well. Check them out for yourself :)

However, if you're just reading and writing files...then there's no need to upgrade....unless of course you're working with XML documents.

In general I would strongly recommend upgrading to 3.5 because of the benefits it offers (LINQ is probably one of the most powerful)
Dec 5 '08 #13
Curtis Rutland
3,256 Expert 2GB
I would say yes as well. Especially if you are doing database stuff...

And if you go to .NET 3, you might as well go to .NET 3.5. It has all kinds of great tools. Plus, Visual Studio 2008 is just soooo much better than VS.NET
Dec 5 '08 #14
David R
14
getting back to my collection question -
so I will have 2 variable from the LogEntry class; a DateTime and Event string.
How do I add them both to a Arraylist?
Dec 5 '08 #15
Curtis Rutland
3,256 Expert 2GB
You don't. You use that class as a variable.

Expand|Select|Wrap|Line Numbers
  1. ArrayList list = new ArrayList();
  2. LogEntry le = new LogEntry(DateTime.Parse("12/1/2008","Something");
  3. list.Add(le);
Except that you need to do something like that in a loop, reading your lines in, and then creating a new LogEntry, then adding that LogEntry into the ArrayList.

You can use DateTime.Parse (which should work with a string like this: 03/01/2009 05:42:00 see this link for more details) or you can create a new DateTime(year, month, day, hour, minute, second.
Dec 5 '08 #16
David R
14
An arraylist would be best if I need to do some searching and some calcualtions?
Dec 5 '08 #17
Curtis Rutland
3,256 Expert 2GB
Hard to say. An array list is a useful way to store a non-fixed-length array of objects in memory. If you were using Framework 2.0 or later I'd tell you to use a System.Collections.Generic.List or even a Dictionary, but that's a moot point. A dictionary would make searching easier.

If you know the length of the array, you could just use LogEvent[numOfEvents] for less memory overhead.

Without knowing what you need to do to these objects, yes, I'd say use an arraylist.

But at a certain point, you're just going to have to try it out...I can't necessarily tell you what's best for your project until you have tried it and determined what the difficulties and problems are. See what you can do with this, and if you have any other problems, I'll be glad to help.

Make sure you make a new thread for new problems, if they don't directly relate to this thread topic.
Dec 5 '08 #18
David R
14
If i upgrade to Framework 3.5 and use the List class...
How would i add a "Date", "Time" and "Event" to the list?
How would I access the data in a list to count occurrances and perform calculations?
Dec 5 '08 #19
Curtis Rutland
3,256 Expert 2GB
Well, for one thing, I'd suggest holding both the date and time in one field, a DateTime field.

As for using a List, you would do things the same way. Here's a few examples:
Expand|Select|Wrap|Line Numbers
  1. //using statement
  2. using System.Collections.Generic;
  3.  
  4. //creating the list
  5. List<LogEntry> list = new List<LogEntry>();
  6.  
  7. //adding an element:
  8. string date = "10/1/2008";
  9. string time = "13:15:05";
  10. string event = "something";
  11.     //you would have gotten these values from your log file
  12. LogEvent le = new LogEvent(DateTime.Parse(date + " " + time,event);
  13. list.Add(le);
  14.  
  15. //retrieving an entry and accessing it's data:
  16. LogEvent le2 = list[0]; //use whatever number it would be in the array, probably from a loop variable
  17. string event2 = le2.Event;
  18. DateTime dt = event2.DateAndTime;
Hope that clears things up. The only difference is if you use an arraylist, you will have to cast the value like this:
Expand|Select|Wrap|Line Numbers
  1. //assuming list is an arraylist:
  2. LogEvent le2 = (LogEvent)list[0];
Dec 5 '08 #20

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

5 posts views Thread by Framework fan | last post: by
6 posts views Thread by Michael | last post: by
reply views Thread by visual-basic-data-mining.net | last post: by
reply views Thread by http://www.visual-basic-data-mining.net/forum | last post: by
1 post views Thread by http://www.visual-basic-data-mining.net/forum | last post: by
reply views Thread by http://www.visual-basic-data-mining.net/forum | last post: by
18 posts views Thread by Jens | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by suresh191 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.