VB.NET Really Slow Filestream

[fcharby]

Hello all, I'm new here and I've searched a lot before posting my question, but I can't seem to find a solution to my problem.

I'm working on a database program that I created myself, I created my own file format and I store information about clients in there. So far, so good, but when I try to search the content of my file, that's where it becomes slow....I try to check if a byte is > than 0 in every client's profile....there are 100 000 clients and the file size is about 200mb.

It goes something like this....

Dim objFile As New System.IO.FileStream("Clients.pcf", IO.FileMode.Open)

For i = 0 To 99999
If Results(i) Then
objFile.Position = (Jump * i) + position
If objFile.ReadByte() > 0 Then
Results(i) = False
End If
End If
Next

Results() is an array of Bool and is used to indicate if the client matches what I'm looking for. It is used several times by different Subs.
Jump indicates the length of each client's profile in bytes.
Position indicates where the information is located within the client's profile.

My computer is old (1gHz), but still, a task like that takes almost 30 seconds...I found out that this line slows everything down: objFile.Position = (Jump * i) + position, but a multiplication is not supposed to take that long (I've tried it separately) and if I click a second time on the button that handles that event, it takes less that a second to get my results....

I've even looked at the CPU usage and it goes up only for the second....what's happening? Is VB copying my file somewhere?

I must say that I'm more that confused right now....I'm sorry if this is a long post, but it's a pretty complex situation and I would appreciate every suggestion.

Thank you

[Plater]

It's doing a file seek. Which means it is going through and reading every byte on it's way to that location.
You might have better luck with a native API call, but I doubt it.
200mb is a pretty big file, 30seconds to scan through the whole thing sounds pretty fast to me.

[fcharby]

Well, then how do people manage to search through files faster than that?

[Plater]

Well, then how do people manage to search through files faster than that?

The API Calls might be faster (or maybe there's a better way in .NET) but not as a Stream object I don't think.

Who scans files faster then that?

[fcharby]

Well, a Database works a whole lot faster than that...

How is that possible?

(Keep in mind that my Database knowledge is pretty low, I'm trying to do things on my own without using SQL or Access or stuff like that)

[Shashi Sadasivan]

Databases use data structures like B+ trees.
They help in indexing/locating records/data.
Currently the only sort of structure that you have is that each record is barely a line (hopefully on a certain length only) and hence you have to go through each line.
To simplify your search you would have to change the way you look for files.
Either create an index, or you could create multiple files and based on certain condition you would need to only access certain files