473,320 Members | 1,853 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

How to determine position within a text file under FSO model?

8,435 Expert 8TB
Hi all.

In the past I've done most file IO using the old built-in VB statements such as Open, Line Input #, Get, Put and so on. In a recent project I decided to try and update a bit, so I'm using a File object to represent each file, and OpenAsTextStream to create a TextStream object, then reading that using ReadLine.

It's working just fine, except for one thing. How can I tell where I am in the file? I'm dealing with large files (hundreds of MB) so I like to provide a progress indicator, based on the file size and my position in the file. That was perfectly simple before, using LOF and Seek functions. But when using a TextStream in this way, how can I determine my current position? I can get the line number of course, but for that to be any use I need to know the total number of lines.

At the moment I just start out by doing a complete read through the file (using .ReadLine) to count the lines. This is fairly quick (anywhere from 5 to 30 seconds), then I can use that count to provide the progress indicator during the "real" processing (which obviously takes much longer, or I wouldn't bother). But it seems silly to have to do a double pass through the file like this.
Sep 26 '07 #1
17 4678
Ali Rizwan
925 512MB
Hello,
Do you want to add a progressbar or any slider type thing.
Sep 27 '07 #2
QVeen72
1,445 Expert 1GB
Hi,

I had answered this a few days ago.

Use "ReadAll" Method, and Split for vbCrLF, you will get Array's ubound as No. of Lines..:

Expand|Select|Wrap|Line Numbers
  1.     Dim FSO As New Scripting.FileSystemObject
  2.     Dim TArr
  3.     Dim fsStr As TextStream
  4.     '
  5.     Set fsStr = FSO.OpenTextFile("C:\MyText.txt", ForReading)
  6.     TArr = Split(fsStr.ReadAll, vbCrLf)
  7.     MsgBox "Total Number Of Lines : " & UBound(TArr)
  8.     fsStr.Close
  9.     Set FSO = Nothing
  10.  
Regards
Veena
Sep 27 '07 #3
Killer42
8,435 Expert 8TB
Use "ReadAll" Method, and Split for vbCrLF, you will get Array's ubound as No. of Lines..:
Thanks for the idea.

I'd rather not, though. I am already counting the number of lines, so I don't need a way to do that (though yours may be faster). Since each file is at least 100-200MB in size, I don't want to be making copies of it all over my RAM. That's why I'm processing it line by line.
Sep 27 '07 #4
Killer42
8,435 Expert 8TB
Do you want to add a progressbar or any slider type thing.
I already have a progress bar, thanks. What I'm looking for is a better way to determine where I'm up to in the file. In other words, I'm looking for an equivalent of the old Seek() function.
Sep 27 '07 #5
QVeen72
1,445 Expert 1GB
Hi,

FSO dosent have any equivalent of "Seek" operation, you have to settle for IO's Seek. Or write a Seek Function of your own.

Regards
Veena
Sep 28 '07 #6
hariharanmca
1,977 1GB
I already have a progress bar, thanks. What I'm looking for is a better way to determine where I'm up to in the file. In other words, I'm looking for an equivalent of the old Seek() function.
I think this Article page will help you

How To Seek Past VBA's 2GB File Limit
Sep 28 '07 #7
Killer42
8,435 Expert 8TB
Thanks for trying, people, but I guess I'll just have to accept that with the FSO model, MS in their wisdom have thrown out the whole concept of "where I'm up to". No wonder their progress indicators are so inaccurate.

Hari, thanks for the link, but I've already seen that article (some years ago). I'm not trying to go beyond 2GB, just want to know my position within a file. I can't use things like Seek() because I'm not using Open.

Looks as though I only have two real options...
  1. Live with the two-step process I'm using now. That is, first count all the lines, then count them again while processing them to track position, or
  2. Go back to good old Open and Line Input # processing, and use Seek() function to track progress.
  3. Just thought of another alternative while I was typing. Since I can easily determine the file size, I can just add up the length of each line as I go, plus 2 for the delimiters, to track my progress. Yippee!
Sep 28 '07 #8
hariharanmca
1,977 1GB
If you don’t mind! Can you post some part of code and let me do some research? (Of-course it will help me to learn.)
Sep 29 '07 #9
hariharanmca
1,977 1GB
Hari, thanks for the link, but I've already seen that article (some years ago). I'm not trying to go beyond 2GB, just want to know my position within a file. I can't use things like Seek() because I'm not using Open.
In text file there is no Row or column ID to indicate where we are. But not willing to say this will b the answer. I want to try more.
It's working just fine, except for one thing. How can I tell where I am in the file? I'm dealing with large files (hundreds of MB) so I like to provide a progress indicator, ........
we cannot show progressbar without knowing Min,CurrentPosition,Max value

Till working on that....
Sep 29 '07 #10
QVeen72
1,445 Expert 1GB
[*]Just thought of another alternative while I was typing. Since I can easily determine the file size, I can just add up the length of each line as I go, plus 2 for the delimiters, to track my progress. Yippee![/list]
YES, Good one, U can get Total Size of the file thru FSO,
and Just get total bytes read till now using the "LenB( )" function,
Convert to kb, and Increment the progress bar accordingly....
Real good option, thanx for that..

Regards
Veena
Sep 29 '07 #11
hariharanmca
1,977 1GB
How about My Program?

I hope this will be usefull!
Attached Files
File Type: zip ReadTextFile.zip (6.6 KB, 171 views)
Sep 29 '07 #12
QVeen72
1,445 Expert 1GB
Hi Hari,

Read Killer's previous posts, he dont want to use "ReadAll"..
Sep 29 '07 #13
hariharanmca
1,977 1GB
Hi Hari,

Read Killer's previous posts, he dont want to use "ReadAll"..
Yha i replied to that post.
In text file there is no Row or column ID to indicate where we are. But not willing to say this will b the answer. I want to try more.
Without row or column id we cannot set directly to any row or column.
But just gussing.....
Sep 29 '07 #14
Killer42
8,435 Expert 8TB
I'm on a different computer and don't have access to my code. So I'll throw together a demo function to show the sort of process I'm using. First is the FSO version. The second function shows how I would normally do this sort of thing, using Open and Line Input # statements.

Expand|Select|Wrap|Line Numbers
  1. Sub ReadFile_FSO_version(FileName As String, PrgBar As ProgressBar)
  2.   ' For simplicity, I'm going to assume the progress bar
  3.   ' has Min = 0, Max = 100.
  4.  
  5.   Dim fso As New FileSystemObject
  6.   Dim fil As File
  7.   Dim str As TextStream
  8.   Dim TotalLines As Long
  9.   Dim OnePercent As Long
  10.  
  11.  
  12.   Set fil = fso.GetFile(FileName)
  13.  
  14.   ' Step 1 - count the lines in the file.
  15.   Set str = fil.OpenAsTextStream(ForReading)
  16.   With str
  17.     Do Until .AtEndOfStream
  18.       .ReadLine
  19.       TotalLines = TotalLines + 1
  20.     Loop
  21.     .Close
  22.   End With
  23.   OnePercent = TotalLines / 100
  24.  
  25.   ' Step 2 - process the data in the file.
  26.   Set str = fil.OpenAsTextStream(ForReading)
  27.   With str
  28.     Do Until .AtEndOfStream
  29.       .ReadLine
  30.       ' Process the data from this line.
  31.  
  32.       If .Line Mod 500 = 0 Then
  33.         PrgBar.Value = .Line / OnePercent
  34.         DoEvents
  35.       End If
  36.     Loop
  37.     .Close
  38.   End With
  39.   Set fil = Nothing
  40.  
  41.  
  42. End Sub
  43.  
  44.  
  45. Sub ReadFile_NonFSO_version(FileName As String, PrgBar As ProgressBar)
  46.   ' For simplicity, I'm going to assume the progress bar
  47.   ' has Min = 0, Max = 100.
  48.  
  49.   Dim OnePercent As Long
  50.   Dim FileSize As Long
  51.   Dim FileNo As Long, LineNo As Long
  52.   Dim Text As String
  53.  
  54.   FileNo = FreeFile
  55.   Open FileName For Input Access Read Shared As #FileNo
  56.   FileSize = LOF(FileNo)
  57.   OnePercent = FileSize / 100
  58.  
  59.   Do Until EOF(FileNo)
  60.     Line Input #FileNo, Text
  61.     LineNo = LineNo + 1
  62.  
  63.     ' Process the data from this line.
  64.  
  65.     If LineNo Mod 500 = 0 Then
  66.       PrgBar.Value = Seek(FileNo) / OnePercent
  67.       DoEvents
  68.     End If
  69.  
  70.   Loop
  71.  
  72.   Close #FileNo
  73.  
  74. End Sub
Sep 30 '07 #15
hariharanmca
1,977 1GB
Hi killer42,
Did you saw the zip file in #12, if that is okay then; i am ready to explain!
Oct 1 '07 #16
Killer42
8,435 Expert 8TB
Did you saw the zip file in #12, if that is okay then; i am ready to explain!
Ok, I've just had a look now.

I see it uses the "ReadAll, Split and Ubound" technique described in this thread. Thanks, it could be a useful technique for getting the count more quickly (I currently read line by line and count them). But it doesn't help in my case, as I want to avoid having to read the file and get the line count.

I think I'll run with the idea that occurred to me while typing my earlier message. That is, get the file size up front by using the old-fashioned FileLen() function, then add up the length of each line as I process them (plus two for the delimiters) to track my position.
Oct 1 '07 #17
hariharanmca
1,977 1GB
Ok, I've just had a look now......
This Seek function will return, how many text a streame read currently So we cannot get the how many line.

We should use progress in timer control ( we can avoid to show each and every line status by changing the timer intervel).
Oct 1 '07 #18

Sign in to post your reply or Sign up for a free account.

Similar topics

7
by: Kamus of Kadizhar | last post by:
Thanks to everyone on this list. I now have a functioning piece of python code!! :-)) Now I'm trying to clean it up. I have the same (or similar) lines repeated several times: ...
14
by: Zenobia | last post by:
Hello folks, Is it possible to position an item within a <td> element? For instance see below. The table has multiple rows, one for each database record. Each row has 3 hrefs associated with it...
2
by: GrantS | last post by:
I am trying to convert the VB.Net code example povided by http://authors.aspalliance.com/JimRoss/Articles/MaintainScrollPos.aspx into C# (ASP.Net)without success. No errors are thrown in the VB...
4
by: JE | last post by:
Hi! I am snooping around all sorts of websites and faqs, in search of a way to pick up any selected (mouse-highlighted) text in a HTML page, and store it in a JavaScript variable. I have...
12
by: Sunner Sun | last post by:
Hi, all Since the OS look both ASCII and binary file as a sequence of bytes, is there any way to determine the file type except to judge the extension? Thank you!
10
by: Marc Jennings | last post by:
Hi there, Can anyone point out any really obvious flaws in the methodology below to determine the likely encoding of a file, please? I know the number of types of encoding is small, but that is...
11
blyxx86
by: blyxx86 | last post by:
Alright, I'm trying to create a form that would allow certain documents to be opened by selecting an option within a combo box and then having a single button to open the pertaining file. I know...
4
by: Aussie Rules | last post by:
Hi, I want to be able to show a txt file(stored on disk) to the end user in some sort of txt editor control, and allow the user to highlight/select words within the text. Once they have...
4
by: sarika | last post by:
Hi all I want to create an online stamp making site. In my site a user select a template from number of templates for making his stamp. The template is and image with some graphics and text. Now...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.