Read and write last few bytes of a large text file

[sarthur]

Hi Friends,

I am trying to get the last 50 Mb of a large Text file(Over 2 GB) using VB. I have to add this to an Access form so that when the user clicks the button a pop up window asks for the source of the text file and the destination of the output file and then it saves only the last 50 MBytes of the file to the output location. I am not that good in VB programming. I would appreciate if anyone can help me on this?


Thanks,

Arthur

[NeoPa]

What have you managed so far Arthur?

I'm happy to help - but I don't see myself spoonfeeding. You will need to be checking help files etc, but I can point you at where to look.

[sarthur]

What have you managed so far Arthur?

I'm happy to help - but I don't see myself spoonfeeding. You will need to be checking help files etc, but I can point you at where to look.

Hi Neopa,

Thanks for your help. I am trying to get only the last 50 MB's of a large file. I read a lot of articles about reading and writing into files using vb and have created a program. The problem is that when I try to run it I get an overflow error. I think its because of the huge file size(more than 2 Gb) and also because I am trying to get the entire contents in a string. I know a string datatype has some limitations. What I meant help doesn't mean that I need spoonfeeding. What I need is some kind of advice, guidance or even if you could suggest me materials to look at. I have come to experts like you as a final resort and to get some guidance. I appreciate all your efforts to help me with my previous post and Im glad that experts like you spend their precious time to guide people like me. I know that there should be some other efficient way of getting that data. I would appreciate any kind of help from you or any of the other experts here on this. I am attaching the code below.

Thanks and regards,

Arthur

Private Sub GetFile()
 
  Dim intFF_In As integer
  Dim intFF_Out As Integer
  Dim strRead$
  Dim SourceFile As String
  Dim DestinationFile As String
 
 
 
  intFF_In = FreeFile
  sourcefile = filename
  DestinationFile = filename  
 
  Open SourceFile For Binary Access Read Lock Read Write As intFF_In
 
 
  While Not EOF(intFF_In)
 
 
 
 
    If Len(Dir(strOutFile)) > 0 Then Kill strOutFile
 
    intFF_Out = FreeFile
    Open DestinationFile For Binary Access Write Lock Read Write As intFF_Out
 
      strRead = String(LOF(intFF_In), " ")
 
      Get #intFF_In, LOF(intFF_In)-52428799, strRead
 
      Put intFF_Out, , strRead
 
 
 
    Close intFF_Out
 
  Wend
  Close intFF_In
 
 
End Sub
 

[Krandor]

The string variable has a 2 gig limitation. So if your file is 2 gigs +, there is your overflow problem.

I'm a little rusty on the exact syntax, but there is a way to input part of the file beginning at a set point (at character 1,000,000 for example). If all you want is the last bit, there is no point in sucking in the entire file into your variable.

You are pretty close using the LOF so I would Google on LOF Input for VBA to see what shows up.

[NeoPa]

Arthur,

I need some time to look up some old code to give the right pointers here. I will come back when I can. Once the relevant commands and statements are all understood it should be quite doable. My only worry at this stage is whether or not the interface can actually handle file accesses >4GB.

[NeoPa]

2GB is not only a string limit (this could be got around), it is also a limit on accessing the file.

Your code is fine as far as it goes, only you should be looking at handling multiple, more sensibly sized, records. A MB buffer is normally efficient enough, but you'd want to "Seek" to the start point, rather than process through reading all the data up to it.

All this is moot I'm afraid though as, without a facility to Seek to the point you're interested in, it won't work anyway :(

[NeoPa]

It might be possible (you will need to experiment yourself) to adjust the file position without using an explicit Seek, by use of multiple Reads.

Let us know how you get on with this.

[FishVal]

Hello, gentlemen.

I think the following will cover the whole issue.

Different file systems specifications
How To Seek Past VBA's 2GB File Limit

Regards,
Fish

P.S. Long live google. ;)

[NeoPa]

That's a neat solution Fish. I'm not sure everyone will be confident enough to try it, but it's there anyway.

It would certainly be my choice if I had a need for it :)

[sarthur]

Thanks for all your help friends.Sorry for my late reply. I think Fishvals method sounds promising. I will try that and see how that goes.

Thanks,

Arthur