473,320 Members | 2,027 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Read and write last few bytes of a large text file

16
Hi Friends,

I am trying to get the last 50 Mb of a large Text file(Over 2 GB) using VB. I have to add this to an Access form so that when the user clicks the button a pop up window asks for the source of the text file and the destination of the output file and then it saves only the last 50 MBytes of the file to the output location. I am not that good in VB programming. I would appreciate if anyone can help me on this?


Thanks,

Arthur
Oct 13 '08 #1
10 3228
NeoPa
32,556 Expert Mod 16PB
What have you managed so far Arthur?

I'm happy to help - but I don't see myself spoonfeeding. You will need to be checking help files etc, but I can point you at where to look.
Oct 13 '08 #2
sarthur
16
What have you managed so far Arthur?

I'm happy to help - but I don't see myself spoonfeeding. You will need to be checking help files etc, but I can point you at where to look.

Hi Neopa,

Thanks for your help. I am trying to get only the last 50 MB's of a large file. I read a lot of articles about reading and writing into files using vb and have created a program. The problem is that when I try to run it I get an overflow error. I think its because of the huge file size(more than 2 Gb) and also because I am trying to get the entire contents in a string. I know a string datatype has some limitations. What I meant help doesn't mean that I need spoonfeeding. What I need is some kind of advice, guidance or even if you could suggest me materials to look at. I have come to experts like you as a final resort and to get some guidance. I appreciate all your efforts to help me with my previous post and Im glad that experts like you spend their precious time to guide people like me. I know that there should be some other efficient way of getting that data. I would appreciate any kind of help from you or any of the other experts here on this. I am attaching the code below.

Thanks and regards,

Arthur


Expand|Select|Wrap|Line Numbers
  1. Private Sub GetFile()
  2.  
  3.   Dim intFF_In As integer
  4.   Dim intFF_Out As Integer
  5.   Dim strRead$
  6.   Dim SourceFile As String
  7.   Dim DestinationFile As String
  8.  
  9.  
  10.  
  11.   intFF_In = FreeFile
  12.   sourcefile = filename
  13.   DestinationFile = filename  
  14.  
  15.   Open SourceFile For Binary Access Read Lock Read Write As intFF_In
  16.  
  17.  
  18.   While Not EOF(intFF_In)
  19.  
  20.  
  21.  
  22.  
  23.     If Len(Dir(strOutFile)) > 0 Then Kill strOutFile
  24.  
  25.     intFF_Out = FreeFile
  26.     Open DestinationFile For Binary Access Write Lock Read Write As intFF_Out
  27.  
  28.       strRead = String(LOF(intFF_In), " ")
  29.  
  30.       Get #intFF_In, LOF(intFF_In)-52428799, strRead
  31.  
  32.       Put intFF_Out, , strRead
  33.  
  34.  
  35.  
  36.     Close intFF_Out
  37.  
  38.   Wend
  39.   Close intFF_In
  40.  
  41.  
  42. End Sub
  43.  
Oct 15 '08 #3
Krandor
50
The string variable has a 2 gig limitation. So if your file is 2 gigs +, there is your overflow problem.

I'm a little rusty on the exact syntax, but there is a way to input part of the file beginning at a set point (at character 1,000,000 for example). If all you want is the last bit, there is no point in sucking in the entire file into your variable.

You are pretty close using the LOF so I would Google on LOF Input for VBA to see what shows up.
Oct 15 '08 #4
NeoPa
32,556 Expert Mod 16PB
Arthur,

I need some time to look up some old code to give the right pointers here. I will come back when I can. Once the relevant commands and statements are all understood it should be quite doable. My only worry at this stage is whether or not the interface can actually handle file accesses >4GB.
Oct 16 '08 #5
NeoPa
32,556 Expert Mod 16PB
2GB is not only a string limit (this could be got around), it is also a limit on accessing the file.

Your code is fine as far as it goes, only you should be looking at handling multiple, more sensibly sized, records. A MB buffer is normally efficient enough, but you'd want to "Seek" to the start point, rather than process through reading all the data up to it.

All this is moot I'm afraid though as, without a facility to Seek to the point you're interested in, it won't work anyway :(
Oct 16 '08 #6
NeoPa
32,556 Expert Mod 16PB
It might be possible (you will need to experiment yourself) to adjust the file position without using an explicit Seek, by use of multiple Reads.

Let us know how you get on with this.
Oct 16 '08 #7
FishVal
2,653 Expert 2GB
Hello, gentlemen.

I think the following will cover the whole issue.

Different file systems specifications
How To Seek Past VBA's 2GB File Limit

Regards,
Fish

P.S. Long live google. ;)
Oct 16 '08 #8
NeoPa
32,556 Expert Mod 16PB
That's a neat solution Fish. I'm not sure everyone will be confident enough to try it, but it's there anyway.

It would certainly be my choice if I had a need for it :)
Oct 16 '08 #9
sarthur
16
Thanks for all your help friends.Sorry for my late reply. I think Fishvals method sounds promising. I will try that and see how that goes.

Thanks,

Arthur
Oct 17 '08 #10
NeoPa
32,556 Expert Mod 16PB
Good for you Arthur :)

Let us know how you get on.
Oct 17 '08 #11

Sign in to post your reply or Sign up for a free account.

Similar topics

3
by: hunterb | last post by:
I have a file which has no BOM and contains mostly single byte chars. There are numerous double byte chars (Japanese) which appear throughout. I need to take the resulting Unicode and store it in a...
0
by: Ahmed A. | last post by:
This will be very helpfull for many! Using RichTextBox Read/Write Unicode File http://www.microsoft.com/indonesia/msdn/wnf_RichTextBox.as p Private Function ReadFile(ByVal myfile As String)...
35
by: RyanS09 | last post by:
Hello- I am trying to write a snippet which will open a text file with an integer on each line. I would like to read the last integer in the file. I am currently using: file = fopen("f.txt",...
0
by: phplasma | last post by:
Hey, I am currently attempting to implement a multi-threaded C# socket, using SSL (.pem file/certification/private key combo) server using Visual Studio C# Express. I have successfully made...
8
by: paraidy | last post by:
Hi all, as from object i need to read all byte from a file example c: \myphoto.jpg and recreate the file with another name to another directory c:\photo\recreatedphoto.jpg can someone write a...
2
by: Kevin Ar18 | last post by:
I posted this on the forum, but nobody seems to know the solution: http://python-forum.org/py/viewtopic.php?t=5230 I have a zip file that is several GB in size, and one of the files inside of it...
9
by: vineeth | last post by:
Hello all, I have come across a weird problem, I need to determine the amount of bytes read from a file, but couldn't figure it out , My program does this : __ file = open("somefile") data =...
4
by: Curious | last post by:
I have a file to which I'll need to at first read until it locates the last line of string. According to content of the last line of string, I'll need to write something to this file right after...
4
by: Keith G Hicks | last post by:
I'm trying to read a text file and alter the contents of specific lines in the file. I know how to use streamreader to read each line of a file. I'm doing that already to get the data into a...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.