473,217 Members | 1,954 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,217 software developers and data experts.

Parsing large files by line

Does anyone have a suggestion for parsing large files line by line without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin
Aug 15 '05 #1
3 1757
Kevin wrote:
Does anyone have a suggestion for parsing large files line by line without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin


$handle=fopen(...);
while($line=fgets($handle)) {
...
}
fclose($handle);

C.
Aug 15 '05 #2
Just what I was looking for, thanks!

- Kevin

"Colin McKinnon" <co**************@andthis.mms3.com> wrote in message
news:dd*******************@news.demon.co.uk...
Kevin wrote:
Does anyone have a suggestion for parsing large files line by line
without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin


$handle=fopen(...);
while($line=fgets($handle)) {
...
}
fclose($handle);

C.

Aug 15 '05 #3
Kevin schrieb:
Does anyone have a suggestion for parsing large files line by line without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin


$handle = fopen("/tmp/inputfile.txt", "r");
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
echo $buffer;
}
fclose($handle);

more at www.php.net/fgets

Markus
Aug 15 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Ratnakar Pedagani | last post by:
Hi, I'm trying to parse the text file, which is of size more than 2mb. I'm using the following sample code Open "c:\sim1.txt" For Input As #1 Do While Not EOF(1) Input #1, Data If...
2
by: Cigdem | last post by:
Hello, I am trying to parse the XML files that the user selects(XML files are on anoher OS400 system called "wkdis3"). But i am permenantly getting that error: Directory0: \\wkdis3\ROOT\home...
9
by: PedroX | last post by:
Hello: I need to parse some large XML files, and save the data in an Access DB. I was using MSXML 2 and ASP, but it turns out to be extremely slow when then XML documents are like 10 mb in...
6
by: ArunPrakash | last post by:
Hi, I have a web application that looks for a particular string in a set of huge files( the files grow into MBs - max i have seen is 30 MB ). ( search using reg expressions ). the string can occur...
8
by: Eric Anderson | last post by:
I have some files that sit on a FTP server. These files contain data stored in a tab-separated format. I need to download these files and insert/update them in a MySQL database. My current basic...
7
by: Alan | last post by:
Hi. I have programmed in C++ before, but I`m a couple of years out of practice. I am seeking some advice on getting started on a quickie project. . . . I have to read a 54MB text file and do a...
3
by: toton | last post by:
Hi, I have some ascii files, which are having some formatted text. I want to read some section only from the total file. For that what I am doing is indexing the sections (denoted by .START in...
1
by: Robert Neville | last post by:
Basically, I want to create a table in html, xml, or xslt; with any number of regular expressions; a script (Perl or Python) which reads each table row (regex and replacement); and performs the...
10
by: len | last post by:
I have created the following program to read a text file which happens to be a cobol filed definition. The program then outputs to a file what is essentially a file which is a list definition...
1
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: mar23 | last post by:
Here's the situation. I have a form called frmDiceInventory with subform called subfrmDice. The subform's control source is linked to a query called qryDiceInventory. I've been trying to pick up the...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.