473,473 Members | 1,901 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

Parsing large files by line

Does anyone have a suggestion for parsing large files line by line without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin
Aug 15 '05 #1
3 1772
Kevin wrote:
Does anyone have a suggestion for parsing large files line by line without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin


$handle=fopen(...);
while($line=fgets($handle)) {
...
}
fclose($handle);

C.
Aug 15 '05 #2
Just what I was looking for, thanks!

- Kevin

"Colin McKinnon" <co**************@andthis.mms3.com> wrote in message
news:dd*******************@news.demon.co.uk...
Kevin wrote:
Does anyone have a suggestion for parsing large files line by line
without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin


$handle=fopen(...);
while($line=fgets($handle)) {
...
}
fclose($handle);

C.

Aug 15 '05 #3
Kevin schrieb:
Does anyone have a suggestion for parsing large files line by line without
loading the entire file into memory first? I don't want to use file()
because the files I'm working with may be multi-gigabyte so loading them
into arrays would be pretty memory intensive.

In Perl, I can do: "while($line=<HANDLE>){do something with $line}".

Is there an equivalent function in PHP?

Thanks,
Kevin


$handle = fopen("/tmp/inputfile.txt", "r");
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
echo $buffer;
}
fclose($handle);

more at www.php.net/fgets

Markus
Aug 15 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Ratnakar Pedagani | last post by:
Hi, I'm trying to parse the text file, which is of size more than 2mb. I'm using the following sample code Open "c:\sim1.txt" For Input As #1 Do While Not EOF(1) Input #1, Data If...
2
by: Cigdem | last post by:
Hello, I am trying to parse the XML files that the user selects(XML files are on anoher OS400 system called "wkdis3"). But i am permenantly getting that error: Directory0: \\wkdis3\ROOT\home...
9
by: PedroX | last post by:
Hello: I need to parse some large XML files, and save the data in an Access DB. I was using MSXML 2 and ASP, but it turns out to be extremely slow when then XML documents are like 10 mb in...
6
by: ArunPrakash | last post by:
Hi, I have a web application that looks for a particular string in a set of huge files( the files grow into MBs - max i have seen is 30 MB ). ( search using reg expressions ). the string can occur...
8
by: Eric Anderson | last post by:
I have some files that sit on a FTP server. These files contain data stored in a tab-separated format. I need to download these files and insert/update them in a MySQL database. My current basic...
7
by: Alan | last post by:
Hi. I have programmed in C++ before, but I`m a couple of years out of practice. I am seeking some advice on getting started on a quickie project. . . . I have to read a 54MB text file and do a...
3
by: toton | last post by:
Hi, I have some ascii files, which are having some formatted text. I want to read some section only from the total file. For that what I am doing is indexing the sections (denoted by .START in...
1
by: Robert Neville | last post by:
Basically, I want to create a table in html, xml, or xslt; with any number of regular expressions; a script (Perl or Python) which reads each table row (regex and replacement); and performs the...
10
by: len | last post by:
I have created the following program to read a text file which happens to be a cobol filed definition. The program then outputs to a file what is essentially a file which is a list definition...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
1
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...
0
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
1
muto222
php
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.