473,325 Members | 2,712 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,325 software developers and data experts.

HELP: Joining 5 large text files

Hi,

Please can anyone help me join 5 large (1.8gb) text files togeather
to create 1 very large file.

I have some code in PHP but it bombs out at 2gb (seems there is a limit
and php needs re compiling, something thats not an option for me)

I don't want to take up all the servers resources so something that uses
little memory would be very good indeed!

here's the php code if it help...

<?php
set_time_limit(14400000);

$file[0] = "file1.txt";
$file[1] = "file2.txt";
$file[2] = "file3.txt";
$file[3] = "file4.txt";
$file[4] = "file5.txt";

$target = "output.txt";
$handle = fopen($target, 'a');

foreach ($file as $var) {
$c_handle = fopen($var, 'r');
do {
$content = fread($c_handle,1000000);
fwrite($handle, $content);
} while (!empty($content));
fclose($c_handle);
}
?>
all the best

Stu
Jul 19 '05 #1
3 4818
st*****@hotmail.com (Stuart) wrote in message news:<76*************************@posting.google.c om>...
I have some code in PHP but it bombs out at 2gb (seems there is a limit
and php needs re compiling, something thats not an option for me)


2gigs is a magic number for the maximum file size on some OSes; 2^31.
(You loose a bit due to signed variable usage on some file systems.)

What OS, Architecture are you using? What type of file system i.e.
XFS, EXT2, WINDoze

Cheers!
Jul 19 '05 #2
st*****@hotmail.com (Stuart) wrote in message news:<76*************************@posting.google.c om>...
Newsgroup: comp.lang.perl I have some code in PHP


Perhaps you should try a newsgroup that:

1) Exists (this one doesn't)
2) Relates to PHP (this one wouldn't, if it existed)
Jul 19 '05 #3
Stuart wrote:
create 1 very large file.

I have some code in PHP but it bombs out at 2gb (seems there is a limit
and php needs re compiling, something thats not an option for me)


You're screwed. Unless PHP is recompiled with LARGE_FILE is enabled,
you will *not* be able to write anything past byte number 2147483647.
-Joe
Jul 19 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Alex | last post by:
Hello, I have a rather large table in MS SQL 2000 that I'm writing reports in Crystal from, but I'm unsure how to get the various data files (from different departments) into the table. Below...
6
by: Chris | last post by:
Hi, I am developing an app that will unzip several zip files and append the contents of the zip files to one text file. The zip files contains one text file with a standard format but the size of...
10
by: comp.lang.php | last post by:
echo mime_content_type('/var/www/html/video/small.mov'); // 1.5mb Quicktime video returns "video/quicktime" echo mime_content_type('/var/www/html/video/huge.mov'); // 10.5mb Quicktime video...
11
by: CSN | last post by:
Is it possible to iterate over an array in plpgsql? Something like: function insert_stuff (rel_ids int) .... foreach rel_ids as id insert into table (rel_id, val) values (id, 5);
0
by: Jon | last post by:
We have a asp.net app that runs like the following in a single-site scenario. Directory structure: c:\inetpub\wwwroot\myapp - aspx files, global.asax and web.config c:\inetpub\wwwroot\myapp\bin...
1
by: Rahul | last post by:
Hi Everybody I have some problem in my script. please help me. This is script file. I have one *.inq file. I want run this script in XML files. But this script errors shows . If u want i am...
1
by: pootlecat | last post by:
Hello everyone, I have a fair sized table now (1,955,041 rows) and it currently has two indexes: PRIMARY is the ID number and Keywords is a FULLTEXT index of the Keywords column (Text). ...
3
by: cuties | last post by:
Hi all.... i'm very new to this programming language. i'm required to fulfill this task in the company i'm doing my practical. i hope i can get guide for my problem... Here is the script i...
7
by: mosscliffe | last post by:
I have 4 text files each approx 50mb. I need to join these into one large text file. I only need to do this very occasionally, as the problem has occurred because of upload limitations. ...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.