473,809 Members | 2,757 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

HELP: Joining 5 large text files

Hi,

Please can anyone help me join 5 large (1.8gb) text files togeather
to create 1 very large file.

I have some code in PHP but it bombs out at 2gb (seems there is a limit
and php needs re compiling, something thats not an option for me)

I don't want to take up all the servers resources so something that uses
little memory would be very good indeed!

here's the php code if it help...

<?php
set_time_limit( 14400000);

$file[0] = "file1.txt" ;
$file[1] = "file2.txt" ;
$file[2] = "file3.txt" ;
$file[3] = "file4.txt" ;
$file[4] = "file5.txt" ;

$target = "output.txt ";
$handle = fopen($target, 'a');

foreach ($file as $var) {
$c_handle = fopen($var, 'r');
do {
$content = fread($c_handle ,1000000);
fwrite($handle, $content);
} while (!empty($conten t));
fclose($c_handl e);
}
?>
all the best

Stu
Jul 19 '05 #1
3 4869
st*****@hotmail .com (Stuart) wrote in message news:<76******* *************** ***@posting.goo gle.com>...
I have some code in PHP but it bombs out at 2gb (seems there is a limit
and php needs re compiling, something thats not an option for me)


2gigs is a magic number for the maximum file size on some OSes; 2^31.
(You loose a bit due to signed variable usage on some file systems.)

What OS, Architecture are you using? What type of file system i.e.
XFS, EXT2, WINDoze

Cheers!
Jul 19 '05 #2
st*****@hotmail .com (Stuart) wrote in message news:<76******* *************** ***@posting.goo gle.com>...
Newsgroup: comp.lang.perl I have some code in PHP


Perhaps you should try a newsgroup that:

1) Exists (this one doesn't)
2) Relates to PHP (this one wouldn't, if it existed)
Jul 19 '05 #3
Stuart wrote:
create 1 very large file.

I have some code in PHP but it bombs out at 2gb (seems there is a limit
and php needs re compiling, something thats not an option for me)


You're screwed. Unless PHP is recompiled with LARGE_FILE is enabled,
you will *not* be able to write anything past byte number 2147483647.
-Joe
Jul 19 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
5800
by: Alex | last post by:
Hello, I have a rather large table in MS SQL 2000 that I'm writing reports in Crystal from, but I'm unsure how to get the various data files (from different departments) into the table. Below is a small chunk of the table: DeptCode;Type;AveMonthVolume;WorkedFTE;TotHours;Benefits I'm getting data from multiple places, but as I import the data, I
6
1283
by: Chris | last post by:
Hi, I am developing an app that will unzip several zip files and append the contents of the zip files to one text file. The zip files contains one text file with a standard format but the size of the file will be an average of 100k to 300k with at least 7000 rows. Could I use a simple string (Dim s as string)? So first I would unzip a file, read its contents, store it in the string then unzip the second, append it's contents to the same...
10
4742
by: comp.lang.php | last post by:
echo mime_content_type('/var/www/html/video/small.mov'); // 1.5mb Quicktime video returns "video/quicktime" echo mime_content_type('/var/www/html/video/huge.mov'); // 10.5mb Quicktime video returns "text/plain" Environment: PHP 5.0.4 - Windows XP --enable-mime-magic PHP 4.3.11 - Fedora Core 4 --enable-mime-magic
11
5896
by: CSN | last post by:
Is it possible to iterate over an array in plpgsql? Something like: function insert_stuff (rel_ids int) .... foreach rel_ids as id insert into table (rel_id, val) values (id, 5);
0
988
by: Jon | last post by:
We have a asp.net app that runs like the following in a single-site scenario. Directory structure: c:\inetpub\wwwroot\myapp - aspx files, global.asax and web.config c:\inetpub\wwwroot\myapp\bin - myapp.dll, strings.dll (localized strings), library.dll c:\inetpub\wwwroot\myapp\UserControls - ascx files c:\inetpub\wwwroot\myapp\TextBlocks - large localized text blocks in html format
1
3727
by: Rahul | last post by:
Hi Everybody I have some problem in my script. please help me. This is script file. I have one *.inq file. I want run this script in XML files. But this script errors shows . If u want i am attach this script files and inq files. I cant understand this error. Please suggest me. You can talk with my yahoo id b_sahoo1@yahoo.com. Now i am online. Plz....Plz..Plz...
1
1343
by: pootlecat | last post by:
Hello everyone, I have a fair sized table now (1,955,041 rows) and it currently has two indexes: PRIMARY is the ID number and Keywords is a FULLTEXT index of the Keywords column (Text). +----------------------+------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +----------------------+------------------+------+-----+---------+----------------+
3
3022
by: cuties | last post by:
Hi all.... i'm very new to this programming language. i'm required to fulfill this task in the company i'm doing my practical. i hope i can get guide for my problem... Here is the script i already wrote but i'm having problem to move forward. my problem is : 1. how do i assign each checkbox to have equal value with the value of the d_id?
7
1517
by: mosscliffe | last post by:
I have 4 text files each approx 50mb. I need to join these into one large text file. I only need to do this very occasionally, as the problem has occurred because of upload limitations. Bearing in mind filesize and memory useage, would I be better reading every line in every file and writing each line to the output file or is there someway I could execute some shell command.
0
9721
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10635
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10378
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9198
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6881
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5687
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4332
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3861
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3013
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.