473,834 Members | 1,794 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Concatenate numerous large binary or text files into one

Hi,
I would like to find out the fastest way to concatenate large (200 or
300MB) and numerous (500 - 700) files (Postscript files) into a single
one (which can ends up being several GigaB) using csharp within a
windows form application.
One of the requirement also is to make sure the output file is closed
only when the concatenation is done so that the next application using
the file (printer spool) does not assume it is finished unledd it is
actually finished.

Thanks in advance for your help.

Sep 30 '06 #1
2 13871
ff*****@gmail.c om wrote:
I would like to find out the fastest way to concatenate large (200 or
300MB) and numerous (500 - 700) files (Postscript files) into a single
one (which can ends up being several GigaB) using csharp within a
windows form application.
One of the requirement also is to make sure the output file is closed
only when the concatenation is done so that the next application using
the file (printer spool) does not assume it is finished unledd it is
actually finished.
Efficient reading and writing of files should just be a
matter of using huge buffers.

I would be more worried about whether PDF files
can just be concatenated.

Arne
Sep 30 '06 #2


<ff*****@gmail. comwrote in message
news:11******** **************@ h48g2000cwc.goo glegroups.com.. .
Hi,
I would like to find out the fastest way to concatenate large (200 or
300MB) and numerous (500 - 700) files (Postscript files) into a single
one (which can ends up being several GigaB) using csharp within a
windows form application.
One of the requirement also is to make sure the output file is closed
only when the concatenation is done so that the next application using
the file (printer spool) does not assume it is finished unledd it is
actually finished.
Well, you don't really need the fastest way. You need a reasonably fast
way. Just use System.IO.FileS tream, open one for your output file, and
iterate over your input files. For each input file, read a buffer from the
input file, write the buffer to the output file, repeat.

You don't need to use huge buffers, and it's probably not worth while to use
async IO or double buffering or any of that.

Here's a simple file concatenation program:

David

using System;
using System.Collecti ons.Generic;
using System.Text;
using System.IO;

namespace FileConcat
{
class Program
{
static int Main(string[] args)
{
if (args.Length != 2)
{
Console.WriteLi ne("usage FileConcat [SourceDir] [DestinationFile]");
}
try
{
Run(args[0],args[1]);
return 0;
}
catch (Exception ex)
{
Console.WriteLi ne(ex);
return 1;
}
}
static void Run(string SourceDir, string OutputFileName)
{
string[] inputFiles = Directory.GetFi les(SourceDir);

int bufSize = 1024 * 64;

byte[] buf = new byte[bufSize];

using (FileStream outFile =
new FileStream(Outp utFileName, FileMode.OpenOr Create,
FileAccess.Writ e, FileShare.None, bufSize))
{
foreach (string inputFile in inputFiles)
{
using (FileStream inFile =
new FileStream(inpu tFile, FileMode.Open, FileAccess.Read ,
FileShare.Read, bufSize))
{
int br = 0;
while ((br = inFile.Read(buf ,0,buf.Length)) 0)
{
outFile.Write(b uf,0,br);
}
}
}
}

}
}
}
Sep 30 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

11
5899
by: CSN | last post by:
Is it possible to iterate over an array in plpgsql? Something like: function insert_stuff (rel_ids int) .... foreach rel_ids as id insert into table (rel_id, val) values (id, 5);
13
2391
by: jt | last post by:
I can't seem to find a way to concatenate strings that have nulls within the string. I have a string that I need another string that has nulls in it and what to append the 2nd string, 3 string and so forth to the 1st string. Any ideas how to go about this? Thanks,
14
2376
by: J.S. | last post by:
In a Windows Form application, which is the better method to concatenate large blocks of code? 1. Reading the text from text files. 2. Adding the text to the VB file itself? Thanks! J.S. --
2
4645
by: gauravkhanna | last post by:
Hi All I need some help for the below problem: Scenario We need to send large binary files (audio file of about 10 MB or so) from the client machine (.Net Windows based application, located outside the home network) to the Web Server and then retrieve the file back from the web server to the client.
68
5274
by: vim | last post by:
hello everybody Plz tell the differance between binary file and ascii file............... Thanks in advance vim
6
4313
by: Sheldon | last post by:
Hi, I am trying to build a large array using concatenate function in python. So as I loop over the number of arrays, of which there are 12 (4 down and 3 across), I create 3 long arrays by concatenating them at the bottom and then concatenating them side by side: for ind in range(num_arrays): if ind == 0: bias_down1 = array(bias)
20
4321
by: mike | last post by:
I help manage a large web site, one that has over 600 html pages... It's a reference site for ham radio folks and as an example, one page indexes over 1.8 gb of on-line PDF documents. The site is structured as an upside-down tree, and (if I remember correctly) never more than 4 levels. The site basically grew (like the creeping black blob) ... all the pages were created in Notepad over the last
3
2369
by: sebastian.harko | last post by:
Helllo, What's the general accepted strategy for dealing with very large binary files in C# ? I have to do a program that reads some "multi frame bitmap " files which can reach up to one hundred megs so I need to know how to optimize reading a file.. Best regards, Seb
1
1398
by: mikeburgesss5355 | last post by:
i wish to use a "button" on a in access switchboard to edit numerous notepad txt files..these are hql files(health query language files used in healthcare gp systems). (normally about 20 notepad files,) that are in a given folder c:\mbc .I need to change automatically a section of text ...ie LOCAL(always this) to a variable number ie G123456 that appears in each of these text files. The format of these files must remain as simple unformatted...
0
9649
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10799
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10515
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10554
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9338
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5629
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5799
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4428
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3084
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.