473,597 Members | 2,342 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

writing to file

Hi,

I have an application where i want to write data to a file, the data is
being sent from an external source. I know the total size of the data and
then i retrieve the data in small segments from the source. This data is
written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to 0 and
start writing to file. whether there will be any performance improvements
because of the allocation of the file earlier and due to less io? or this
doesnt matter because every fwrite writes to disk(OS may schedule writing to
disk).
2. If this is helps performance, do i have an api to initialise the file
with some size and empty strings?

TFI
ishekara.
Jul 22 '05 #1
3 3392
ishekar wrote:
Hi,

I have an application where i want to write data to a file, the data
is being sent from an external source. I know the total size of the
data and then i retrieve the data in small segments from the source.
This data is written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to
0 and start writing to file. whether there will be any performance
improvements because of the allocation of the file earlier and due to
less io? or this doesnt matter because every fwrite writes to disk(OS
may schedule writing to disk).
This is very platform specific, so you might want to ask in a newsgroup
about your OS. I think that usually, the performance when writing is
better, since the kernel doesn't need to search for free clusters on
the disk. But of course, creating the file initially will take time,
too. So if you need your hard drive to be as fast as possible so you
can keep up with a high data rate, it's probably a good idea to create
the file initially. If not, you won't probably gain anything or even
make the total run time worse due to the additional "initialization " of
the file.
2. If this is helps performance, do i have an api to initialise the
file with some size and empty strings?


Exept for just writing it yourself (which is actually very simple),
there is none in standard C++.

Jul 22 '05 #2

"ishekar" <is******@vsnl. net> wrote in message
news:c1******** **@news.mch.sbs .de...
Hi,

I have an application where i want to write data to a file, the data is
being sent from an external source. I know the total size of the data and
then i retrieve the data in small segments from the source. This data is
written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to 0 and start writing to file. whether there will be any performance improvements
because of the allocation of the file earlier and due to less io? or this
doesnt matter because every fwrite writes to disk(OS may schedule writing to disk).
2. If this is helps performance, do i have an api to initialise the file
with some size and empty strings?

TFI
ishekara.


The answer to this sort of question is always

1) Do it the simplest way first (in this case don't precreate the file).
2) Once the program is working to some profiling to see if the file I/O
speed is a problem or not. If not then you've saved yourself the bother of
writing the more complex code.
3) If it is a problem then try your alternate method, profile that and see
if it is any faster. Bear in mind that the results on one computer may not
be reflected on another computer.

There is no way in standard C++ of creating a file so big, except writing
out the required number of bytes. Your operating system may have a more
efficient way of doing this.

john
Jul 22 '05 #3

"ishekar" <is******@vsnl. net> wrote in message
news:c1******** **@news.mch.sbs .de...
Hi,

I have an application where i want to write data to a file, the data is
being sent from an external source. I know the total size of the data and
then i retrieve the data in small segments from the source. This data is
written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to 0 and start writing to file. whether there will be any performance improvements
because of the allocation of the file earlier and due to less io? or this
doesnt matter because every fwrite writes to disk(OS may schedule writing to disk).
2. If this is helps performance, do i have an api to initialise the file
with some size and empty strings?

1. Keep it simple - don't try to optimize until you know there is a problem
2. There is no way in C++
3. In unix like OSes the function you want is truncate or possibly ftruncate
4. On many file systems it will make no difference because although it will
say that the
file is so big it doesn't necessarily allocate the blocks until you
write to them.
5. The fastest way will almost certainly to use truncate and then map the
file into memory with mmap as it
will remove a layer of copying.
6. I think you really should just use ofstream TFI
ishekara.

Jul 22 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

48
8452
by: Joseph | last post by:
Hi I'm writing a commercial program which must be reliable. It has to do some basic reading and writing to and from files on the hard disk, and also to a floppy. I have foreseen a potential problem. The program may crash unexpectedly while writing to the file. If so, my program should detect this during startup, and then (during startup) probably delete the data added to the file and redo the writing operation.
6
23582
by: Sebastian Kemi | last post by:
How should a write a class to a file? Would this example work: object *myobject = 0; tfile.write(reinterpret_cast<char *>(myobject), sizeof(*object)); / sebek
1
717
by: Daniel | last post by:
System.IO.StreamWriter Close or Flush method to shut down the computer in such a way that just part of the file is written? or an empty file is written? Also if the Close or Flush is to a streamwriter writing to a network share, is it possible for the network to go down in such a way that the tartet file is only partialy written? or are there some kind of check sums to prevent this.
2
6847
by: melanieab | last post by:
Hi, I'm trying to store all of my data into one file (there're about 140 things to keep track of). I have no problem reading a specific string from the array file, but I wasn't sure how to replace just one item. I know I can get the entire array, then save the whole thing (with a for loop and if statements so that the changed data will be saved), but it seems like a lot of unnecessary reading and writing. Is there a way to directly save...
4
2172
by: HNguyen | last post by:
Hi, I have a Web application in ASP.NET. My Application allows the users upload files into the server after checking their user names and passwords. For each transaction, the Web program will write the information about user name, filename upload, filesize, date and time of uploading into the log file. (The name of the log file is constructed by Current Year and Current Month in my program). Is there any problems with writing into the...
0
1709
by: Yunus's Group | last post by:
Yunus's Group May 23, 3:36 pm show options Newsgroups: microsoft.public.dotnet.languages.vb From: "Yunus's Group" <yunusasm...@gmail.com> - Find messages by this author Date: 23 May 2005 12:36:14 -0700 Local: Mon,May 23 2005 3:36 pm Subject: Writing to text file Reply | Reply to Author | Forward | Print | Individual Message | Show original | Remove | Report Abuse
16
7171
by: Claudio Grondi | last post by:
I have a 250 Gbyte file (occupies the whole hard drive space) and want to change only eight bytes in this file at a given offset of appr. 200 Gbyte (all other data in that file should remain unchanged). How can I do that in Python? Claudio Grondi
6
5255
by: arne.muller | last post by:
Hello, I've come across some problems reading strucutres from binary files. Basically I've some strutures typedef struct { int i; double x; int n; double *mz;
3
2687
by: Barry Flynn | last post by:
Hi I am working with a VB 2005 program which has been converted from VB6. It writes data out to a flat file, with code like the following line WriteLine(riFileNo, "Hist", lsAssetID, lsRecordType, lsXNbr, lsFiscYr, "Beg", CStr(H.BegBalAccDepn), CStr(H.BegBalCost), CStr(H.BegBalCostReval), CStr(H.BegBalDepCost), CStr(H.BegBalDepnReval)) The program is running from within a Virtual PC
0
7886
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8272
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8381
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8258
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
5847
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5431
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
3886
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
2404
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
1238
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.