473,405 Members | 2,210 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,405 software developers and data experts.

What is the fastest way to do Data File I/O in C++

Hi everyone,

I am currently using Visual C++ 8 Standard.

I have a quite large data array (1GB) in the form of an unsigned char
array.
I need to save this to disk in 16-bit format (two chars form a 16-bit
value) or 32-bit format (half of 32 bit value empty).

I was wondering what class I should use to save this data as fast as
possible.
Saving it as bytes is not best, as unfortunately the reading program
(Matlab) will have to convert this to 16-bit, and this will take quite
long.

What matters is for this data to end up on disk as fast as possible in
16-bit or 32-bit format.

Any help would be greatly appreciated!

Thank you,
Best regards,
Tele

Aug 7 '07 #1
6 2879
On Aug 7, 6:48 am, elet.mir...@gmail.com wrote:
Hi everyone,

I am currently using Visual C++ 8 Standard.

I have a quite large data array (1GB) in the form of an unsigned char
array.
I need to save this to disk in 16-bit format (two chars form a 16-bit
value) or 32-bit format (half of 32 bit value empty).

I was wondering what class I should use to save this data as fast as
possible.
Saving it as bytes is not best, as unfortunately the reading program
(Matlab) will have to convert this to 16-bit, and this will take quite
long.

What matters is for this data to end up on disk as fast as possible in
16-bit or 32-bit format.

Any help would be greatly appreciated!

Thank you,
Best regards,
Tele
Use ofstream class.

Aug 7 '07 #2
On Aug 6, 7:55 pm, HumbleWorker <amardeep.develo...@gmail.comwrote:
On Aug 7, 6:48 am, elet.mir...@gmail.com wrote:
Hi everyone,
I am currently using Visual C++ 8 Standard.
I have a quite large data array (1GB) in the form of an unsigned char
array.
I need to save this to disk in 16-bit format (two chars form a 16-bit
value) or 32-bit format (half of 32 bit value empty).
I was wondering what class I should use to save this data as fast as
possible.
Saving it as bytes is not best, as unfortunately the reading program
(Matlab) will have to convert this to 16-bit, and this will take quite
long.
What matters is for this data to end up on disk as fast as possible in
16-bit or 32-bit format.
Any help would be greatly appreciated!
Thank you,
Best regards,
Tele

Use ofstream class.
Any particular reason why I should use ofstream?
As opposed for example to binarywriter or other.

Regards,
Tele

Aug 7 '07 #3
HumbleWorker wrote:
On Aug 7, 6:48 am, elet.mir...@gmail.com wrote:
>Hi everyone,

I am currently using Visual C++ 8 Standard.

I have a quite large data array (1GB) in the form of an unsigned char
array.
I need to save this to disk in 16-bit format (two chars form a 16-bit
value) or 32-bit format (half of 32 bit value empty).

I was wondering what class I should use to save this data as fast as
possible.
Saving it as bytes is not best, as unfortunately the reading program
(Matlab) will have to convert this to 16-bit, and this will take quite
long.

What matters is for this data to end up on disk as fast as possible in
16-bit or 32-bit format.

Any help would be greatly appreciated!

Use ofstream class.
That's probably bad advice!

The good advice is to ask on a windows group for the appropriate
platform specific API.

--
Ian Collins.
Aug 7 '07 #4
On Aug 7, 8:14 am, elet.mir...@gmail.com wrote:
On Aug 6, 7:55 pm, HumbleWorker <amardeep.develo...@gmail.comwrote:


On Aug 7, 6:48 am, elet.mir...@gmail.com wrote:
Hi everyone,
I am currently using Visual C++ 8 Standard.
I have a quite large data array (1GB) in the form of an unsigned char
array.
I need to save this to disk in 16-bit format (two chars form a 16-bit
value) or 32-bit format (half of 32 bit value empty).
I was wondering what class I should use to save this data as fast as
possible.
Saving it as bytes is not best, as unfortunately the reading program
(Matlab) will have to convert this to 16-bit, and this will take quite
long.
What matters is for this data to end up on disk as fast as possible in
16-bit or 32-bit format.
Any help would be greatly appreciated!
Thank you,
Best regards,
Tele
Use ofstream class.

Any particular reason why I should use ofstream?
As opposed for example to binarywriter or other.

Regards,
Tele- Hide quoted text -

- Show quoted text -
BinaryWriters are .NET classes, and they are wrappers around the
streams only. They may not be specifically wrapping around Standard C+
+ streams, but they definitely wrap around something conceptually
similar. If you set the constant ios::binary in the constructor for
fstream, you get a binary writer.

if you can send me your code i can revert back with a complete code.

Thanks.
HW

Aug 7 '07 #5
On Aug 6, 8:58 pm, HumbleWorker <amardeep.develo...@gmail.comwrote:
On Aug 7, 8:14 am, elet.mir...@gmail.com wrote:
On Aug 6, 7:55 pm, HumbleWorker <amardeep.develo...@gmail.comwrote:
On Aug 7, 6:48 am, elet.mir...@gmail.com wrote:
Hi everyone,
I am currently using Visual C++ 8 Standard.
I have a quite large data array (1GB) in the form of an unsigned char
array.
I need to save this to disk in 16-bit format (two chars form a 16-bit
value) or 32-bit format (half of 32 bit value empty).
I was wondering what class I should use to save this data as fast as
possible.
Saving it as bytes is not best, as unfortunately the reading program
(Matlab) will have to convert this to 16-bit, and this will take quite
long.
What matters is for this data to end up on disk as fast as possible in
16-bit or 32-bit format.
Any help would be greatly appreciated!
Thank you,
Best regards,
Tele
Use ofstream class.
Any particular reason why I should use ofstream?
As opposed for example to binarywriter or other.
Regards,
Tele- Hide quoted text -
- Show quoted text -

BinaryWriters are .NET classes, and they are wrappers around the
streams only. They may not be specifically wrapping around Standard C+
+ streams, but they definitely wrap around something conceptually
similar. If you set the constant ios::binary in the constructor for
fstream, you get a binary writer.

if you can send me your code i can revert back with a complete code.

Thanks.
HW
Thanks, this information helps.
I didn't know BinaryWriters wrapped around streams, but I know how to
use ios::binary.
Thanks!
Tele

Aug 7 '07 #6
I am currently using Visual C++ 8 Standard.

fstream are quite slow on Visual Studio's implementation. I found
fopen/fwrite to be pretty fast, but the API might be an even better
choice.
comp.os.ms-windows.programmer.win32 might be a place to ask.
Aug 7 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: SP | last post by:
I've production sql server 7 sp3 on windows NT. I had a 8GB data file of which 5GB were used and 3GB were unused. I wanted to take back the unused 3GB. So I did the following with EM GUI: 1. I...
7
by: Bob | last post by:
Hi, I am trying to use BULK INSERT with format file. All of our data has few bytes of header in the data file which I would like to skip before doing BULK INSERT. Is it possible to write...
11
by: Todd | last post by:
I have been tasked to create an app that will be run from a CD on stand-alone machines that needs to search and retrieve data from an XML Data Island. The best approach that I can see to handle...
12
by: Steven Bethard | last post by:
Ok, so I have a module that is basically a Python wrapper around a big lookup table stored in a text file. The module needs to provide a few functions:: get_stem(word, pos, default=None)...
2
by: nepdae | last post by:
Please forgive me, this is a long one. My 11-user Access 2000 database is having recurring corruption problems. The symptoms include the following: 1) corrupted fields in recently created or...
10
by: HardySpicer | last post by:
I have a cvs data file orbit.csv and found this code. function ReadFile() { var fso, f1, ts, s; var ForReading = 1; fso = new ActiveXObject("Scripting.FileSystemObject"); f =...
2
by: phpnoob | last post by:
I have a php script that processes a form and then posts the user input to a data file on the server in a comma delimited format. For simplicity call the file "data.csv." The script is working...
3
by: =?Utf-8?B?YzY3NjIyOA==?= | last post by:
Hi all, I have a question for you. I have a .csv file which has many lines of data. Each line has many data fields which are delimited by ",". Now I need to extract part of data from this...
1
by: star01daisy | last post by:
This is what i have, but when i run the program the only thing that comes out is the inro line "This program opens...". What do i have to do for it to run smoothly.The program is supposed to sort the...
0
by: blehhhhh | last post by:
hi guys, So i have this java assignment where I have to create an employee class with given datatypes and have accessor methods in it. Along with that i have to create a text file(data file) with...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.