Hello,
I have code written using the basic fstream object on a Win32 system. This
is of course simply a typedef for basic_fstream<c har, char_traits<cha r.
This object can not be used to write or read files that are larger than 2Gb
in size, as it would appear (from looking at the iosfwd header) that the
underlying streamoff and streamsize types are typedef'd to long and int. On
Win64 systems, these are typedef'd to __int64, so it seems that on Win64
systems fstream can in fact be used to read and write large files.
However, is there a way I can define a new type and define a char_traits for
this type so that I can use basic_fstream to read and write large files on
Win32? Looking into the std c++ iostream headers, it is not clear that this
is in fact doable, as many things seem to depend on these basic streamoff
and streamsize typedefs, which are not mutable in my own class.
I would like to know whether I am reading this correctly, or whether people
have had success in doing this, i.e. creating a char_traits type that when
used with basic_fstream will allow for large (>2Gb) file reading and
writing.
Thanks,
-Eric Twietmeyer
(MSDN no-spam alias: MS***********@n ospam.nospam) 3  3082 
Eric Twietmeyer wrote:
:: Hello,
::
:: I have code written using the basic fstream object on a Win32
:: system. This is of course simply a typedef for basic_fstream<c har,
:: char_traits<cha r.
::
:: This object can not be used to write or read files that are larger
:: than 2Gb in size, as it would appear (from looking at the iosfwd
:: header) that the underlying streamoff and streamsize types are
:: typedef'd to long and int. On Win64 systems, these are typedef'd to
:: __int64, so it seems that on Win64 systems fstream can in fact be
:: used to read and write large files.
::
:: However, is there a way I can define a new type and define a
:: char_traits for this type so that I can use basic_fstream to read
:: and write large files on Win32? Looking into the std c++ iostream
:: headers, it is not clear that this is in fact doable, as many things
:: seem to depend on these basic streamoff and streamsize typedefs,
:: which are not mutable in my own class.
::
:: I would like to know whether I am reading this correctly, or whether
:: people have had success in doing this, i.e. creating a char_traits
:: type that when used with basic_fstream will allow for large (>2Gb)
:: file reading and writing.
::
You CAN read and write larger files, as long as you do it sequentially. The
size of streamoff limits the amount of positioning you can do. Do you really
do seeks with offsets >2GB?
Bo Persson
"Bo Persson" <bo*@gmb.dkwrot e in message
news:56******** *****@mid.indiv idual.net...
Eric Twietmeyer wrote:
:: Hello,
::
:: I have code written using the basic fstream object on a Win32
:: system. This is of course simply a typedef for basic_fstream<c har,
:: char_traits<cha r.
::
:: This object can not be used to write or read files that are larger
:: than 2Gb in size, as it would appear (from looking at the iosfwd
:: header) that the underlying streamoff and streamsize types are
:: typedef'd to long and int. On Win64 systems, these are typedef'd to
:: __int64, so it seems that on Win64 systems fstream can in fact be
:: used to read and write large files.
::
:: However, is there a way I can define a new type and define a
:: char_traits for this type so that I can use basic_fstream to read
:: and write large files on Win32? Looking into the std c++ iostream
:: headers, it is not clear that this is in fact doable, as many things
:: seem to depend on these basic streamoff and streamsize typedefs,
:: which are not mutable in my own class.
::
:: I would like to know whether I am reading this correctly, or whether
:: people have had success in doing this, i.e. creating a char_traits
:: type that when used with basic_fstream will allow for large (>2Gb)
:: file reading and writing.
::
You CAN read and write larger files, as long as you do it sequentially.
The size of streamoff limits the amount of positioning you can do. Do you
really do seeks with offsets >2GB?
Bo Persson
Yes, unfortunately, I do in fact need to be able to offset in the stream to
locations greater than 2Gb from the current point.
I'm also just curious how one may specialize the basic stream / buf objects
using different char_traits, whether that is actually doable given this
implementation provided with DevStudio, or whether it is expected that only
char_traits<cha rand char_traits<wch ar_twould be used, and no others.
-Eric
Eric Twietmeyer wrote:
:: "Bo Persson" <bo*@gmb.dkwrot e in message
:: news:56******** *****@mid.indiv idual.net...
::: Eric Twietmeyer wrote:
::::: Hello,
:::::
::::: I have code written using the basic fstream object on a Win32
::::: system. This is of course simply a typedef for
::::: basic_fstream<c har, char_traits<cha r.
:::::
::::: This object can not be used to write or read files that are larger
::::: than 2Gb in size, as it would appear (from looking at the iosfwd
::::: header) that the underlying streamoff and streamsize types are
::::: typedef'd to long and int. On Win64 systems, these are typedef'd
::::: to __int64, so it seems that on Win64 systems fstream can in fact
::::: be used to read and write large files.
:::::
::::: However, is there a way I can define a new type and define a
::::: char_traits for this type so that I can use basic_fstream to read
::::: and write large files on Win32? Looking into the std c++ iostream
::::: headers, it is not clear that this is in fact doable, as many
::::: things seem to depend on these basic streamoff and streamsize
::::: typedefs, which are not mutable in my own class.
:::::
::::: I would like to know whether I am reading this correctly, or
::::: whether people have had success in doing this, i.e. creating a
::::: char_traits type that when used with basic_fstream will allow for
::::: large (>2Gb) file reading and writing.
:::::
:::
::: You CAN read and write larger files, as long as you do it
::: sequentially. The size of streamoff limits the amount of
::: positioning you can do. Do you really do seeks with offsets >2GB?
:::
:::
::: Bo Persson
:::
:::
::
:: Yes, unfortunately, I do in fact need to be able to offset in the
:: stream to locations greater than 2Gb from the current point.
Too bad.
The Band-Aid solution is to do multiple seeks, 2 GB at a time. Works for
moderate file sizes.
::
:: I'm also just curious how one may specialize the basic stream / buf
:: objects using different char_traits, whether that is actually doable
:: given this implementation provided with DevStudio, or whether it is
:: expected that only char_traits<cha rand char_traits<wch ar_twould
:: be used, and no others.
It is possible, but hard. It will probably not help you here, anyway.
Eventually, all the seeks will probably come down to a C level fseek(),
which also takes a long parameter. The real solution is to have a 'long'
type that is actually sort-of-long, and not just a different name for 'int'.
A bit late for that now, unfortunately.
Bo Persson
This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Avner |
last post by:
I have a C++ code which I want to upgrade to the standard C++
The code constructs a file and sets the file permission
The new iostream library does not include filebuf::openprot
The old ofstream constructor (<fstream.h>) accepts a third parameter (int
nProt = filebuf::openprot)
i.e. ofstream( const char* szName, int nMode = ios::out, int nProt =
filebuf::openprot );
|
by: Greg |
last post by:
I am working on a project that will have about 500,000 records in an XML
document. This document will need to be queried with XPath, and records
will need to be updated. I was thinking about splitting up the XML into
several XML documents (perhaps 50,000 per document) to be more efficient but
this will make things a lot more complex because the searching needs to go
accross all 500,000 records. Can anyone point me to some best practices...
|
by: Buddy Ackerman |
last post by:
I'm trying to write files directly to the client so that it forces the client to open the Save As dialog box rather than
display the file. On some occasions the files are very large (100MB+). On these files teh time that it takes until the
client displays the Save As dialog can be extrordinarily long (3+ minutes). I don't understand why.
I was initiall using the format:
Respnse.writefile("filepath", offset, length)
but that simply...
|
by: A.M-SG |
last post by:
Hi,
I have a ASP.NET aspx file that needs to pass large images from a network
storage to client browser. The requirement is that users cannot have access
to the network share. The aspx file must be the only method that users
receive image files.
|
by: jdev8080 |
last post by:
We are looking at creating large XML files containing binary data
(encoded as base64) and passing them to transformers that will parse
and transform the data into different formats.
Basically, we have images that have associated metadata and we are
trying to develop a unified delivery mechanism. Our XML documents may
be as large as 1GB and contain up to 100,000 images.
My question is, has anyone done anything like this before?
| | |
by: Lars B |
last post by:
Hey guys,
I have written a C++ program that passes data from a file to an FPGA board and back again using software and DMA buffers.
In my program I need to compare the size of a given file against a software buffer of size 3MB. This is needed so as to see which function to use to read from the file. As the files used range from very large (>30GB) to very small (<3MB), I have enabled large file support and I obtain the file size by using the...
|
by: theCancerus |
last post by:
Hi All,
I am not sure if this is the right place to ask this question but i am
very sure you may have faced this problem, i have already found some
post related to this but not the answer i am looking for.
My problem is that i have to upload images and store them. I am using
filesystem for that.
setup is something like this, their will be items/groups/user each can
|
by: =?Utf-8?B?UVNJRGV2ZWxvcGVy?= |
last post by:
Using .NET 2.0 is it more efficient to copy files to a single folder versus
spreading them across multiple folders.
For instance if we have 100,000 files to be copied,
Do we copy all of them to a single folder called 'All Files'
Do we spread them out and copy them to multiple folders like
Folder 000 - Copy files from 0 to 1000
Folder 001 - Copy files from 1000 to 2000
Folder 002 - Copy files from 2000 to 2999
|
by: byte8bits |
last post by:
How does C++ safely open and read very large files? For example, say I
have 1GB of physical memory and I open a 4GB file and attempt to read
it like so:
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main () {
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
