473,842 Members | 1,439 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

streampos difficulties, tellg() > numeric_limits< unsigned long int>::max()


Hello!

This is my first time dealing with Very Large Files.

I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.

All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits< unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.

So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.

How does one use this streampos type? sizeof(streampo s)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits< streampos>::max () but it says 0.

I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?

I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?

I tried this:
ifstream in("myfile");
in.seekg(0,ios_ base::end);
streampos sp = in.tellg();
cout << sp << endl;

but this throws (!) an exception with what() of:
"basic_string:: _S_construct NULL not valid".

Does that mean that cout cannot be used with this streampos type?

So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..120 00000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?

TIA,
- J.
Feb 6 '07 #1
1 8226
Jacek Dziedzic wrote:
>
Hello!

This is my first time dealing with Very Large Files.

I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.

All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits< unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.

So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.

How does one use this streampos type? sizeof(streampo s)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits< streampos>::max () but it says 0.

I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?

I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?

I tried this:
ifstream in("myfile");
in.seekg(0,ios_ base::end);
streampos sp = in.tellg();
cout << sp << endl;

but this throws (!) an exception with what() of:
"basic_string:: _S_construct NULL not valid".

Does that mean that cout cannot be used with this streampos type?

So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..120 00000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?

TIA,
- J.
I just found out about streamoff. After a bit of testing,
numeric_limits< streamoff>::max () successfully prints
a Very Large Number, which is good. However this
still

ifstream in("myfile");
in.seekg(0,ios_ base::end);
streampos sp = in.tellg();
cout << (streamoff)sp << endl;

throws, as earlier.

- J.
Feb 6 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
2145
by: Bengt Richter | last post by:
>>> hex(-5) __main__:1: FutureWarning: hex()/oct() of negative int will return a signed string in Python 2.4 and up '0xfffffffb' >>> hex(-5) '0xfffffffb' >>> hex(-5L) '-0x5L' That is sooo ugly. I suppose it is for a backwards compatible repr, but couldn't we
16
637
by: ^_^ | last post by:
conversion from: a="a"; to a=0x????; If there are many unicode strings to convert, how can I do batch-conversion?
3
2253
by: Piotre Ugrumov | last post by:
I have done the overload on the operator >> and << in the class Attore. These 2 overload work correctly. I have done the overload of the same overload in the class Film. The class film ha inside an array of pointer to Attore. I have written these overload in these ways. the overload of << work correctly the overload of >> I don't know. I compile the class correctly, when I insert a film through the operator >> I don't receive error, but...
5
2184
by: Gianni Mariani | last post by:
The spirit of this arguably pointless exercise, is that the numeric_limits<T> class could be replaced with a totally generic template of compile-time, template computed constants. The problem is that it takes a *long* time to compile and the "long double" version instantiates so many classes that gcc runs out of memory on my 1 gig machine ! So the question is - is there a simple way to reduce the number of template instantiations from...
4
2670
by: Simon | last post by:
Hi, I am trying to create a round kind of function, (to round float and double to int). In the H file I can do the following with not too much trouble... // #include <limits> #include <math.h>
1
2633
by: liam_herron | last post by:
Does anyone know why this is? Is there a compile flag to have this return 'inf' as opposed to zero? Any help would be much appreciated. I have included a sample program with sample output: #include <limits> #include <iostream> using namespace std;
2
2420
by: Peter Lee | last post by:
ASSUME unsigned int = 32 bits #q1 unsigned int u1 = 3u << 0u valid? #q2 unsigned int u2 = 3u >0u valid? #q3 unsigned int u3 = 3u << 32u valid? #q4 unsigned int u4 = 3u >32u valid?
4
4445
by: john | last post by:
The code: #include <iostream> #include <limits> int main() { using namespace std;
42
4575
by: barcaroller | last post by:
In the boost::program_options tutorial, the author included the following code: cout << "Input files are: " << vm.as< vector<string() << "\n"; Basically, he is trying to print a vector of string, in one line. I could
0
9876
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9717
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10950
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10617
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10681
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10318
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9459
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5886
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
4096
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.