By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
448,589 Members | 1,202 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 448,589 IT Pros & Developers. It's quick & easy.

streampos difficulties, tellg() > numeric_limits<unsigned long int>::max()

P: n/a

Hello!

This is my first time dealing with Very Large Files.

I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.

All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits<unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.

So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.

How does one use this streampos type? sizeof(streampos)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits<streampos>::max() but it says 0.

I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?

I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?

I tried this:
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << sp << endl;

but this throws (!) an exception with what() of:
"basic_string::_S_construct NULL not valid".

Does that mean that cout cannot be used with this streampos type?

So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..12000000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?

TIA,
- J.
Feb 6 '07 #1
Share this Question
Share on Google+
1 Reply


P: n/a
Jacek Dziedzic wrote:
>
Hello!

This is my first time dealing with Very Large Files.

I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.

All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits<unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.

So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.

How does one use this streampos type? sizeof(streampos)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits<streampos>::max() but it says 0.

I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?

I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?

I tried this:
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << sp << endl;

but this throws (!) an exception with what() of:
"basic_string::_S_construct NULL not valid".

Does that mean that cout cannot be used with this streampos type?

So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..12000000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?

TIA,
- J.
I just found out about streamoff. After a bit of testing,
numeric_limits<streamoff>::max() successfully prints
a Very Large Number, which is good. However this
still

ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << (streamoff)sp << endl;

throws, as earlier.

- J.
Feb 6 '07 #2

This discussion thread is closed

Replies have been disabled for this discussion.