Hello!
This is my first time dealing with Very Large Files.
I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.
All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits<unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.
So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.
How does one use this streampos type? sizeof(streampos)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits<streampos>::max() but it says 0.
I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?
I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?
I tried this:
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << sp << endl;
but this throws (!) an exception with what() of:
"basic_string::_S_construct NULL not valid".
Does that mean that cout cannot be used with this streampos type?
So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..12000000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?
TIA,
- J. 1 8199
Jacek Dziedzic wrote:
>
Hello!
This is my first time dealing with Very Large Files.
I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.
All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits<unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.
So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.
How does one use this streampos type? sizeof(streampos)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits<streampos>::max() but it says 0.
I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?
I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?
I tried this:
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << sp << endl;
but this throws (!) an exception with what() of:
"basic_string::_S_construct NULL not valid".
Does that mean that cout cannot be used with this streampos type?
So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..12000000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?
TIA,
- J.
I just found out about streamoff. After a bit of testing,
numeric_limits<streamoff>::max() successfully prints
a Very Large Number, which is good. However this
still
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << (streamoff)sp << endl;
throws, as earlier.
- J. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Bengt Richter |
last post by:
>>> hex(-5)
__main__:1: FutureWarning: hex()/oct() of negative int will return a signed string in Python 2.4
and up
'0xfffffffb'
>>> hex(-5)
'0xfffffffb'
>>> hex(-5L)
'-0x5L'
That is sooo...
|
by: ^_^ |
last post by:
conversion from:
a="a";
to
a=0x????;
If there are many unicode strings to convert, how can I do batch-conversion?
|
by: Piotre Ugrumov |
last post by:
I have done the overload on the operator >> and << in the class Attore.
These 2 overload work correctly.
I have done the overload of the same overload in the class Film. The class
film ha inside...
|
by: Gianni Mariani |
last post by:
The spirit of this arguably pointless exercise, is that the
numeric_limits<T> class could be replaced with a totally generic
template of compile-time, template computed constants.
The problem is...
|
by: Simon |
last post by:
Hi,
I am trying to create a round kind of function, (to round float and double
to int).
In the H file I can do the following with not too much trouble...
//
#include <limits>
#include...
|
by: liam_herron |
last post by:
Does anyone know why this is? Is there a compile flag to have this
return 'inf' as opposed to zero? Any help would be much appreciated.
I have included a sample program with sample output:
...
|
by: Peter Lee |
last post by:
ASSUME unsigned int = 32 bits
#q1 unsigned int u1 = 3u << 0u valid?
#q2 unsigned int u2 = 3u >0u valid?
#q3 unsigned int u3 = 3u << 32u valid?
#q4 unsigned int u4 = 3u >32u valid?
|
by: john |
last post by:
The code:
#include <iostream>
#include <limits>
int main()
{
using namespace std;
|
by: barcaroller |
last post by:
In the boost::program_options tutorial, the author included the following
code:
cout << "Input files are: "
<< vm.as< vector<string()
<< "\n";
Basically, he is trying to print a vector...
|
by: DolphinDB |
last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation.
Take...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: ArrayDB |
last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
|
by: PapaRatzi |
last post by:
Hello,
I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: Defcon1945 |
last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
|
by: af34tf |
last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
| | |