maximum file size of FOPEN

[uremae]

I tried to open some large files in my computer. (ram 512MB)

1. is there limitation of file size FOPEN?

2. if I have a file that is larger than the maximum size, how can I
open the file?

[Walter Roberson]

In article <11************ **********@z14g 2000cwz.googleg roups.com>,
uremae <ur****@gmail.c om> wrote:

I tried to open some large files in my computer. (ram 512MB) 1. is there limitation of file size FOPEN?
No, not unless there is something unusual and system-specific.

There could, for example, -potentially- be issues with
opening a device or pseudo-device, if the device driver for some
reason decided it needed to take a snapshot into memory
(e.g., opening /dev/core might take a snapshot of kernel state,)
Any such behaviour would be outside the bounds of C: such
behaviour does not happen for plain files.

2. if I have a file that is larger than the maximum size, how can I
open the file?

Any such matter would be system specific. fopen() is all that
C itself provides.

It isn't usually a problem to *open* an existing large file: the
problem is usually in *reading* the large file. Some systems
are only able to read to about the 2 gigabyte mark, or are
allowed to read indefinitely but cannot position (ftell/fseek)
beyond 2 gigabytes without using system-specific calls.
--
"Never install telephone wiring during a lightning storm." -- Linksys

[Malcolm]

"uremae" <ur****@gmail.c om> wrote

1. is there limitation of file size FOPEN?

2. if I have a file that is larger than the maximum size, how can I
open the file?

There's ususally some limit, though it might be so large that there is no
chance of exceeding it.
If fopen() fails to open a huge file, or the read functions won't allow you
to read it all, the best solution is to look for lower-level system specific
calls. Sometimes you might be able to redesign your files so that they fall
within the limit.

However make sure that the size of the file really is the problem, not that
there is some hardware problem with the disk drive, or that the file is
corrupt in some way.

[Walter Roberson]

In article <d6**********@n wrdmz02.dmz.ncs .ea.ibs-infra.bt.com>,
Malcolm <re*******@btin ternet.com> wrote:

"uremae" <ur****@gmail.c om> wrote

1. is there limitation of file size FOPEN?

There's ususally some limit, though it might be so large that there is no
chance of exceeding it.

Could you give us an example of a system with such a limit, Malcolm ?

I have certainly run into systems whose filesize was limited
(with different limits for different filesystem types), but I
cannot think of anything in C or POSIX.1 or any implementation
that I have -encountered- that would fopen() for opening a
[plain] file for read once the file had made it on to the
filesystem.

Ability to read all of a large file is questionable, though:
the internal file position counter can overflow (e.g., if one
is NFS or SMB'ing over the contents of a file which resides on
a remote filesystem that supports much larger files than the local
system expects.)

And ability to position/ reposition in a large file is quite
questionable seeing as fseek() is limited to taking a 'long' for the
offset...
--
Beware of bugs in the above code; I have only proved it correct,
not tried it. -- Donald Knuth

[Mark McIntyre]

On 19 May 2005 21:57:05 GMT, in comp.lang.c ,
ro******@ibd.nr c-cnrc.gc.ca (Walter Roberson) wrote:

In article <d6**********@n wrdmz02.dmz.ncs .ea.ibs-infra.bt.com>,
Malcolm <re*******@btin ternet.com> wrote:

"uremae" <ur****@gmail.c om> wrote

1. is there limitation of file size FOPEN?

There's ususally some limit, though it might be so large that there is no
chance of exceeding it.

Could you give us an example of a system with such a limit, Malcolm ?

I have certainly run into systems whose filesize was limited
(with different limits for different filesystem types), but I
cannot think of anything in C or POSIX.1 or any implementation
that I have -encountered- that would fopen() for opening a
[plain] file for read once the file had made it on to the
filesystem.

The file could exist on network attached storage which supports larger
file sizes than the local OS - f'rexample you could have Win95
locally, and be mapping a W2k3 or *nix drive.
..
--
Mark McIntyre
CLC FAQ <http://www.eskimo.com/~scs/C-faq/top.html>
CLC readme: <http://www.ungerhu.com/jxh/clc.welcome.txt >

[uremae]

then how can I read the large file that is larger than gigabye with
what system-specific calls in linux or unix.

[Walter Roberson]

In article <p7************ *************** *****@4ax.com>,
Mark McIntyre <ma**********@s pamcop.net> wrote:

On 19 May 2005 21:57:05 GMT, in comp.lang.c ,
ro******@ibd.n rc-cnrc.gc.ca (Walter Roberson) wrote:

In article <d6**********@n wrdmz02.dmz.ncs .ea.ibs-infra.bt.com>,
Malcolm <re*******@btin ternet.com> wrote:

"uremae" <ur****@gmail.c om> wrote 1. is there limitation of file size FOPEN? There's ususally some limit, though it might be so large that there is no
chance of exceeding it.

Could you give us an example of a system with such a limit, Malcolm ?

The file could exist on network attached storage which supports larger
file sizes than the local OS

I see that Malcolm was correct. Checking the opengroup.org fopen() man
page, I see that one of the defined error returns is

[EOVERFLOW]
The named file is a regular file and the size of the file
cannot be represented correctly in an object of type off_t.
--
"Never install telephone wiring during a lightning storm." -- Linksys

[Walter Roberson]

In article <11************ *********@g43g2 000cwa.googlegr oups.com>,
uremae <ur****@gmail.c om> wrote:

then how can I read the large file that is larger than gigabye with
what system-specific calls in linux or unix.

You are guaranteed up to (2 gigabytes - 1) in any conforming
unix system (provided the filesystem supports files that large.)

There is no standard unix interface for reading files whose
size cannot be represented as off_t .

In Linux, if your version of Linux has Large Files System support,
then you can open() using the O_LARGEFILE flag.
--
"Mathematic s? I speak it like a native." -- Spike Milligan

[Peter Nilsson]

Walter Roberson wrote:

Ability to read all of a large file is questionable, though:
the internal file position counter can overflow (e.g., if one
is NFS or SMB'ing over the contents of a file which resides on
a remote filesystem that supports much larger files than the local
system expects.)
AFAICS, the counter can't overflow per se (this implies undefined
behaviour). Rather, an attempt to get the current file position can
fail.
And ability to position/ reposition in a large file is quite
questionable seeing as fseek() is limited to taking a 'long'
for the offset...

You are allowed to perform multiple seeks and keep your own file
position data.

--
Peter

[William Ahern]

uremae <ur****@gmail.c om> wrote:

then how can I read the large file that is larger than gigabye with
what system-specific calls in linux or unix.

The folks over in comp.os.linux.d evelopment.apps and comp.unix.progr ammer
would be more helpful since this is a really platform dependent thing.

On more recent Linux platforms you would typically define _FILE_OFFSET_BI TS
to 64 before you include _any_ standard or system header. This would then
automagically select the 64 bit interface allowing for > 2GB file access for
those interfaces which normally wouldn't be able to handle files that large.

For example:

gcc -D_FILE_OFFSET_B ITS=64 -o main main.c

- Bill