473,395 Members | 1,937 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,395 software developers and data experts.

fopen large file (tera byte) support?

Hello Group,

I have to support 200 Gigabyte support to open a file in my program.
My program exits with

open("/mnt/d1/tmp/<file name>", O_RDONLY) = -1 EFBIG (File too large)

This open system call is a result of fopen C++ library call, and there
is no way I can provide

O_LARGEFILE mode in fopen call.

Also the return valud of fopen is a file handle, where open system
call returns the file descriptor and hence I can not substitute one vs
other.

Is there a way I can get around this problem.

Thanks.

nagrik
Jun 27 '08 #1
4 6487
Sam
Nagrik writes:
Hello Group,

I have to support 200 Gigabyte support to open a file in my program.
My program exits with

open("/mnt/d1/tmp/<file name>", O_RDONLY) = -1 EFBIG (File too large)

This open system call is a result of fopen C++ library call, and there
is no way I can provide

O_LARGEFILE mode in fopen call.

Also the return valud of fopen is a file handle, where open system
call returns the file descriptor and hence I can not substitute one vs
other.

Is there a way I can get around this problem.
If you're using Linux and gcc, try compiling with -D_FILE_OFFSET_BITS=64

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEABECAAYFAkha/i8ACgkQx9p3GYHlUOLQLQCfTKJSgmVf1gro15i9wPK6oQcz
hw8AniJQT2ellR9k4mqo4JX1funQ3E/5
=UwfC
-----END PGP SIGNATURE-----

Jun 27 '08 #2
Hi nagrik,

i'm not sure if the C++ library file commands support large files on 32Bit,
fseek for example takes just an long int.
The way i would choose is to use raw system calls

Cheers

Markus

Nagrik wrote:
Hello Group,

I have to support 200 Gigabyte support to open a file in my program.
My program exits with

open("/mnt/d1/tmp/<file name>", O_RDONLY) = -1 EFBIG (File too large)

This open system call is a result of fopen C++ library call, and there
is no way I can provide

O_LARGEFILE mode in fopen call.

Also the return valud of fopen is a file handle, where open system
call returns the file descriptor and hence I can not substitute one vs
other.

Is there a way I can get around this problem.

Thanks.

nagrik
Jun 27 '08 #3
Nagrik wrote:
Hello Group,

I have to support 200 Gigabyte support to open a file in my program.
My program exits with

open("/mnt/d1/tmp/<file name>", O_RDONLY) = -1 EFBIG (File too large)

This open system call is a result of fopen C++ library call, and there
is no way I can provide

O_LARGEFILE mode in fopen call.

Also the return valud of fopen is a file handle, where open system
call returns the file descriptor and hence I can not substitute one vs
other.

Is there a way I can get around this problem.

Thanks.

nagrik
is fdopen OR _fdopen available on your compiler?

--
Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
Jun 27 '08 #4
Nagrik wrote:
Hello Group,

I have to support 200 Gigabyte support to open a file in my program.
My program exits with

open("/mnt/d1/tmp/<file name>", O_RDONLY) = -1 EFBIG (File too large)
comp.unix.programmer would be a better place to ask.

--
Ian Collins.
Jun 27 '08 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Mister Zimbu | last post by:
I'm having problems with a program I wrote- when it comes time to output a file, the call to fopen locks up and I have to break the program manually. I've pinpointed the actual stopping point to...
7
by: Joseph | last post by:
Hi, I'm having bit of questions on recursive pointer. I have following code that supports upto 8K files but when i do a file like 12K i get a segment fault. I Know it is in this line of code. ...
14
by: Aaron Couts | last post by:
I have a program that writes to a log file. It's compiled on RH Linux 7.3 (Kernel 2.4.18-18.7). It's using fopen in append mode. When the file reaches 51200000 bytes in size, the program will no...
15
by: uremae | last post by:
I tried to open some large files in my computer. (ram 512MB) 1. is there limitation of file size FOPEN? 2. if I have a file that is larger than the maximum size, how can I open the file?
3
by: A.M-SG | last post by:
Hi, I have a ASP.NET aspx file that needs to pass large images from a network storage to client browser. The requirement is that users cannot have access to the network share. The aspx file...
10
by: Peter Duniho | last post by:
This is kind of a question about C# and kind of one about the framework. Hopefully, there's an answer in there somewhere. :) I'm curious about the status of 32-bit vs 64-bit in C# and the...
14
by: Bryan Parkoff | last post by:
Do you know that current C++ Compiler limits to 64KB segments in source code? It is good news that Microsoft Visual C++ 2005 has expanded to 4GB segments in source code. 4GB segment is ideal for...
10
by: Julia | last post by:
Hi, there, I am trying to append a binary file by using: FILE *strean; stream = fopen( "c:\temp.dat", "ba+" )); Is there a way that I can check if the file exists or not before fopen,...
0
by: mgilligan | last post by:
Hi I am having a problem and can not for the life of me figure out what is wrong. I have a program that iterates through every file in a directory using FindFirstFile and FindNextFile and add them...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.