Large Files

[raj]

Hi friends,

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"

and the secod 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

Does anybody know how?

Thanks for answering!

[James Kuyper]

raj wrote:

Hi friends,

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"

and the secod 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

Does anybody know how?

Go to groups.google.com and search comp.lang.c for messages with "large
files" in the name. The most recent occurrence was 2007-11-08.

[Malcolm McLean]

"raj" <ra*@spamtrap.invalidwrote in message

Hi friends,

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"

and the secod 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

Does anybody know how?

Thanks for answering!

A long will give you 2G of space. Since you only need 4/5 G for the for the
"Hello" and another 4/5 for the "World" you are just within limits.

It would be prudent to check ferror after each call to fprintf / fwrite,
since it is not unlikely that the filesystem cannot support such large
files, or will run out of space. However it is just an ordinary C function
call job, not different in any way from if the requirement was to write 1K
or each.
That assumes a cross-platform question. Particular architecures may have
poor standard libraries that require special calls for large files. You
can't reasonably be expected to know all these details, though questioner
might not realise that - in which case it is tricky social but not technical
situation.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

[Ian Collins]

Malcolm McLean wrote:

"raj" <ra*@spamtrap.invalidwrote in message

>Hi friends,

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"

and the secod 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

Does anybody know how?

Thanks for answering!

A long will give you 2G of space.

Says who?

--
Ian Collins.

[Eric Sosman]

raj wrote:

Hi friends,

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"

and the secod 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

Does anybody know how?

Output 800000000 copies of "Hello", then output 800000000
copies of "World". Finally, use ferror() to see whether any
I/O errors occurred, and make sure fclose() succeeds before
your program declares success.

Notes:

1) The symbol "4GB" usually means 4294967296 to computer
people, but the task would be impossible if that were the
case in this instance: both "Hello" and "World" are five
bytes long, and 4294967296 is not divisible by five. Therefore
the prefix "G" presumably denotes its meaning under international
standards, namely, 1000000000. The assignment therefore calls
for 4000000000 bytes to be filled with each word, not 4294967296.
Besides making the task possible, this observation will make your
program run about seven percent faster; be sure to point this
out to the interviewer, who will be impressed with your devotion
to efficiency.

2) Since the task does not mention writing any newline
characters, the output cannot be a well-formed text stream
because each line of such a stream ends with a '\n'. (Even
on systems where an unterminated line is allowed, the length
of the generated line would exceed the portable limit.) So
we conclude that the output is to be a binary stream; keep
this in mind when you call fopen().

--
Eric Sosman
es*****@ieee-dot-org.invalid

[Tor Rustad]

raj wrote:

Hi friends,

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"

and the secod 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

Does anybody know how?

#include <stdio.h>
#include <stdlib.h>

#define FNAME "big-file"

int main(void)
{
int rc=EXIT_FAILURE, i,j,k;
FILE *out = fopen(FNAME,"w+");

if (out != NULL)
{
printf("Writing 4Gb 'Hello' to file '%s'...\n", FNAME);

for (i=0; i<4*1024; i++)
for (j=0; j<1024; j++)
for (k=0; k<1024; k++)
fprintf(out, "%c", "Hello"[k%5]);

printf("Writing 4Gb 'World' to file '%s'...\n", FNAME);
for (i=0; i<4*1024; i++)
for (j=0; j<1024; j++)
for (k=0; k<1024; k++)
fprintf(out, "%c", "World"[k%5]);

fclose(out);
rc = EXIT_SUCCESS;
}
return rc;
}
--
Tor <bw****@wvtqvm.vw | tr i-za-h a-z>

[Tor Rustad]

Eric Sosman wrote:

[...]

>
Output 800000000 copies of "Hello", then output 800000000
copies of "World". Finally, use ferror() to see whether any
I/O errors occurred, and make sure fclose() succeeds before
your program declares success.

Good point, I forgot to call ferror()! :)

Notes:

1) The symbol "4GB" usually means 4294967296 to computer
people, but the task would be impossible if that were the
case in this instance: both "Hello" and "World" are five
bytes long, and 4294967296 is not divisible by five. Therefore
the prefix "G" presumably denotes its meaning under international
standards, namely, 1000000000. The assignment therefore calls

Not agreeing here, filling don't mean the last word has to be "Hello"
and "World".

Hence, if using the 1000x1000x1000 or the 1024x1024x1024 definition of
gigabyte, shouldn't make a difference.

2) Since the task does not mention writing any newline
characters, the output cannot be a well-formed text stream
because each line of such a stream ends with a '\n'. (Even
on systems where an unterminated line is allowed, the length
of the generated line would exceed the portable limit.) So
we conclude that the output is to be a binary stream; keep
this in mind when you call fopen().

Another good point.

--
Tor <bw****@wvtqvm.vw | tr i-za-h a-z>

[Ben Pfaff]

Tor Rustad <to********@hotmail.comwrites:

for (i=0; i<4*1024; i++)
for (j=0; j<1024; j++)
for (k=0; k<1024; k++)
fprintf(out, "%c", "Hello"[k%5]);

1024 is not evenly divisible by 5, so this will lead to a uneven
boundary between the end of one kilobyte of output and the start
of the next.
--
"What is appropriate for the master is not appropriate for the novice.
You must understand the Tao before transcending structure."
--The Tao of Programming

[Paul Hsieh]

On Dec 16, 2:24 pm, raj <[e-mail]:

In an interview I was asked to write a C program to create a large file
of 8GB

The first 4GB is filled with "Hello"
and the second 4GB is filled with "World"

Sorry to say that I don't know how to do that in an elegant way. I think
it is a trick question depending on if size_t is 32 bits or 64 bits.

The way to deal with 32 bits elegantly, is to use 64 bits:

#include <stdio.h>
#include <stdlib.h>
#include "pstdint.h" /* http://www.pobox.com/~qed/pstdint.h */

int write4GB (char * rept, FILE * fp) {
int64_t ofs;
size_t slen = strlen (rept);

for (ofs = slen;
ofs < INT64_C(4294967296);
ofs += slen) {
fprintf (fp, "%s", rept);
if (ferror (fp)) return -__LINE__;
}
rept[(size_t) (INT64_C(4294967296)+slen-ofs)] = '\0';
fprintf (fp, "%s", rept);
if (ferror (fp)) return -__LINE__;
return 0;
}

int main () {
char hello[] = "Hello";
char world[] = "World";
FILE * fp = fopen ("file.txt", "w");
int ret = EXIT_FAILURE;

if (fp) {
if (0 == write4GB (hello, fp) && 0 == write4GB (world, fp))
ret = EXIT_SUCCESS;
fclose (fp);
}
return ret;
}

You could solve this with 32 bits and a do { ... } while(), but you
know what? Life is too short, and you are IO limited anyways.

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

[Malcolm McLean]

"Ian Collins" <ia******@hotmail.comwrote in message

Malcolm McLean wrote:

>A long will give you 2G of space.

Says who?

ANSI / ISO/

[Walter Roberson]

In article <__******************************@bt.com>,
Malcolm McLean <re*******@btinternet.comwrote:

>
"Ian Collins" <ia******@hotmail.comwrote in message

>Malcolm McLean wrote:

>>A long will give you 2G of space.

Says who?

ANSI / ISO/

Perhaps you meant "at least" in your original statement.
The C standards permit larger long.
--
So you found your solution
What will be your last contribution?
-- Supertramp (Fool's Overture)

[Roland Pibinger]

On Sun, 16 Dec 2007 23:12:57 -0800 (PST), Paul Hsieh wrote:

if (fp) {
if (0 == write4GB (hello, fp) && 0 == write4GB (world, fp))
ret = EXIT_SUCCESS;
fclose (fp);
}
return ret;

Ignoring the return value of fclose (fp) means that some error
conditons are reported as success.
--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch

[Tor Rustad]

Ben Pfaff wrote:

Tor Rustad <to********@hotmail.comwrites:

> for (i=0; i<4*1024; i++)
for (j=0; j<1024; j++)
for (k=0; k<1024; k++)
fprintf(out, "%c", "Hello"[k%5]);

1024 is not evenly divisible by 5, so this will lead to a uneven

5 isn't a factor in 2^64 either. :)

boundary between the end of one kilobyte of output and the start
of the next.

Yup, which was the reason I didn't print the whole word on each
fprintf() call.

--
Tor <bw****@wvtqvm.vw | tr i-za-h a-z>