473,231 Members | 1,841 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,231 software developers and data experts.

Slow file copying in C program?

Hi.

I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?

/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;

/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;

/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}

/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);
}

Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.

Aug 26 '07 #1
18 2342

"mike3" <mi******@yahoo.comwrote in message
news:11*********************@l22g2000prc.googlegro ups.com...
Hi.

I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?

/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;

/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;

/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}

/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);
}

Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.
There could be several reasons.

Try this

void copy(char *in, char *out)
{
FILE *fpin = fopen(in, "rb);
FILE *fpout = fopen(out, "wb");
int ch;

assert(fpin && fpout);

while( (ch = getc(fpin)) != EOF)
putc(ch, fpout);
fclose(fpin);
fclose(fpout);
}

That will tell you how fast the machine can copy 128 MB, using default
buffering as provided by the standard library. Running you own buffering
scheme over the top will tend to slow things down, due to cache effects and
the like.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

Aug 26 '07 #2
On Aug 25, 7:44 pm, "Malcolm McLean" <regniz...@btinternet.comwrote:
"mike3" <mike4...@yahoo.comwrote in message

news:11*********************@l22g2000prc.googlegro ups.com...
Hi.
I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?
/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;
/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}
/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);
}
Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.

There could be several reasons.

Try this

void copy(char *in, char *out)
{
FILE *fpin = fopen(in, "rb);
FILE *fpout = fopen(out, "wb");
int ch;

assert(fpin && fpout);

while( (ch = getc(fpin)) != EOF)
putc(ch, fpout);
fclose(fpin);
fclose(fpout);

}

That will tell you how fast the machine can copy 128 MB, using default
buffering as provided by the standard library. Running you own buffering
scheme over the top will tend to slow things down, due to cache effects and
the like.
It still takes the same ~13 sec. So then why does Windows's
"copy" command go faster?
--
Free games and programming goodies.http://www.personal.leeds.ac.uk/~bgy1mm

Aug 26 '07 #3
mike3 wrote:
>
On Aug 25, 7:44 pm, "Malcolm McLean" <regniz...@btinternet.comwrote:
"mike3" <mike4...@yahoo.comwrote in message

news:11*********************@l22g2000prc.googlegro ups.com...
Hi.
I have an interesting problem.
The C program presented below takes around
12 seconds to copy 128 MB of data on my machine.
Yet I know the machine can go
faster since a copying done at
a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.).
So why is this thing like 3x worse, anyway?
/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;
/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}
/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);
}
Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.
There could be several reasons.

Try this

void copy(char *in, char *out)
{
FILE *fpin = fopen(in, "rb);
FILE *fpout = fopen(out, "wb");
int ch;

assert(fpin && fpout);

while( (ch = getc(fpin)) != EOF)
putc(ch, fpout);
fclose(fpin);
fclose(fpout);

}

That will tell you how fast the machine
can copy 128 MB, using default
buffering as provided by the standard library.
Running you own buffering
scheme over the top will tend to slow things down,
due to cache effects and the like.

It still takes the same ~13 sec. So then why does Windows's
"copy" command go faster?
There's a chance that it might use some efficient opcodes
that don't directly correlate to any C code constructs.

--
pete
Aug 26 '07 #4
mike3 wrote:
On Aug 25, 7:44 pm, "Malcolm McLean" <regniz...@btinternet.comwrote:
>"mike3" <mike4...@yahoo.comwrote in message

news:11*********************@l22g2000prc.googlegr oups.com...
>>Hi.
I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?
[snip]
It still takes the same ~13 sec. So then why does Windows's
"copy" command go faster?
It could be because of any number of reasons.

Perhaps the Windows COPY is not written in C.

Or, perhaps it has been tuned to an optimal I/O pattern for Windows.

Or, perhaps it does not use C stdio, but instead uses Windows-specific I/O
facilities

Or, perhaps it does not actually copy files, but instead manipulates file
directory entries in a Windows-specific way to achieve the same results.

The answer to your question can only be obtained from Microsoft.

--
Lew Pitcher

Master Codewright & JOAT-in-training | Registered Linux User #112576
http://pitcher.digitalfreehold.ca/ | GPG public key available by request
---------- Slackware - Because I know what I'm doing. ------

Aug 26 '07 #5
MQ
On Aug 26, 11:31 am, mike3 <mike4...@yahoo.comwrote:
Hi.

I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?
If you need fast copying of files, and are unhappy with the
performance of the C standard library, you could use file copy
routines of the operating system, which may be more efficient.
However, this will of course make the code non-portable and outside
the scope of comp.lang.c

Aug 26 '07 #6
mike3 wrote:
"Malcolm McLean" <regniz...@btinternet.comwrote:
.... snip ...
>>
void copy(char *in, char *out) {
FILE *fpin = fopen(in, "rb);
FILE *fpout = fopen(out, "wb");
int ch;

assert(fpin && fpout);

while( (ch = getc(fpin)) != EOF)
putc(ch, fpout);
fclose(fpin);
fclose(fpout);
}

That will tell you how fast the machine can copy 128 MB, using
default buffering as provided by the standard library. Running
you own buffering scheme over the top will tend to slow things
down, due to cache effects and the like.

It still takes the same ~13 sec. So then why does Windows's
"copy" command go faster?
Look at the generated code and see if putc and getc are implemented
with a macro. Also, check the results on an immediate second run -
the input file may be cached.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>
--
Posted via a free Usenet account from http://www.teranews.com

Aug 26 '07 #7
On Aug 25, 10:17 pm, MQ <michaelquinli...@gmail.comwrote:
On Aug 26, 11:31 am, mike3 <mike4...@yahoo.comwrote:
Hi.
I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?

If you need fast copying of files, and are unhappy with the
performance of the C standard library, you could use file copy
routines of the operating system, which may be more efficient.
However, this will of course make the code non-portable and outside
the scope of comp.lang.c
But basically with exclusively the C language, and not resorting
to anything outside that this is as fast as it gets, then. Alright.

Aug 26 '07 #8
On Aug 25, 9:31 pm, mike3 <mike4...@yahoo.comwrote:
Hi.

I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?

/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;

/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;

/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}

/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);

}

Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.
The Windows API has "overlapped i/o" calls that admit simultaneous DMA
for disk input and output if your hardware allows it. I'd be
surprised if the system copy did not use this in a very highly tuned
way. A simple copy loop isn't going to compete.
Aug 26 '07 #9
"mike3" <mi******@yahoo.comwrote in message
news:11*********************@l22g2000prc.googlegro ups.com...
I have an interesting problem. The C program presented below
takes around 12 seconds to copy 128 MB of data on my
machine. Yet I know the machine can go faster since a copying
done at a DOS prompt takes only 4 seconds to do the same
128 MB copy (and a copy is both a read and write.). So why is
this thing like 3x worse, anyway?

/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;

/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;

/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}

/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);
}
That's rather ugly and seems suboptimal. I'd write it like this:

/* note: untested, just to show a general idea */
#include <stdio.h>
int copy_file(FILE *src, FILE *dst) {
size_t amt_read, amt_written;
char buf[DISK_BUF_SIZE];

rewind(src);
rewind(dst);
do {
amt_read = fread(buf, 1, sizeof *buf, src);
amt_written = fwrite(buf, 1, amt_read, dst);
if ((amt_read!=amt_written)||ferror(dst))
return ERR_WRITEFAIL;
if (ferror(src))
return ERR_READFAIL;
} while (!feof(src));

return ERR_SUCCESS;
}

Of course, if you don't want to copy the entire thing, you'll need to add a
bit more logic, but then it wouldn't be a fair test against the "COPY"
command.

<OT>
Still, I bet a non-portable version would be faster since most OSes provide
an API call to do it in one operation, like CopyFile() on Windows and
sendfile() frequently found on POSIX systems. Simply eliminating hundreds
of transitions back and forth between kernel and user modes (and copying the
buffer between address spaces) will speed things up quite a bit.
</OT>
Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).
Why open the source file with write permission? You're not writing to it.
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.
What's with "DIGIT"? If you're just copying files, it doesn't matter.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
--
Posted via a free Usenet account from http://www.teranews.com

Aug 26 '07 #10
mike3 wrote:
I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?
Noting that performance isn't something the Standard says
anthing about, and that you're asking an implementation-
dependant question, and I know almost nothing about DOS,
I note:
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.
that you're reading and writing in chunks of `sizeof(unsigned long)`,
which is probably 4 or 8. If you were looking for efficiency in
copying, I would have thought that something a little bigger might
help.

And why `DIGIT`?!

--
Digital Hedgehog
"Never ask that question!" Ambassador Kosh, /Babylon 5/

Aug 26 '07 #11
On 2007-08-26 07:47, Chris Dollin <eh@electrichedgehog.netwrote:
mike3 wrote:
> fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
>Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.

that you're reading and writing in chunks of `sizeof(unsigned long)`,
which is probably 4 or 8.
No, he's reading and writing sizeof(unsigned long) elements of BufferLen
size. I suspect that's not what he wanted, though.

hp
--
_ | Peter J. Holzer | I know I'd be respectful of a pirate
|_|_) | Sysadmin WSR | with an emu on his shoulder.
| | | hj*@hjp.at |
__/ | http://www.hjp.at/ | -- Sam in "Freefall"
Aug 26 '07 #12
On Sun, 26 Aug 2007 01:16:34 -0500, "Stephen Sprunk" wrote:
>I'd write it like this:

/* note: untested, just to show a general idea */
#include <stdio.h>
int copy_file(FILE *src, FILE *dst) {
size_t amt_read, amt_written;
char buf[DISK_BUF_SIZE];
char buf[BUFSIZ];
>
rewind(src);
rewind(dst);
do {
amt_read = fread(buf, 1, sizeof *buf, src);
amt_read = fread(buf, 1, sizeof buf, src); // ;-)
amt_written = fwrite(buf, 1, amt_read, dst);
if ((amt_read!=amt_written)||ferror(dst))
return ERR_WRITEFAIL;
if (ferror(src))
return ERR_READFAIL;
} while (!feof(src));

return ERR_SUCCESS;
}

--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch
Aug 26 '07 #13
mike3 wrote:
On Aug 25, 10:17 pm, MQ <michaelquinli...@gmail.comwrote:
>On Aug 26, 11:31 am, mike3 <mike4...@yahoo.comwrote:
>>Hi.
I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?
If you need fast copying of files, and are unhappy with the
performance of the C standard library, you could use file copy
routines of the operating system, which may be more efficient.
However, this will of course make the code non-portable and outside
the scope of comp.lang.c

But basically with exclusively the C language, and not resorting
to anything outside that this is as fast as it gets, then. Alright.
For what it's worth, the following C program..

/* Simple file copy */
#include <stdio.h>
#include <stdlib.h>

void usage(void) {
puts("Usage: cpy <file1<file2>");
}

int main(int argc, char *argv[]) {
FILE *in, *out;
size_t size;
char *buff;
if (argc != 3) usage();
if ((in = fopen(argv[1], "rb")) == NULL)
printf("Can't open %s\n", argv[1]), exit(1);
if ((out = fopen(argv[2], "wb")) == NULL)
printf("Can't make %s\n", argv[2]), exit(1);
if ((buff = malloc(BUFSIZ)) == NULL)
puts("Can't allocate memory"), exit(1);
while ((size = fread(buff, 1, BUFSIZ, in)))
fwrite(buff, 1, size, out);
fclose(in);
fclose(out);
return 0;
}

...takes 3 to 5 seconds on my kit, exactly the same as COPY.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---
Aug 26 '07 #14
Joe Wright wrote:
mike3 wrote:
>On Aug 25, 10:17 pm, MQ <michaelquinli...@gmail.comwrote:
>>On Aug 26, 11:31 am, mike3 <mike4...@yahoo.comwrote:

Hi.
I have an interesting problem. The C program presented below takes
around
12 seconds to copy 128 MB of data on my machine. Yet I know the
machine can go
faster since a copying done at a DOS prompt takes only 4 seconds to do
the same
128 MB copy (and a copy is both a read and write.). So why is this
thing like 3x
worse, anyway?
If you need fast copying of files, and are unhappy with the
performance of the C standard library, you could use file copy
routines of the operating system, which may be more efficient.
However, this will of course make the code non-portable and outside
the scope of comp.lang.c

But basically with exclusively the C language, and not resorting
to anything outside that this is as fast as it gets, then. Alright.
For what it's worth, the following C program..

/* Simple file copy */
#include <stdio.h>
#include <stdlib.h>

void usage(void) {
puts("Usage: cpy <file1<file2>");
}

int main(int argc, char *argv[]) {
FILE *in, *out;
size_t size;
char *buff;
if (argc != 3) usage();
if ((in = fopen(argv[1], "rb")) == NULL)
printf("Can't open %s\n", argv[1]), exit(1);
if ((out = fopen(argv[2], "wb")) == NULL)
printf("Can't make %s\n", argv[2]), exit(1);
if ((buff = malloc(BUFSIZ)) == NULL)
puts("Can't allocate memory"), exit(1);
while ((size = fread(buff, 1, BUFSIZ, in)))
fwrite(buff, 1, size, out);
fclose(in);
fclose(out);
return 0;
}

..takes 3 to 5 seconds on my kit, exactly the same as COPY.
I see at least one error, I don't exit() from usage(). Sorry.

--
Joe Wright
"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---
Aug 26 '07 #15
On Sun, 26 Aug 2007 16:21:35 -0400, Joe Wright wrote:
>>
I see at least one error, I don't exit() from usage(). Sorry.
You don't free the malloc-ed memory (why malloc at all?). More
important, you don't check for fread, fwrite and fclose errors (but
return 0 (EXIT_SUCCESS)).
>"Everything should be made as simple as possible, but not simpler."
--- Albert Einstein ---
Absolutely!
--
Roland Pibinger
"The fact that the program works has no relevance." - Bartosz Milewski
Aug 26 '07 #16
On Aug 26, 12:16 am, "Stephen Sprunk" <step...@sprunk.orgwrote:
"mike3" <mike4...@yahoo.comwrote in message

news:11*********************@l22g2000prc.googlegro ups.com...


I have an interesting problem. The C program presented below
takes around 12 seconds to copy 128 MB of data on my
machine. Yet I know the machine can go faster since a copying
done at a DOS prompt takes only 4 seconds to do the same
128 MB copy (and a copy is both a read and write.). So why is
this thing like 3x worse, anyway?
/* Copy len digits of b starting at b->StartOffs to
* a starting at a->StartOffs.
*/
int DiskInt_Copy(DiskInt *a, DiskInt *b, long len)
{
long DigitsRemaining, BufferLen, MaxBuffer;
/* Set up lengths */
DigitsRemaining = len;
MaxBuffer = DISK_BUF_SIZE/sizeof(DIGIT);
BufferLen = MaxBuffer;
if(DigitsRemaining < 0) return(ERR_SUCCESS);
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
/* Copy digits */
DiskInt_SeekDigit(a, 0);
DiskInt_SeekDigit(b, 0);
while(DigitsRemaining)
{
fread(diskbuf1, BufferLen, sizeof(DIGIT), b->fd);
fwrite(diskbuf1, BufferLen, sizeof(DIGIT), a->fd);
DigitsRemaining -= BufferLen;
if(DigitsRemaining < MaxBuffer)
BufferLen = DigitsRemaining;
}
/* Done! */
DiskInt_SeekDigit(a, 0);
return(ERR_SUCCESS);
}

That's rather ugly and seems suboptimal. I'd write it like this:

/* note: untested, just to show a general idea */
#include <stdio.h>
int copy_file(FILE *src, FILE *dst) {
size_t amt_read, amt_written;
char buf[DISK_BUF_SIZE];

rewind(src);
rewind(dst);
do {
amt_read = fread(buf, 1, sizeof *buf, src);
amt_written = fwrite(buf, 1, amt_read, dst);
if ((amt_read!=amt_written)||ferror(dst))
return ERR_WRITEFAIL;
if (ferror(src))
return ERR_READFAIL;
} while (!feof(src));

return ERR_SUCCESS;

}

Of course, if you don't want to copy the entire thing, you'll need to add a
bit more logic, but then it wouldn't be a fair test against the "COPY"
command.
Anyway, this doesn't seem to go any faster, and "COPY"
now seems to take 13 seconds, so I guess I was shooting
for "phantom" speed improvements, when really my HD
caching was probably creating the _appearance_ of
added speed.
<OT>
Still, I bet a non-portable version would be faster since most OSes provide
an API call to do it in one operation, like CopyFile() on Windows and
sendfile() frequently found on POSIX systems. Simply eliminating hundreds
of transitions back and forth between kernel and user modes (and copying the
buffer between address spaces) will speed things up quite a bit.
</OT>
Both files, a->fd and b->fd are opened with "wb+" (read/write,
binary).

Why open the source file with write permission? You're not writing to it.
Because in the full program, this thing is supposed
to be used as part of a disk-based multiprecision
integer/fixed point arithmetic package, designed for
calculating pi to tens of millions of base 26 digits
(A...Z, instead of 0...9).

So hence you need to be able to both read and write
to the integers, right?

Here, "DIGIT" is defined as "unsigned long", and "DISK_BUF_SIZE"
equals 1,048,576.

What's with "DIGIT"? If you're just copying files, it doesn't matter.
DIGIT is what the data on disk represents. Namely,
the digits of the multiprecision integer, stored
in base 456,976 (26 to the 4th power -- 4 digits
per DIGIT).
S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking

--
Posted via a free Usenet account fromhttp://www.teranews.com

Aug 26 '07 #17
"Roland Pibinger" <rp*****@yahoo.comwrote in message
news:46**************@news.utanet.at...
On Sun, 26 Aug 2007 01:16:34 -0500, "Stephen Sprunk" wrote:
>>I'd write it like this:

/* note: untested, just to show a general idea */
#include <stdio.h>
int copy_file(FILE *src, FILE *dst) {
size_t amt_read, amt_written;
char buf[DISK_BUF_SIZE];

char buf[BUFSIZ];
*shrug* Call the constant what you want.
> rewind(src);
rewind(dst);
do {
amt_read = fread(buf, 1, sizeof *buf, src);

amt_read = fread(buf, 1, sizeof buf, src); // ;-)
Oops, that typo is important. That's why the disclaimer at the top :)

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking
--
Posted via a free Usenet account from http://www.teranews.com

Aug 27 '07 #18
On Sun, 26 Aug 2007 22:35:30 -0500, "Stephen Sprunk" wrote:
>"Roland Pibinger" <rpbg123@...comwrote:
> char buf[BUFSIZ];

*shrug* Call the constant what you want.
Not what I want but what is defined in stdio.h.
--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch
Aug 27 '07 #19

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Rocky A | last post by:
This is my first posting so please be gentle I've been writing access programs for only about a year and may have bit off more than I can chew but....... I've written an operations program for...
2
by: GSpiggle | last post by:
Have recently upgraded to Windows XP and latest Access 2003 from Windows 98 and Access 97. Distribute a lot of front end applications linked to network based data files. During normal working...
7
by: ddsvi78 | last post by:
I am a complete idiot when it comes to access. Now that said, I work for a computer security company and one of our customers came to us with an access problem. They had been running fine for a...
3
by: Daz | last post by:
Hi everyone! This is my first time posting in this group, although I have been watching it for the past few months and have to say this is possibly the best group I have seen on Google so far!...
1
by: fcharby | last post by:
Hello all, I'm new here and I've searched a lot before posting my question, but I can't seem to find a solution to my problem. I'm working on a database program that I created myself, I created my...
5
by: fcharby | last post by:
Hello all, I'm new here and I've searched a lot before posting my question, but I can't seem to find a solution to my problem. I'm working on a database program that I created myself, I created my...
8
by: =?Utf-8?B?RnJhbms=?= | last post by:
Hello all, I'm new here and I've searched a lot before posting my question, but I can't seem to find a solution to my problem. I'm working on a database program that I created myself, I created...
9
by: tdahsu | last post by:
All, I have the following code: for fileTarget in dircache.listdir("directory"): (dirName, fileName) = os.path.split(fileTarget) f = open(fileTarget).readlines() copying = False for i in...
7
by: colin | last post by:
Hi, Ive written a 3dmodel editor, and it works fairly well it harldy uses any cpu exept when its loading a texture from bitmap in the debugger, at all other times it hardly uses any cpu but from...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.