Hi there,
I got a program to write data to a randomly accessed file (the program
moves file pointer to a certain position of the file according the
current "keyword" and then writes data). It compiles and runs in Win2K
and various Unix and Linux systems. The main program is in Fortran,
calling a data writing library writen in C++.
The weird thing is, it takes much longer elapse time (wall-clock time)
than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in
all other systems, these two kinds of time are almost the same (e.g.
2min). The extra waiting in IBM machine does happen during file
pointer locating and data writing stage. In all cases, this program is
the only runing application during testing.
Do you have any idea what may cause the long waiting time in IBM
machine? It created a big performance headache for me now.
Thanks for any suggestion.
Regards,
Lu 8 3104
Lu wrote: Hi there,
I got a program to write data to a randomly accessed file (the program moves file pointer to a certain position of the file according the current "keyword" and then writes data). It compiles and runs in Win2K and various Unix and Linux systems. The main program is in Fortran, calling a data writing library writen in C++.
The weird thing is, it takes much longer elapse time (wall-clock time) than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in all other systems, these two kinds of time are almost the same (e.g. 2min). The extra waiting in IBM machine does happen during file pointer locating and data writing stage. In all cases, this program is the only runing application during testing.
Do you have any idea what may cause the long waiting time in IBM machine? It created a big performance headache for me now.
Thanks for any suggestion.
Regards, Lu
sounds like seismic :-)
BTW, I think this question is better for comp.unix.programmer, this is a
newsgroup for std C++ questions, not OS systems calls.
~S
In comp.lang.fortran Lu <ql*@lgc.com> wrote: The weird thing is, it takes much longer elapse time (wall-clock time) than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in all other systems, these two kinds of time are almost the same (e.g. 2min). The extra waiting in IBM machine does happen during file pointer locating and data writing stage. In all cases, this program is the only runing application during testing.
Do you have any idea what may cause the long waiting time in IBM machine? It created a big performance headache for me now.
I/O should not generally use a lot of CPU time, so there is nothing wrong
with wall time being much longer than CPU time. It's just slow I/O. Are
you doing the I/O over NFS? How old and slow are the hard disks? It
could be an AIX problem, but I don't use it so can't help there - that's
for the AIX folks to help with.
Regards,
Daniel.
------------------------------------------------------------------------
Dr. Daniel Grimwood Department of Chemistry
Email : re****@theochem.uwa.edu.au The University of Western Australia
Phone : +61 8 93808563 35 Stirling Highway
Fax : +61 8 93801005 Crawley WA 6009
Lu,
you statement sounds as if AIX is much slower than any other machine. Is
that true or do you mean it completed in the same time but with less CPU?
Regards Rolf ql*@lgc.com (Lu) wrote in message news:<89**************************@posting.google. com>...
Thanks for all your response.
All tests wrote the data to local hard disks to avoid network traffic.
I tested in AIX, SGI, Linux and W2K, the CPU times are roughly the
same (2- 4 mins), only AIX got a long extra waiting time. The IBM-AIX
machines were bought about two years ago, so the hard disk should be
good enough.
If writing to a sequentialy accessed file, the extra waiting time is
gone.
I also tried to run the executable in a SGI machine and write
(randomly accessed) data file to a remote IBM hard disk (through NFS),
or run the executable in a IBM machine and write data file to a remote
SGI hard disk, the long waiting time also occured. But if I run the
executable in SGI machine and wrote to a remote SGI hard disk (through
NFS), there is no extra waiting time.
Thanks,
Lu
> Thanks for all your response. All tests wrote the data to local hard disks to avoid network traffic. I tested in AIX, SGI, Linux and W2K, the CPU times are roughly the same (2- 4 mins), only AIX got a long extra waiting time. The IBM-AIX machines were bought about two years ago, so the hard disk should be good enough.
If writing to a sequentialy accessed file, the extra waiting time is gone.
I also tried to run the executable in a SGI machine and write (randomly accessed) data file to a remote IBM hard disk (through NFS), or run the executable in a IBM machine and write data file to a remote SGI hard disk, the long waiting time also occured. But if I run the executable in SGI machine and wrote to a remote SGI hard disk (through NFS), there is no extra waiting time.
Thanks, Lu
1. This is not a DB2 problem.
2. This is not a problem with an IBM machine running AIX that is correctly
configured. Otherwise, IBM would not be able to sell any RS/6000 machines if
it were an inherent problem with the platform.
"Lu" <ql*@lgc.com> wrote in message
news:89**************************@posting.google.c om... ql*@lgc.com (Lu) wrote in message
news:<89**************************@posting.google. com>... Thanks for all your response.
All tests wrote the data to local hard disks to avoid network traffic. I tested in AIX, SGI, Linux and W2K, the CPU times are roughly the same (2- 4 mins), only AIX got a long extra waiting time. The IBM-AIX machines were bought about two years ago, so the hard disk should be good enough.
If writing to a sequentialy accessed file, the extra waiting time is gone.
It might be that, in the case of direct access (random access) files each
write is flushed (fflush in C) to the disk, where normal sequential writes
go through the library and OS buffers and disk cache. That might be true
even if the random access file is sequentially accessed.
-- glen
Hi
Did you check the size of the file that you were writing to.
It would not be the first time that someone ( myself included) tried to fill
a 1,3 file with squigabytes of data. Pick is so brilliant it will cope
unfortunately it will all be in overflow.
Peter McMurray
See in comp.unix.aix about vmtune and numclust, maxrandwrt
vmtune is part of bos.adt.samples
The current default settings in AIX 4.3.3 ML11 is -c 0 -W 0
Example of test program with seeks and writes to timex and see how
changing the parms affects the elapsed time.
#include <stdio.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/ioctl.h>
#define NWRITES 3000
extern int errno;
main(argc,argv)
int argc;
char *argv[];
{ int fd1,j,i=0;
int arg1;
char ar_1[1024000];
if(argc==1)
fd1=open("tmp0",O_WRONLY|O_TRUNC|O_CREAT);
if(fd1==-1) {printf("\nError %d on open %s \n",errno,ar_1);
exit(1);
}
for(i=0;i<1024;i++) ar_1[i]='1';
i=0;
for(i=0;i<NWRITES;i++)
{lseek(fd1,i*1024,SEEK_SET);
j=write(fd1,ar_1,1024);
if(j==-1) {printf("\n Error %d on write ",errno);break;
}
lseek(fd1,NWRITES*1024+i*1024,SEEK_SET);
j=write(fd1,ar_1,1024);
if(j==-1) {printf("\n Error %d on write ",errno);break;
}
}
printf("\nTotal writes is: %d\n",i);
}
Bernard Dhooghe ql*@lgc.com (Lu) wrote in message news:<89**************************@posting.google. com>... Hi there,
I got a program to write data to a randomly accessed file (the program moves file pointer to a certain position of the file according the current "keyword" and then writes data). It compiles and runs in Win2K and various Unix and Linux systems. The main program is in Fortran, calling a data writing library writen in C++.
The weird thing is, it takes much longer elapse time (wall-clock time) than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in all other systems, these two kinds of time are almost the same (e.g. 2min). The extra waiting in IBM machine does happen during file pointer locating and data writing stage. In all cases, this program is the only runing application during testing.
Do you have any idea what may cause the long waiting time in IBM machine? It created a big performance headache for me now.
Thanks for any suggestion.
Regards, Lu This discussion thread is closed Replies have been disabled for this discussion. Similar topics
6 posts
views
Thread by Sebastian Kemi |
last post: by
|
3 posts
views
Thread by ishekar |
last post: by
|
8 posts
views
Thread by Lu |
last post: by
|
19 posts
views
Thread by wetherbean |
last post: by
|
3 posts
views
Thread by Mike Kelly |
last post: by
|
10 posts
views
Thread by m00nm0nkey |
last post: by
|
1 post
views
Thread by laredotornado |
last post: by
|
4 posts
views
Thread by Dameon |
last post: by
|
12 posts
views
Thread by glennanthonyb |
last post: by
| | | | | | | | | | |