IOERROR Broken Pipe

[Ruben]

I am reading a large text file a chunk at a time using the
readlines(buffer_size) statement. I get an IOERROR ERRNO 32 Broken
Pipe command when I "pipe" the output to MYSQL database using the
following command:

python script.py | mysql

I am using RED HAT 9 Linux. The problem only happens if I use the pipe
to MYSQL. It breaks but it gets to process around 1000 MYSQL
statements.

If I only use the following statement:

python script.py

The programs runs ok and reads all text records without problems. I
do need to sent the pipe to MYSQL so that I can populate a database.

I tried using the following statement but it still breaks:

import signal

signal.signal(signal.SIGPIPE, signal.SIG_DFL)

Thanks,

Ruben

[Steve Holden]

Ruben wrote:

I am reading a large text file a chunk at a time using the
readlines(buffer_size) statement. I get an IOERROR ERRNO 32 Broken
Pipe command when I "pipe" the output to MYSQL database using the
following command:

python script.py | mysql

I am using RED HAT 9 Linux. The problem only happens if I use the pipe
to MYSQL. It breaks but it gets to process around 1000 MYSQL
statements.

If I only use the following statement:

python script.py

The programs runs ok and reads all text records without problems. I
do need to sent the pipe to MYSQL so that I can populate a database.

I tried using the following statement but it still breaks:

import signal

signal.signal(signal.SIGPIPE, signal.SIG_DFL)

Broken pipe errors normally occur when the consumer (in this case the
mysql client program) stops reading before the producer (your program)
stops writing.

I would therefore suspect that there may be a syntax error or similar in
the SQL your program generates.

Test this by running

python script.py > tmpfile
mysql < tmpfile

regards
Steve