473,226 Members | 1,319 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,226 software developers and data experts.

timeout with read operations

Jim
Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.
Jul 18 '05 #1
5 16428
P
Jim wrote:
Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?


I don't think anything could happen
that would cause a block. That's the
assumption I made when implementing:
http://www.pixelbeat.org/libs/subProcess.py

If you find this is not the case you can
always make the fileno non blocking but
I really don't think this is required.

Pádraig.
Jul 18 '05 #2
ji******@hotmail.com (Jim) wrote in message news:<c5**************************@posting.google. com>...
Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
What's the application? It's easier to make rational suggestions if
we know what you're trying to do.

Disk reads are uninterruptable--things like setting O_NONBLOCK don't
have any effect on files AFAICS (they work fine on sockets/pipes).

In general (for non-disk system calls), investigate alarm() and
setitimer(). Python wraps the former in signal.alarm (check the docu
for the signal module), there is no standard Python wrapper for the
latter that I know of.

Possible solutions for disk reads include:
1. Use async I/O (I don't know of a Python API for this)
2. Use some sort of parallel processing, either threads or processes;
I'd prefer processes for this particular application (unless you're on
a platform like Windows where there is a significant performance
penalty to processes).
3. Read files into memory before entering the time-sensitive portion
of the code

If 3 is infeasible, consider 2; set up a pipe with os.pipe, fork with
os.fork, and have the child process read off disk and write to the
parent. The parent can then select on the pipe and read--the pipe
doesn't select as readable until data is already in the pipe buffer,
so you're guaranteed not to block on THAT (you may get an error
returned immediately). If you're lazy, popen("/bin/cat filename")
will work, though you may more control and efficiency if you do the
fork, pipe, and read calls on your own (popen invokes an unnecessary
shell)

Threads won't be any easier to integrate with a select loop, but if
there's a major performance issue you can avoid a memory copy by
having a thread do the read and using a pipe, socket, or signal to
trigger the select loop. Almost certainly not worth the
implementation pain.
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.


AFAIK, SO_RCVTIMEO and SO_SNDTIMEO are not writeable under Linux, so
I'd guess that settimeout plays games with alarm() or setitimer().

Sumner
Jul 18 '05 #3
P@draigBrady.com wrote in message news:<40**************@draigBrady.com>...
Jim wrote:
but .. what if between the select and read call something happens and
the read blocks indefinitly?


I don't think anything could happen
that would cause a block.


Another process could start up and hog the disk.

A cable could be loose.

Actually, I think select() on a regular file always returns readable
(except if you're at EOF maybe?)

In practice, it's unlikely that you'll block for very long, which is
sufficient for many applications.
If you find this is not the case you can
always make the fileno non blocking


O_NONBLOCK has no effect on regular files--see my other reply in this
thread.

Sumner
Jul 18 '05 #4
ji******@hotmail.com (Jim) wrote in message news:<c5**************************@posting.google. com>...
Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.

Could you not use signals to do what you need?

Someone just pointed me in that direction, and it worked great for
me. About 5 lines of code.

http://www.python.org/doc/current/li...le-signal.html

Check the example out.

I hope I have helped! I am usually the n00b asking the questions
around here.

googleboy
Jul 18 '05 #5
ji******@hotmail.com (Jim) wrote in message news:<c5**************************@posting.google. com>...
Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.

Could you not use signals to do what you need?

Someone just pointed me in that direction, and it worked great for
me. About 5 lines of code.

http://www.python.org/doc/current/li...le-signal.html

Check the example out.

I hope I have helped! I am usually the n00b asking the questions
around here.

googleboy
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: Yang Li Ke | last post by:
Hi guys! I have some datas that I must check everytime a visitor comes to my site What is better to do: 1- Read data from a file or 2- Read data from a mysql db Thank you
3
by: Bob Bedford | last post by:
Some of my code has a long process, and my ISP limits the scripts to 30 seconds. Now, I must absolutely create a file with some string (simply a number and an email), then process any line with...
1
by: bdinmstig | last post by:
I refined my attempt a little further, and the following code does seem to work, however it has 2 major problems: 1. Very limited support for XPath features Basic paths are supported for...
5
by: jdh2358 | last post by:
I have a python file that is trying to read raw data from a raw partition on a dying dist, eg f = file('/dev/sda') f.seek(SOMEWHERE) s = f.read(SOMEBYTES) On some blocks, the read succeeds,...
2
by: Madhusudan Singh | last post by:
python-gpib provides Gpib.py (see end of post) for Linux. I am trying to use the method called read. I usually use it without arguments (the default length being 512). However, I am trying to...
1
by: Hewit | last post by:
DataTable Class documentation says. Thread Safety "This type is safe for multithreaded read operations. You must synchronize any write operations." In my web based application (ASP.Net)...
2
by: Lisa Pearlson | last post by:
Hi, I am wanting to write an internet 'server' program. For ease, I want to use PHP-CLI with XINET super server. XINET communicates with the 'server' program via STDIN/STDOUT. I'm not sure...
3
by: carmelo8 | last post by:
Hi there, I want to timeout Read method of System.Stream class after 30 secs. of no activity given a certain opened connection. That is, because some incoming connections to my application just...
4
by: zl2k | last post by:
hi, there I have a appendable binary file of complex data structure named data.bin created by myself. It is written in the following format: number of Data, Data array Suppose I have...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.