473,320 Members | 2,094 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Re: better scheduler with correct sleep times

sokol wrote:
What was a surprise to me was that python sched.py makes the same
mistake as I did in my first version.
The sched module is *not* designed for multithreading. It
assumes that the thread waiting for events is the only one
putting events into the queue, so it's impossible for an
event to get scheduled while in the midst of a sleep. It
also doesn't take any measures to protect its data structures
from concurrent access.

The documentation could make this clearer, especially since
it confusingly talks about "allowing other threads to run".

--
Greg
Oct 21 '08 #1
5 1731
On Oct 21, 2:19*am, greg <g...@cosc.canterbury.ac.nzwrote:
sokol wrote:
What was a surprise to me was that python sched.py makes the same
mistake as I did in my first version.

The sched module is *not* designed for multithreading. It
assumes that the thread waiting for events is the only one
putting events into the queue, so it's impossible for an
event to get scheduled while in the midst of a sleep. It
also doesn't take any measures to protect its data structures
from concurrent access.

The documentation could make this clearer, especially since
it confusingly talks about "allowing other threads to run".
I find that hard to believe. Scheduler in single threaded
application is useless (well, you can correct me because
right now I can't come up with an example). Also, the
scheduler runs inside a loop. How do you suppose to
run other code while the loop is executing? Remember, all
you have is a single thread. The consequence of this is
that the only way to insert something new inside a queue
is by doing it from scheduled function. Furthermore,
if scheduler is single threaded, why does is
check if the top event has changed after sleep period?

What I can agree is that python sched as it is (not
designed for multithreading) is quite useless.

--
Tvrtko
Oct 21 '08 #2
On Oct 21, 4:58*am, sokol <tvrtko.sokolov...@gmail.comwrote:
On Oct 21, 2:19*am, greg <g...@cosc.canterbury.ac.nzwrote:
sokol wrote:
What was a surprise to me was that python sched.py makes the same
mistake as I did in my first version.
The sched module is *not* designed for multithreading. It
assumes that the thread waiting for events is the only one
putting events into the queue, so it's impossible for an
event to get scheduled while in the midst of a sleep. It
also doesn't take any measures to protect its data structures
from concurrent access.
The documentation could make this clearer, especially since
it confusingly talks about "allowing other threads to run".

I find that hard to believe. Scheduler in single threaded
application is useless (well, you can correct me because
right now I can't come up with an example). Also, the
scheduler runs inside a loop. How do you suppose to
run other code while the loop is executing? Remember, all
you have is a single thread. The consequence of this is
that the only way to insert something new inside a queue
is by doing it from scheduled function.
Yeah, that does happen.

I often used this sort of thing (pre-Python days, so a long time ago)
in single-threaded, real-time simulations.

Don't know mcuh about the sched module, but time scheduling in general
doesn't need multithreading, not one bit.
Carl Banks

Oct 21 '08 #3
sokol wrote:
Also, the
scheduler runs inside a loop. How do you suppose to
run other code while the loop is executing?
The sleep function could be doing a select with a
timeout on some other source of events, such as a
socket or a gui input event stream. One possible
response to such an event is to schedule another
event. There's not so much need for that nowadays,
since most gui libraries provide a way of scheduling
timed events as part of their built-in event loop,
but you might want to use something like this in
a server that deals with network connections.

Another possible use is discrete-event simulation,
where the "sleep" function doesn't physically sleep
but just advances a simulated time, and all events
(other than the first one that starts everything off)
are scheduled by callbacks for other events.

So while its uses are rather specialized, I wouldn't
say it's useless. The main problem is that its nature
needs to be much more clearly spelled out in the
docs -- it's something of an attractive nuisance the
way it is.

--
Greg
Oct 22 '08 #4
On 2008-10-21, sokol <tv***************@gmail.comwrote:
On Oct 21, 2:19*am, greg <g...@cosc.canterbury.ac.nzwrote:
>sokol wrote:
What was a surprise to me was that python sched.py makes the same
mistake as I did in my first version.

The sched module is *not* designed for multithreading. It
assumes that the thread waiting for events is the only one
putting events into the queue, so it's impossible for an
event to get scheduled while in the midst of a sleep. It
also doesn't take any measures to protect its data structures
from concurrent access.

The documentation could make this clearer, especially since
it confusingly talks about "allowing other threads to run".

I find that hard to believe. Scheduler in single threaded
application is useless (well, you can correct me because
right now I can't come up with an example).
Imagine your environment doesn't provide any kind of multithreading
support. Couldn't you write an interactive game, a FTP server, a
messaging system? Programmers have done that for years. The fact that
you *can* write such things using multiple threads doesn't mean that't
the only way to do that, nor the best one. It's like `fork`, the Unix
programmer's hammer: every problem becomes a nail.
Also, the
scheduler runs inside a loop. How do you suppose to
run other code while the loop is executing? Remember, all
you have is a single thread. The consequence of this is
that the only way to insert something new inside a queue
is by doing it from scheduled function. Furthermore,
if scheduler is single threaded, why does is
check if the top event has changed after sleep period?

What I can agree is that python sched as it is (not
designed for multithreading) is quite useless.
The sched module (and mutex too, BTW) exists right from the beginning
of Python, ages before multithreading support were added to the
language. The algorithm hasn't changed in years; I wouldn't say it's
useless, it's just not suitable for the kind of scheduling one usually
wants to do in a multithreaded environment.
The latest release (2.6) contains this warning::

In multi-threaded environments, the scheduler class has
limitations with respect to thread-safety, inability to insert a new
task before the one currently pending in a running scheduler, and
holding up the main thread until the event queue is empty. Instead, the
preferred approach is to use the threading.Timer class instead.

--
Gabriel Genellina

Oct 22 '08 #5
On Oct 22, 2:28*am, greg <g...@cosc.canterbury.ac.nzwrote:
sokol wrote:
Also, the
scheduler runs inside a loop. How do you suppose to
run other code while the loop is executing?

The sleep function could be doing a select with a
timeout on some other source of events, such as a
socket or a gui input event stream. One possible
response to such an event is to schedule another
event. There's not so much need for that nowadays,
since most gui libraries provide a way of scheduling
timed events as part of their built-in event loop,
but you might want to use something like this in
a server that deals with network connections.

Another possible use is discrete-event simulation,
where the "sleep" function doesn't physically sleep
but just advances a simulated time, and all events
(other than the first one that starts everything off)
are scheduled by callbacks for other events.

So while its uses are rather specialized, I wouldn't
say it's useless. The main problem is that its nature
needs to be much more clearly spelled out in the
docs -- it's something of an attractive nuisance the
way it is.

--
Greg
I see. The delayfunc is user defined function so it
doesn't have to sleep at all. If you are creative
enough, you can use this scheduler in many ways.

--
Tvrtko
Oct 23 '08 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: project2501 | last post by:
i'm doing some benchmarking and python is certainly fast enough (the time.time resolution is more than good enough). however, i am using the threading module to implement worker trheads to hit a...
12
by: Conrad | last post by:
Greetings, Q: Is there some way to gracefully suspend a python app for a second or so then resume? I could write the classic basic dumb loop-tenzillion-times delay, but that seems inelegant, and...
11
by: Codemonkey | last post by:
Hi, I am writing an App in .Net that involves some scheduling of tasks. I was wondering if anybody has come accross any components or examples of how to implement a schedule manager like the one...
21
by: Alo Sarv | last post by:
Hi From what I have understood from various posts in this newsgroup, writing event loops pretty much comes down to this: while (true) { handleEvents(); sleep(1); // or _sleep() or...
1
by: kmounkhaty | last post by:
Hi Guru, Our HR sql 2000 production server is 800.760 built (SP3a) with 2GB of RAM and 2 CPU hiper threading. This server is very low activity where processor runs average 1-3 during the day....
1
by: Nagendra Kumar | last post by:
Hello ALL, I am trying to schdule some of my class methods using sched module of python import sched, time s=sched.scheduler(time.time, time.sleep) event1=s.enter(60, 1,...
1
TRScheel
by: TRScheel | last post by:
Here's a good one. I have spent all day researching this, and I can not for the life of me find a method that will work. All I want to do... is add a scheduled task to the Task Scheduler. I...
1
by: Prof. William Battersea | last post by:
I'd like a class method to fire every n seconds. I tried this: class Timed: def.__init__(self): self.t = Timer(3, self.dothing) def.start(self): self.t.start()
3
by: Scott David Daniels | last post by:
qvx wrote: The trick is to use Queue's timeout argument to interrupt your sleep when new requests come in. def time_server(commands): '''Process all scheduled operations that arrive on...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.