Determining actual elapsed (wall-clock) time

[Peter Hansen]

I would like to determine the "actual" elapsed time of an operation
which could take place during a time change, in a platform-independent
manner (at least across Linux/Windows machines).

Using time.time() doesn't appear to be suitable, since time might jump
forwards or backwards at the user's whim, if the system clock is reset,
or when a daylight savings time change occurs.

Using time.clock() doesn't appear to be suitable on Linux, though it
appears sufficient on Windows. (I'm willing to assume nobody will reset
whatever QueryPerformanceCounter() uses to return the time.clock() value
on Windows.)

There used to be a "timing" module (according to
http://effbot.org/librarybook/timing.htm) which would likely have worked
(for Unix at least), but it's not in the latest Pythons.

I can deal with wraparounds if necessary. I'm not too concerned about
the resolution, though better than one second would be useful.

Thanks for any suggestions.

-Peter

[Roy Smith]

Peter Hansen <pe***@engcorp.com> wrote:

I would like to determine the "actual" elapsed time of an operation
which could take place during a time change, in a platform-independent
manner (at least across Linux/Windows machines).

Using time.time() doesn't appear to be suitable, since time might jump
forwards or backwards at the user's whim, if the system clock is reset,
or when a daylight savings time change occurs.

If you get the UTC time, daylight savings time doesn't enter the equation.
If the system clock is reset, however, you're out of luck. I can't think
of any time-related API which doesn't rely on the system clock as a
reference. If the system clock is good, you get good time. If the system
clock sucks, or changes, you don't.

If you care about time, you want your system clock controlled by NTP.
There's just no excuse not to.

Is there some reason you can't just use the system clock? I suppose if you
had to, you could hobble together your own NTP client which keeps network
time independent of the system clock. But that would be a lot of work and
it's hard to imagine the effort would be justified.

[Peter Hansen]

Roy Smith wrote:

Peter Hansen <pe***@engcorp.com> wrote:
If you get the UTC time, daylight savings time doesn't enter the equation.
Of course... I didn't think of that approach. I don't actually care
about absolute time, so this should work fine for at least the DST case.
If you care about time, you want your system clock controlled by NTP.
There's just no excuse not to.
I guess as long as the NTP client is set up to ensure the time
adjustments are smaller than some value X, it would be acceptable. I'll
have to look into how to set up Windows XP to prevent users from
changing the time on their own, assuming that's possible.
Is there some reason you can't just use the system clock?

I was anticipating use of the system (running on Windows in this case)
by users who might decide to change the time, and in certain cases that
could cripple the software. I hadn't thought of the possibility of
simply preventing that, though if I insist on an NTP client being
installed, they shouldn't have a problem with that.

Thanks for correcting my thought process...

-Peter

[Roy Smith]

Peter Hansen <pe***@engcorp.com> wrote:

I guess as long as the NTP client is set up to ensure the time
adjustments are smaller than some value X, it would be acceptable.
NTP is generally capable of keeping the various system clocks on a LAN
within a few ms of each other, and within a few 10's of ms over the
Internet from GPS, WWV, or similar international time references.
I'll have to look into how to set up Windows XP to prevent users from
changing the time on their own, assuming that's possible.

On a single-user system like Windows, you pretty much have to assume the
user can do anything. They can turn off NTP, reset the clock, reboot the
system, uninstall your software, whatever.

If you could check to see that NTP is running, it doesn't prove anything.
A malicious and determined user could set up another machine as a NTP
server to synch against, and even configure that machine to look like it
was a stratum-1 reference (i.e. an atomic clock).

At some point, you need to decide if you trust the system administrator to
supply you with an accurate system clock or not. If you don't, and it's
really important that you have an accurate time reference, you've got an
interesting engineering problem on your hands.

[Peter Hansen]

Roy Smith wrote:

Peter Hansen <pe***@engcorp.com> wrote:

I'll have to look into how to set up Windows XP to prevent users from
changing the time on their own, assuming that's possible.

On a single-user system like Windows, you pretty much have to assume the
user can do anything. They can turn off NTP, reset the clock, reboot the
system, uninstall your software, whatever.

Actually, I suspect that a non-administrator user can be restricted from
doing several of those things under XP, which would simplify things.

And, in any case, I'm not remotely concerned about malicious users, just
"fiddlers" who might think their watch time is somehow more correct than
what the computer says...

-Peter

[Peter Hansen]

Roy Smith wrote:

Peter Hansen <pe***@engcorp.com> wrote:

Using time.time() doesn't appear to be suitable, since time might jump
forwards or backwards at the user's whim, if the system clock is reset,
or when a daylight savings time change occurs.

If you get the UTC time, daylight savings time doesn't enter the equation.

Hmmm... not only that, but at least under XP the return value of
time.time() _is_ UTC. At least, it's entirely unaffected by the
daylight savings time change, or (apparently) by changes in time zone.

Looks like time.time() might be sufficient. (That might be what Roy
intended, but I initially thought he meant time.gmtime(time.time()),
which could have resulted in a race condition if time.time() jumped
ahead or back separately from some internal daylight savings time flag,
and the value was read between the two operations.)

-Peter

[Tim Peters]

[Peter Hansen]

Hmmm... not only that, but at least under XP the return value of
time.time() _is_ UTC. At least, it's entirely unaffected by the
daylight savings time change, or (apparently) by changes in time zone.

On all platforms, time.time() returns the number of seconds "since the
epoch". All POSIX systems agree on when "the epoch" began, but that
doesn't really matter to your use case. Number of seconds since the
epoch is insensitive to daylight time, time zone, leap seconds, etc.
Users can nevertheless make it appear to jump (into the future or the
past) by changing their system clock. If you need an absolute measure
of time immune to user whims, you need to connect to special hardware,
or to an external time source.

[John Machin]

Roy Smith wrote:

Peter Hansen <pe***@engcorp.com> wrote:

I guess as long as the NTP client is set up to ensure the time
adjustments are smaller than some value X, it would be acceptable.

NTP is generally capable of keeping the various system clocks on a LAN
within a few ms of each other, and within a few 10's of ms over the
Internet from GPS, WWV, or similar international time references.

I'll have to look into how to set up Windows XP to prevent users from
changing the time on their own, assuming that's possible.

On a single-user system like Windows, you pretty much have to assume the
user can do anything. They can turn off NTP, reset the clock, reboot the
system, uninstall your software, whatever.

If you could check to see that NTP is running, it doesn't prove anything.
A malicious and determined user could set up another machine as a NTP
server to synch against, and even configure that machine to look like it
was a stratum-1 reference (i.e. an atomic clock).

At some point, you need to decide if you trust the system administrator to
supply you with an accurate system clock or not. If you don't, and it's
really important that you have an accurate time reference, you've got an
interesting engineering problem on your hands.

A couple of quick thoughts: (1) stupidity is much more prevalent than
malice in that environment.

(2) Peter, if your app has something else to measure e.g. it is
processing zillions of rows from a database, grab the [UTC] wall time
every N things, and apply plausibility checks to the speed N/delta(wall)
-- if it goes negative or "too high" or "too slow", holler fer a mountie.

[Dennis Lee Bieber]

On Sat, 02 Jul 2005 18:55:46 -0400, Peter Hansen <pe***@engcorp.com>
declaimed the following in comp.lang.python:

And, in any case, I'm not remotely concerned about malicious users, just
"fiddlers" who might think their watch time is somehow more correct than
what the computer says...
In my case, it often is... I've got four WWVB clocks in my
apartment (one wall powered with LASER/LED projector to the ceiling, one
part of a "weather station" that also reads temp/humidity from three
remotes, a battery alarm clock, and a travel clock), along with two GPS
units, and a DVD-R and VCR that time synch to PBS signals... My watch
tends to be less than 20 seconds off when the 6-month DST change takes
place. My computers tend to be a few minutes off in that time span.

-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[flamesrock]

I've needed to do something similar in the past and used free ntp
servers. Of course you need an internet connection:
http://ntp.isc.org/bin/view/Servers/NTPPoolServers
http://www.nightsong.com/phr/python/setclock.py

You could also have a startup script spawn a thread that stores the
time persistently in a shelf object, sleeps for a minute, and checks to
see if the current clock time is greater than a minute (+/-sec,) and
subtracts the difference. Or even check it against an NTP server if you
can.

[Bengt Richter]

On Sat, 2 Jul 2005 19:44:19 -0400, Tim Peters <ti********@gmail.com> wrote:

[Peter Hansen]

Hmmm... not only that, but at least under XP the return value of
time.time() _is_ UTC. At least, it's entirely unaffected by the
daylight savings time change, or (apparently) by changes in time zone.

On all platforms, time.time() returns the number of seconds "since the
epoch". All POSIX systems agree on when "the epoch" began, but that
doesn't really matter to your use case. Number of seconds since the
epoch is insensitive to daylight time, time zone, leap seconds, etc.=20
Users can nevertheless make it appear to jump (into the future or the
past) by changing their system clock. If you need an absolute measure
of time immune to user whims, you need to connect to special hardware,
or to an external time source.

For the latter, Peter, you can probably adapt Paul Rubin' setclock.py
found at
http://www.nightsong.com/phr/python/setclock.py

Regards,
Bengt Richter

[Aahz]

In article <3Z********************@powergate.ca>,
Peter Hansen <pe***@engcorp.com> wrote:

I would like to determine the "actual" elapsed time of an operation
which could take place during a time change, in a platform-independent
manner (at least across Linux/Windows machines).

Using time.time() doesn't appear to be suitable, since time might jump
forwards or backwards at the user's whim, if the system clock is reset,
or when a daylight savings time change occurs.

You have to do one of two things: rely on system clock or require a
network connection (so you can call a timeserver directly).
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

f u cn rd ths, u cn gt a gd jb n nx prgrmmng.

[zooko]

The traditional use of gettimeofday() to (insecurely and unreliably)
approximate elapsed local time is one of my pet peeves.

Fortunately a real monotonic clock has finally been added to the linux
kernel and glibc:

http://www.imperialviolet.org/page24.html#e474

If you have a recent enough kernel and glibc you can use that. On
Windows you can use system timers and handle wraparound yourself.

And you can write an abstract Pythonic layer over both and give it to
me. ;-)

[Peter Hansen]

zooko wrote:

The traditional use of gettimeofday() to (insecurely and unreliably)
approximate elapsed local time is one of my pet peeves.

Fortunately a real monotonic clock has finally been added to the linux
kernel and glibc:

http://www.imperialviolet.org/page24.html#e474

Interestingly, the author of that page appears to have made a number of
the same misassumptions about the actual behaviour of the timeofday
clock (assuming time.time() fairly faithfully reproduces/wraps that
behaviour). Either that or time.time() does magic that avoids the bulk
of the problems in which case I can say only "sucks not to be using
Python, doesn't it?"

I'd be more interested in this monotonic clock feature if it were not
the case that time.time() is pretty much monotonic except in the face of
the minor (sub-100ms) tweaks that might occur once every week or two
with an NTP client running. It certainly doesn't cause the one hour
jumps forwards and backwards which I and the author of that page both
thought it would.

-Peter