Making sure script only runs once instance at a time.

[Hari Sekhon]

I have written a script and I would like to ensure that the script is
never run more than once at any given time.

What is the best way of testing and exiting if there is another version
of this script running somewhere on this machine?

I guess what I'm asking is how to handle system processes the way I can
in shell. I am running this on linux.
Thanks

-h

--
Hari Sekhon

[Lawrence D'Oliveiro]

In message <ma**************************************@python.o rg>, Hari
Sekhon wrote:

I have written a script and I would like to ensure that the script is
never run more than once at any given time.

<http://groups.google.co.nz/gr***********************************@lust.ihug.co .nz>

[Stephan Kuhagen]

Hari Sekhon wrote:

I have written a script and I would like to ensure that the script is
never run more than once at any given time.

What is the best way of testing and exiting if there is another version
of this script running somewhere on this machine?

I guess what I'm asking is how to handle system processes the way I can
in shell. I am running this on linux.

Although the other solution suggested seems to be more general (I have not
read the code, only the info in the header), the following may be
sufficient for what you want, and can be used as code inside of your
script. It searches /proc/ for another process that is a script interpreted
by python and has the same name as your script. So it can fail, if there is
another python script running with the same name, but which is not
identical to your script. OTOH it's a very short and simple test:

-----
#!/bin/env python

import os.path
import linecache

pid=os.getpid()
script=linecache.getline(os.path.join('/proc', str(pid), 'cmdline'),
1).split('\0')[0:2]
for pid in filter(lambda x: x.isdigit() and x != str(pid),
os.listdir('/proc')):
other=linecache.getline(os.path.join('/proc', str(pid), 'cmdline'),
1).split('\0')[0:2]
if script[0] == other[0] and os.path.basename(script[-1]) ==
os.path.basename(other[-1]):
raise "already running!"
-----

HTH
Stephan

[MaR]

A very brutal but simple and effective method is to bind() to a socket
on localhost eg (127.0.0.1, 4711), listen() but never accept().
Any other process trying to to bind() on the same port will fail..
When the process dies, the port is released automatically, pending som
timedelay..

But this assumes you have an execution environment where this is
acceptable. A sysadmin may have objections ;o)

[Hari Sekhon]

Seeing as there doesn't seem to be a good answer to this (or at least
not one that we have so far some up with) I have decided to fall back to
my old friend the unix shell. It's as portable as python, but is very
flexible and fast at doing real things and will tell me if another
process by this name is running. If so, print msg and exit. simple.

-h

Hari Sekhon

MaR wrote:

A very brutal but simple and effective method is to bind() to a socket
on localhost eg (127.0.0.1, 4711), listen() but never accept().
Any other process trying to to bind() on the same port will fail..
When the process dies, the port is released automatically, pending som
timedelay..

But this assumes you have an execution environment where this is
acceptable. A sysadmin may have objections ;o)

[Paul Rubin]

Hari Sekhon <hp******@googlemail.comwrites:

Seeing as there doesn't seem to be a good answer to this (or at least
not one that we have so far some up with) I have decided to fall back
to my old friend the unix shell. It's as portable as python, but is
very flexible and fast at doing real things and will tell me if
another process by this name is running. If so, print msg and
exit. simple.

Huh? The obvious way to check for another instance is with a lock
file. Just open the file and use fcntl to set an exclusive lock. If
the lock acquisition fails, another instance has the file.

[Fredrik Lundh]

Hari Sekhon wrote:

I'm not sure if that is a very old way of doing it, which is why I was
reluctant to do it. My way actually uses the process list of the os
(linux) and counts the number of instances. If it is more than 0 then
another process is running and the script exits gracefully.

the code that reliably identifies instances of a given program would be
interesting to see.

Also, apart from the fact the using lockfiles feels a bit 1970s, I have
found that in real usage of other programs within the company that use
lockfiles, it sometimes causes a bit of troubleshooting time when
it stops working due to a stale lockfile.

to minimize that risk, store the pid in the lockfile (and preferrably
also the host name), and make sure that the program checks that the pid
is still active before it "stops working".

</F>

[Fredrik Lundh]

Hari Sekhon wrote:

How exactly do you check that the pid is still active in python? Is
there a library or something that will allow me to manipulate system
processes and listings etc the way everybody does in unix shells....

by passing zero to the os.kill primitive:

os.kill(pid, 0)

if this raises an OSError, there's no active process with the given pid.

I'm a huge fan of shell so I've done my own thing which leans on shell
as follows:

import sys,commands,os

scriptpath = sys.argv[0]
scriptname = os.path.basename(scriptpath)

number_procs=commands.getstatusoutput('ps -ef|grep %s|grep -v grep|wc
-l' % scriptpath)

if number_procs 1:
print "There appears to be another %s process running." % scriptname

what if you have commands with overlapping names (e.g. "bar.py" and
"foobar.py"), or some other user on the machine happens to run a
command that, on purpose or by accident, contains your script's name
(e.g. "emacs mybar.py") ?

</F>

[MonkeeSage]

Here's a class using Fredrik's suggestions to provide generic,
cross-platform file locking (only tested on *nix however, with the two
test files listed [i.e., run test1.py in one terminal then test2.py in
another]):

http://pastie.caboo.se/15851

Ps. The lockfile should also always be cleaned up (even when your
program excepts) because of the magic __del__ method, but I don't
guarentee it will be (nor depend on it: I used the pid + hostname in
the lockfile approach).

Regards,
Jordan

[Eric S. Johansson]

MonkeeSage wrote:

Here's a class using Fredrik's suggestions to provide generic,
cross-platform file locking (only tested on *nix however, with the two
test files listed [i.e., run test1.py in one terminal then test2.py in
another]):

http://pastie.caboo.se/15851

Ps. The lockfile should also always be cleaned up (even when your
program excepts) because of the magic __del__ method, but I don't
guarentee it will be (nor depend on it: I used the pid + hostname in
the lockfile approach).

the problem with this solution is that it does not handle the read
nonexclusive/write exclusive locking model. In this model, reads don't
block, they only register that the request is in process. writes lock
request block until all outstanding reads have completed. When there is
a write lock waiting, all subsequent reads lock requests block until the
write has completed.

Things get more complicated when you add exclusive read capability to
the system. But fortunately, that capability isn't needed very often
and sometimes reduces to a write exclusive lock.

The next level up in terms of locking system requirements is some form
of scoreboard mechanism where you can lock individual records in a file
(i.e. dbm). I do something like this in a demon which owns the dbm,
mediates database access, and record locking so that I can have multiple
readers and writers at the same time using gdbm. But you can also use
the same technique for metakit.
---eric

[MonkeeSage]

Eric S. Johansson wrote:

the problem with this solution is that it does not handle the read
nonexclusive/write exclusive locking model. In this model, reads don't
block, they only register that the request is in process. writes lock
request block until all outstanding reads have completed. When there is
a write lock waiting, all subsequent reads lock requests block until the
write has completed.

I should have said "lock file" -- I'm not trying to do file locking for
concurrent access to a file; I'm trying to provide a lock file to
determine if an instance of a process is already spawned.

Regards,
Jordan

[Eric S. Johansson]

MonkeeSage wrote:

Eric S. Johansson wrote:

>the problem with this solution is that it does not handle the read
nonexclusive/write exclusive locking model. In this model, reads don't
block, they only register that the request is in process. writes lock
request block until all outstanding reads have completed. When there is
a write lock waiting, all subsequent reads lock requests block until the
write has completed.

I should have said "lock file" -- I'm not trying to do file locking for
concurrent access to a file; I'm trying to provide a lock file to
determine if an instance of a process is already spawned.

That difference was clear to me and I appreciate your comment clarifying
the issue further.

I was trying to point out how there are more models of locking that we
could use in Python than the single point lock as you had described.
Unfortunately, most system locks are hard to get right and difficult to
test so you're never quite sure until something breaks and then you know
you did it wrong.

I really should grab the code you pasted and use it as a rewrite for
portalocker. Someday. <sigh>

--- eric