469,589 Members | 2,177 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,589 developers. It's quick & easy.

Windows service and pyc files


Hello,

I have a win32 service written in Python that starts a plain
application, written in Python.

The win32 service tries to launch the application in a while loop and
logs the return value of the os.system call. That's all.

The application is a simple Python program that connects to an https
xml/rpc server, and works with the data retrieved from that server. It
is written as an application because it must be able to download updates
for itself. Here is how it works:

a.) connect to server
b.) compare current version with the latest
c.) if there is a difference, then download all sources from the server,
delete all pyc files and exit; otherwise start processing

I could not write a self-restarting server, and I did not want to
write/install two services for one problem. The win32 service is very
small and primitive so probably I will not need to update it. I think
the basic idea is good, but...

When there is a client update available, my application updates itself
cleanly and exists. Then the service tries to restart the application,
but it cannot. os.system returns with OS error code -1. The pyc files
are NOT generated for the application. However, if I start the
application from the command line, then pyc files are created, and then
the service will also start the application immediatelly. The win32
service is running as "Localsystem" so it is sure that it has write
permission on all files.

I cannot log out the error from the application since it is not started.
The only error message I have is OSError -1, but it tells me nothing
about the nature of the error.

Thanks,

Laszlo

Apr 4 '07 #1
7 2490
On Apr 4, 10:48 am, Laszlo Nagy <gand...@designaproduct.bizwrote:
Hello,

I have a win32 service written in Python that starts a plain
application, written in Python.

The win32 service tries to launch the application in a while loop and
logs the return value of the os.system call. That's all.

The application is a simple Python program that connects to an https
xml/rpc server, and works with the data retrieved from that server. It
is written as an application because it must be able to download updates
for itself. Here is how it works:

a.) connect to server
b.) compare current version with the latest
c.) if there is a difference, then download all sources from the server,
delete all pyc files and exit; otherwise start processing

I could not write a self-restarting server, and I did not want to
write/install two services for one problem. The win32 service is very
small and primitive so probably I will not need to update it. I think
the basic idea is good, but...

When there is a client update available, my application updates itself
cleanly and exists. Then the service tries to restart the application,
but it cannot. os.system returns with OS error code -1. The pyc files
are NOT generated for the application. However, if I start the
application from the command line, then pyc files are created, and then
the service will also start the application immediatelly. The win32
service is running as "Localsystem" so it is sure that it has write
permission on all files.

I cannot log out the error from the application since it is not started.
The only error message I have is OSError -1, but it tells me nothing
about the nature of the error.

Thanks,

Laszlo
Have you tried the subprocess module for this rather than os.system?
You might be able to pipe errors to a file with it. You might be able
to use the traceback module for more verbose error catching too.

Mike

Apr 4 '07 #2
>
Have you tried the subprocess module for this rather than os.system?
You might be able to pipe errors to a file with it. You might be able
to use the traceback module for more verbose error catching too.
Okay, I'm tried this instead of os.system:

def dumpexc(e):
import sys,traceback,StringIO
f = StringIO.StringIO('')
ei = sys.exc_info()
traceback.print_exception(ei[0],ei[1],ei[2],file=f)
return f.getvalue()

def spawn():
# ... setup mydir here....
os.chdir(mydir)
prog = os.path.join(mydir,"Application.py")
params = [sys.executable,prog]
logger.info("Spawing %s",str(params))
fout = file(os.path.join(mydir,'errorlog.txt'),'wb+')
try:
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
except Exception, e:
logger.error(dumpexc(e))
return -1
retcode = p.wait()
logger.info("Subprocess exited, return code: %d",retcode)
fout.close()
return retcode

When I call spawn() from a service, this is written into the logfile:

2007-04-05 17:52:53,828 INFO .Spawner Spawing
['C:\\Python25\\lib\\site-packages\\win32\\PythonService.exe',
'T:\\Python\\Projects\\Test\\Application.py']
2007-04-05 17:52:53,828 ERROR .Spawner Traceback (most recent call last):
File "T:\Python\Projects\Test\Processor.py", line 40, in spawn_downloader
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
File "C:\Python25\lib\subprocess.py", line 586, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python25\lib\subprocess.py", line 681, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python25\lib\subprocess.py", line 722, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
errorlog.txt is - of course - becomes an empty file.

When I call spawn() from an application, it works just fine. Any ideas?

Thanks,

Laszlo

Apr 5 '07 #3
When I call spawn() from a service, this is written into the logfile:

2007-04-05 17:52:53,828 INFO .Spawner Spawing
['C:\\Python25\\lib\\site-packages\\win32\\PythonService.exe',
'T:\\Python\\Projects\\Test\\Application.py']
2007-04-05 17:52:53,828 ERROR .Spawner Traceback (most recent call last):
File "T:\Python\Projects\Test\Processor.py", line 40, in spawn_downloader
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
File "C:\Python25\lib\subprocess.py", line 586, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python25\lib\subprocess.py", line 681, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python25\lib\subprocess.py", line 722, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required

Okay, here is what I learnt:

1. subprocess.Popen cannot redirect stderr and stdout when called from a
win32 service. This is not documented, and makes debugging almost
impossible.
2. sys.executable becomes "pythonservice.exe" inside a win32 service.

If I specify r"C:\Python25\python.exe" instead of sys.executable, and
if I do not specify stdout and stderr parameters for subprocess.Popen,
then my program starts to work. Here arises the question: how can I
find r"C:\Python25\python.exe" from inside a win32 service? Can I use this:

interpreter = os.path.join( os.path.split(sys.executable),[0],
os.sep,os.sep,os.sep,'Python.exe' )

Is it safe? Please advise.
Thanks,

Laszlo
Apr 5 '07 #4
En Thu, 05 Apr 2007 13:00:52 -0300, Laszlo Nagy
<ga*****@designaproduct.bizescribió:
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())

When I call spawn() from a service, this is written into the logfile:

2007-04-05 17:52:53,828 INFO .Spawner Spawing
['C:\\Python25\\lib\\site-packages\\win32\\PythonService.exe',
'T:\\Python\\Projects\\Test\\Application.py']
2007-04-05 17:52:53,828 ERROR .Spawner Traceback (most recent call last):
File "T:\Python\Projects\Test\Processor.py", line 40, in spawn_downloader
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
File "C:\Python25\lib\subprocess.py", line 586, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python25\lib\subprocess.py", line 681, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python25\lib\subprocess.py", line 722, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
With a bit of guessing, I think I've found what's happening.
Since you don't provide a value for stdin, None is used. Inside
subprocess.py, method _get_handles, line 670, GetStdHandle *may* return
None; in that case _make_inheritable fails.
If you print the value of p2cread in line 670 I bet you'll get None.
The fix is to test for None in _make_inheritable (line 720):

if handle is not None:
return DuplicateHandle(...)

(else return None, implicit)
When I call spawn() from an application, it works just fine. Any ideas?
According to http://msdn2.microsoft.com/en-us/library/ms683231.aspx
GetStdHandle may return NULL (translated to None in Python) when invoked
from a service with no redirected standard handles. From an application,
there is no problem.

Please try the simple fix above to comfirm it works; I'll submit a patch
if that's the case.

--
Gabriel Genellina

Apr 5 '07 #5
En Thu, 05 Apr 2007 13:50:16 -0300, Laszlo Nagy
<ga*****@designaproduct.bizescribió:
1. subprocess.Popen cannot redirect stderr and stdout when called from a
win32 service. This is not documented, and makes debugging almost
impossible.
Without the patch menctioned in my previous message, you must redirect all
stdin, stdout AND stderr (because the child cannot inherit the handles
from the parent service, as a service has no standard handles assigned
usually) or none of them.
2. sys.executable becomes "pythonservice.exe" inside a win32 service.

If I specify r"C:\Python25\python.exe" instead of sys.executable, and
if I do not specify stdout and stderr parameters for subprocess.Popen,
then my program starts to work. Here arises the question: how can I
find r"C:\Python25\python.exe" from inside a win32 service? Can I use
this:

interpreter = os.path.join( os.path.split(sys.executable),[0],
os.sep,os.sep,os.sep,'Python.exe' )
I think you meant to write: os.path.join(os.path.split(sys.executable)[0],
os.pardir, os.pardir, os.pardir, 'python.exe')

pythonservice.exe is so Windows-specific that using os.pardir only makes
the code harder to read.
Anyway I'd use os.path.join(sys.prefix, 'python.exe') (sys.prefix would be
C:\Python25 in your case)

--
Gabriel Genellina

Apr 5 '07 #6
>1. subprocess.Popen cannot redirect stderr and stdout when called from a
win32 service. This is not documented, and makes debugging almost
impossible.

Without the patch menctioned in my previous message, you must redirect all
stdin, stdout AND stderr (because the child cannot inherit the handles
from the parent service, as a service has no standard handles assigned
usually) or none of them.
The truth is that subprocess.Popen raises an exception when I try to
redirect stdin,stdout or stderr. If I do not redirect any of them, then
everything works fine for me. (Well, of course standard output is lost.)
>interpreter = os.path.join( os.path.split(sys.executable),[0],
os.sep,os.sep,os.sep,'Python.exe' )

I think you meant to write: os.path.join(os.path.split(sys.executable)[0],
os.pardir, os.pardir, os.pardir, 'python.exe')
Yes, sorry.
pythonservice.exe is so Windows-specific that using os.pardir only makes
the code harder to read.
Anyway I'd use os.path.join(sys.prefix, 'python.exe') (sys.prefix would be
C:\Python25 in your case)
Oh, I did not know about that variable. Thanks! :-)

Best,

Laszlo

Apr 5 '07 #7
With a bit of guessing, I think I've found what's happening.
Since you don't provide a value for stdin, None is used. Inside
subprocess.py, method _get_handles, line 670, GetStdHandle *may* return
None; in that case _make_inheritable fails.
If you print the value of p2cread in line 670 I bet you'll get None.
The fix is to test for None in _make_inheritable (line 720):

if handle is not None:
return DuplicateHandle(...)

(else return None, implicit)

>When I call spawn() from an application, it works just fine. Any ideas?

According to http://msdn2.microsoft.com/en-us/library/ms683231.aspx
GetStdHandle may return NULL (translated to None in Python) when invoked
from a service with no redirected standard handles. From an application,
there is no problem.

Please try the simple fix above to comfirm it works; I'll submit a patch
if that's the case.
Yes, it fixed the problem. Please submit the patch. Thank you!

Laszlo

Apr 5 '07 #8

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

10 posts views Thread by dermot | last post: by
4 posts views Thread by ags5406 | last post: by
2 posts views Thread by =?Utf-8?B?dmlzaHJ1dGg=?= | last post: by
1 post views Thread by =?Utf-8?B?dmlzaHJ1dGg=?= | last post: by
5 posts views Thread by dm3281 | last post: by
reply views Thread by suresh191 | last post: by
4 posts views Thread by guiromero | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.