473,320 Members | 1,744 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Windows service and pyc files


Hello,

I have a win32 service written in Python that starts a plain
application, written in Python.

The win32 service tries to launch the application in a while loop and
logs the return value of the os.system call. That's all.

The application is a simple Python program that connects to an https
xml/rpc server, and works with the data retrieved from that server. It
is written as an application because it must be able to download updates
for itself. Here is how it works:

a.) connect to server
b.) compare current version with the latest
c.) if there is a difference, then download all sources from the server,
delete all pyc files and exit; otherwise start processing

I could not write a self-restarting server, and I did not want to
write/install two services for one problem. The win32 service is very
small and primitive so probably I will not need to update it. I think
the basic idea is good, but...

When there is a client update available, my application updates itself
cleanly and exists. Then the service tries to restart the application,
but it cannot. os.system returns with OS error code -1. The pyc files
are NOT generated for the application. However, if I start the
application from the command line, then pyc files are created, and then
the service will also start the application immediatelly. The win32
service is running as "Localsystem" so it is sure that it has write
permission on all files.

I cannot log out the error from the application since it is not started.
The only error message I have is OSError -1, but it tells me nothing
about the nature of the error.

Thanks,

Laszlo

Apr 4 '07 #1
7 2643
On Apr 4, 10:48 am, Laszlo Nagy <gand...@designaproduct.bizwrote:
Hello,

I have a win32 service written in Python that starts a plain
application, written in Python.

The win32 service tries to launch the application in a while loop and
logs the return value of the os.system call. That's all.

The application is a simple Python program that connects to an https
xml/rpc server, and works with the data retrieved from that server. It
is written as an application because it must be able to download updates
for itself. Here is how it works:

a.) connect to server
b.) compare current version with the latest
c.) if there is a difference, then download all sources from the server,
delete all pyc files and exit; otherwise start processing

I could not write a self-restarting server, and I did not want to
write/install two services for one problem. The win32 service is very
small and primitive so probably I will not need to update it. I think
the basic idea is good, but...

When there is a client update available, my application updates itself
cleanly and exists. Then the service tries to restart the application,
but it cannot. os.system returns with OS error code -1. The pyc files
are NOT generated for the application. However, if I start the
application from the command line, then pyc files are created, and then
the service will also start the application immediatelly. The win32
service is running as "Localsystem" so it is sure that it has write
permission on all files.

I cannot log out the error from the application since it is not started.
The only error message I have is OSError -1, but it tells me nothing
about the nature of the error.

Thanks,

Laszlo
Have you tried the subprocess module for this rather than os.system?
You might be able to pipe errors to a file with it. You might be able
to use the traceback module for more verbose error catching too.

Mike

Apr 4 '07 #2
>
Have you tried the subprocess module for this rather than os.system?
You might be able to pipe errors to a file with it. You might be able
to use the traceback module for more verbose error catching too.
Okay, I'm tried this instead of os.system:

def dumpexc(e):
import sys,traceback,StringIO
f = StringIO.StringIO('')
ei = sys.exc_info()
traceback.print_exception(ei[0],ei[1],ei[2],file=f)
return f.getvalue()

def spawn():
# ... setup mydir here....
os.chdir(mydir)
prog = os.path.join(mydir,"Application.py")
params = [sys.executable,prog]
logger.info("Spawing %s",str(params))
fout = file(os.path.join(mydir,'errorlog.txt'),'wb+')
try:
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
except Exception, e:
logger.error(dumpexc(e))
return -1
retcode = p.wait()
logger.info("Subprocess exited, return code: %d",retcode)
fout.close()
return retcode

When I call spawn() from a service, this is written into the logfile:

2007-04-05 17:52:53,828 INFO .Spawner Spawing
['C:\\Python25\\lib\\site-packages\\win32\\PythonService.exe',
'T:\\Python\\Projects\\Test\\Application.py']
2007-04-05 17:52:53,828 ERROR .Spawner Traceback (most recent call last):
File "T:\Python\Projects\Test\Processor.py", line 40, in spawn_downloader
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
File "C:\Python25\lib\subprocess.py", line 586, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python25\lib\subprocess.py", line 681, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python25\lib\subprocess.py", line 722, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
errorlog.txt is - of course - becomes an empty file.

When I call spawn() from an application, it works just fine. Any ideas?

Thanks,

Laszlo

Apr 5 '07 #3
When I call spawn() from a service, this is written into the logfile:

2007-04-05 17:52:53,828 INFO .Spawner Spawing
['C:\\Python25\\lib\\site-packages\\win32\\PythonService.exe',
'T:\\Python\\Projects\\Test\\Application.py']
2007-04-05 17:52:53,828 ERROR .Spawner Traceback (most recent call last):
File "T:\Python\Projects\Test\Processor.py", line 40, in spawn_downloader
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
File "C:\Python25\lib\subprocess.py", line 586, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python25\lib\subprocess.py", line 681, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python25\lib\subprocess.py", line 722, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required

Okay, here is what I learnt:

1. subprocess.Popen cannot redirect stderr and stdout when called from a
win32 service. This is not documented, and makes debugging almost
impossible.
2. sys.executable becomes "pythonservice.exe" inside a win32 service.

If I specify r"C:\Python25\python.exe" instead of sys.executable, and
if I do not specify stdout and stderr parameters for subprocess.Popen,
then my program starts to work. Here arises the question: how can I
find r"C:\Python25\python.exe" from inside a win32 service? Can I use this:

interpreter = os.path.join( os.path.split(sys.executable),[0],
os.sep,os.sep,os.sep,'Python.exe' )

Is it safe? Please advise.
Thanks,

Laszlo
Apr 5 '07 #4
En Thu, 05 Apr 2007 13:00:52 -0300, Laszlo Nagy
<ga*****@designaproduct.bizescribió:
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())

When I call spawn() from a service, this is written into the logfile:

2007-04-05 17:52:53,828 INFO .Spawner Spawing
['C:\\Python25\\lib\\site-packages\\win32\\PythonService.exe',
'T:\\Python\\Projects\\Test\\Application.py']
2007-04-05 17:52:53,828 ERROR .Spawner Traceback (most recent call last):
File "T:\Python\Projects\Test\Processor.py", line 40, in spawn_downloader
p = subprocess.Popen(params, bufsize=1, stdout=fout.fileno(),
stderr=fout.fileno())
File "C:\Python25\lib\subprocess.py", line 586, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "C:\Python25\lib\subprocess.py", line 681, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "C:\Python25\lib\subprocess.py", line 722, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
With a bit of guessing, I think I've found what's happening.
Since you don't provide a value for stdin, None is used. Inside
subprocess.py, method _get_handles, line 670, GetStdHandle *may* return
None; in that case _make_inheritable fails.
If you print the value of p2cread in line 670 I bet you'll get None.
The fix is to test for None in _make_inheritable (line 720):

if handle is not None:
return DuplicateHandle(...)

(else return None, implicit)
When I call spawn() from an application, it works just fine. Any ideas?
According to http://msdn2.microsoft.com/en-us/library/ms683231.aspx
GetStdHandle may return NULL (translated to None in Python) when invoked
from a service with no redirected standard handles. From an application,
there is no problem.

Please try the simple fix above to comfirm it works; I'll submit a patch
if that's the case.

--
Gabriel Genellina

Apr 5 '07 #5
En Thu, 05 Apr 2007 13:50:16 -0300, Laszlo Nagy
<ga*****@designaproduct.bizescribió:
1. subprocess.Popen cannot redirect stderr and stdout when called from a
win32 service. This is not documented, and makes debugging almost
impossible.
Without the patch menctioned in my previous message, you must redirect all
stdin, stdout AND stderr (because the child cannot inherit the handles
from the parent service, as a service has no standard handles assigned
usually) or none of them.
2. sys.executable becomes "pythonservice.exe" inside a win32 service.

If I specify r"C:\Python25\python.exe" instead of sys.executable, and
if I do not specify stdout and stderr parameters for subprocess.Popen,
then my program starts to work. Here arises the question: how can I
find r"C:\Python25\python.exe" from inside a win32 service? Can I use
this:

interpreter = os.path.join( os.path.split(sys.executable),[0],
os.sep,os.sep,os.sep,'Python.exe' )
I think you meant to write: os.path.join(os.path.split(sys.executable)[0],
os.pardir, os.pardir, os.pardir, 'python.exe')

pythonservice.exe is so Windows-specific that using os.pardir only makes
the code harder to read.
Anyway I'd use os.path.join(sys.prefix, 'python.exe') (sys.prefix would be
C:\Python25 in your case)

--
Gabriel Genellina

Apr 5 '07 #6
>1. subprocess.Popen cannot redirect stderr and stdout when called from a
win32 service. This is not documented, and makes debugging almost
impossible.

Without the patch menctioned in my previous message, you must redirect all
stdin, stdout AND stderr (because the child cannot inherit the handles
from the parent service, as a service has no standard handles assigned
usually) or none of them.
The truth is that subprocess.Popen raises an exception when I try to
redirect stdin,stdout or stderr. If I do not redirect any of them, then
everything works fine for me. (Well, of course standard output is lost.)
>interpreter = os.path.join( os.path.split(sys.executable),[0],
os.sep,os.sep,os.sep,'Python.exe' )

I think you meant to write: os.path.join(os.path.split(sys.executable)[0],
os.pardir, os.pardir, os.pardir, 'python.exe')
Yes, sorry.
pythonservice.exe is so Windows-specific that using os.pardir only makes
the code harder to read.
Anyway I'd use os.path.join(sys.prefix, 'python.exe') (sys.prefix would be
C:\Python25 in your case)
Oh, I did not know about that variable. Thanks! :-)

Best,

Laszlo

Apr 5 '07 #7
With a bit of guessing, I think I've found what's happening.
Since you don't provide a value for stdin, None is used. Inside
subprocess.py, method _get_handles, line 670, GetStdHandle *may* return
None; in that case _make_inheritable fails.
If you print the value of p2cread in line 670 I bet you'll get None.
The fix is to test for None in _make_inheritable (line 720):

if handle is not None:
return DuplicateHandle(...)

(else return None, implicit)

>When I call spawn() from an application, it works just fine. Any ideas?

According to http://msdn2.microsoft.com/en-us/library/ms683231.aspx
GetStdHandle may return NULL (translated to None in Python) when invoked
from a service with no redirected standard handles. From an application,
there is no problem.

Please try the simple fix above to comfirm it works; I'll submit a patch
if that's the case.
Yes, it fixed the problem. Please submit the patch. Thank you!

Laszlo

Apr 5 '07 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Ken Lindner | last post by:
I have a need to become familiar with SQL Server 2000 for work. Needless to say I am new to SQL Server any version, but not IT in general. My employer has provided me with the SQL Server 2000...
3
by: belgiozen | last post by:
Hi, I have a working windows service,it is looking for files on the disk and when some of the files are cupdated it calls an executable. But it takes a lot of time(about 10 minutes) to run the...
10
by: dermot | last post by:
I have wrriten a small windows service application in visual studio ..net 2003 which listens for incoming FTP files. These files would overwrite over time due to duplicate file names. However any...
3
by: Chung Leong | last post by:
Here's the rest of the tutorial I started earlier: Aside from text within a document, Indexing Service let you search on meta information stored in the files. For example, MusicArtist and...
4
by: sajid_yusuf | last post by:
Hi I am trying to develop a Windows service in VB.NET which has timer enabled and keeps checking a folder (or group of folders) for any new file or changed files. As soon as it detects any new...
4
by: ags5406 | last post by:
Hi All I have a task that I'm thinking about attacking in two different ways but don't know enough about Windows Services. First: We have a server application written in VB.net that acts as...
4
by: Steven De Smet | last post by:
Hello, This is my first post. I searched on the internet for answers but I was unable to solve my problem. So I hope that you guy's can help me with my VB.NET problem I tried to create a...
2
by: =?Utf-8?B?dmlzaHJ1dGg=?= | last post by:
Hi, I have 2 applications running, one Windows application project and the other windows services project. I want to call my Windows application in my windows services. I want to run them as...
1
by: =?Utf-8?B?dmlzaHJ1dGg=?= | last post by:
Hi, I have 2 applications running, one Windows application project and the other windows services project. I want to call my Windows application in my windows services. I want to run them as...
5
by: dm3281 | last post by:
I'm really starting to hate writing services -- or trying to, anyway. Why do I need to rename my project to the service name? Why do I need to set the "ServiceName" property to my service name?...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.