Using Python with COM to communicate with proprietary Windows software

[Joakim Persson]

Hello all. I am involved in a project where we have a desire to
improve our software testing tools, and I'm in charge of looking for
solutions regarding the logging of our software (originating from
embedded devices). Currently, we are using a heavyweight, proprietary
log tool developed by another part of the company. This tool contains
all "standard" logging functionality, but we also need to insert
"debug" log points in the software of the embedded device, and we
cannot get new releases of this big log tool very often (typically,
these debug log points might be very temporarily inserted into the
software during the development phase).

Therefore, we have added a requirement to the log tool where we made
them implement a COM server in the log tool, so that we can access
arbitrary log points in the embedded device. My thoughts then, as
someone who wants to bring in more interpreted, lightweight languages
(and SW development processes) into the organization, was to use
Python to easily add log parsing and log presentation in different
ways.

This means that I must come up with a way to interface Python with
this log tool. After some investigations, I have run inte problems --
my inital approach was to use Python + win32com to communicate with
the COM interface. Unfortunately, it seems as if win32com, even if I
makepy the log tool executable (which gives me the full range of
available functions of the interface, just as expected), cannot be
used, since the interface is not a simple IDispatch interface. Also, I
discovered that the Python client must also implement a COM server to
receive the log information as it is generated by the embedded device,
passed into the log tool and then sent by the log tool COM server.

I have a rather bloated C++ client prototype which "works", so I
should have all the code needed to invoke the interface, but this
client is of course rather bloated in itself and not suitable for my
experimental programming approach. Am I right in believing that the
COM interface I am trying to use is of what is called a "custom" type,
that I cannot access using Python + win32com? I can use Python +
win32com just fine when accessing most normal COM servers (typical
example: MS Office programs implementing IDispatch interfaces), but I
cannot access the proprietary interface -- I get "interface not
supported"-type messages (I can query the CoClass for IUnknown and
IDispatch interfaces, but not using the known IID for the custom
interface).

So, I'm trying to sum up my options. I have looked at Boost to try to
merge C++ and Python code, take care of the COM server/client creation
and messaging by the C++ part, and then take care of the data
processing/presentation part in Python, like so:

+----+
|PRES| <-- Python and friends (wxPython could be interesting)
+----+
||||
+----+
|DATA| <-- Pure python
+----+
||||
+----+
|COM | <-- C++
+----+

.... all linked together into, eventually, one Python executable
(linking in the C++ through a DLL which is then imported by Python).

I haven't fully explored how to do this, but it seems as if Boost is
usable using MSVC++ (the organizations standard IDE) -- has anyone
tried something similar? My final idea is to "give up" on one single
program for this, and use a C++ intermediate program to communicate
with the log tool and then let it forward its data through sockets or
named pipes, which Python seems to deal with more easily.

I have tried using Java + jawin as well just for a quick exploratory
spike, and ran into the same problems ("interface not supported" etc).
The jawin documentation talks a bit about wrapping the custom
interface into java objects, but this is rather obscure and
undocumented, and untangling it would take quite some time -- so I
figured that somebody could have done something similar in the past.

Does anyone have any hints on how to go forward? The whole point with
my project is to demonstrate the usefulness of rapid prototyping using
Python (this has already been successful for smaller test tools, test
scripts and smaller intermediary programs), but I don't want something
like COM client/server obscurity getting in the way...

Thanks in advance,

--
Joakim Persson
M.Sc student, CS/E @ LTH

[Thomas Heller]

Joakim Persson <za*****@home.se> writes:

[...]

This means that I must come up with a way to interface Python with
this log tool. After some investigations, I have run inte problems --
my inital approach was to use Python + win32com to communicate with
the COM interface. Unfortunately, it seems as if win32com, even if I
makepy the log tool executable (which gives me the full range of
available functions of the interface, just as expected), cannot be
used, since the interface is not a simple IDispatch interface. Also, I
discovered that the Python client must also implement a COM server to
receive the log information as it is generated by the embedded device,
passed into the log tool and then sent by the log tool COM server.

I have a rather bloated C++ client prototype which "works", so I
should have all the code needed to invoke the interface, but this
client is of course rather bloated in itself and not suitable for my
experimental programming approach. Am I right in believing that the
COM interface I am trying to use is of what is called a "custom" type,
that I cannot access using Python + win32com?

Sounds like a perfect job for comtypes, which is a COM library
implemented in pure Python, based on ctypes. comtypes should make it
easy to access custom (non-dispatch derived) com interfaces, or the
vtable based part of dual interfaces - it would be good however, if you
have a type library for the interfaces.

http://sourceforge.net/projects/comtypes/

No docs yet, but there are tests included which should get you started.

(I have released and announced this 3 weeks ago, but haven't got a
single feedback. So it seems the need to access custom interfaces is
very low.)

Thommas

[Joakim Persson]

On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <th*****@python.net>
wrote:

Joakim Persson <za*****@home.se> writes:

I have a rather bloated C++ client prototype which "works", so I
should have all the code needed to invoke the interface, but this
client is of course rather bloated in itself and not suitable for my
experimental programming approach. Am I right in believing that the
COM interface I am trying to use is of what is called a "custom" type,
that I cannot access using Python + win32com?

Sounds like a perfect job for comtypes, which is a COM library
implemented in pure Python, based on ctypes. comtypes should make it
easy to access custom (non-dispatch derived) com interfaces, or the
vtable based part of dual interfaces - it would be good however, if you
have a type library for the interfaces.

http://sourceforge.net/projects/comtypes/

No docs yet, but there are tests included which should get you started.

(I have released and announced this 3 weeks ago, but haven't got a
single feedback. So it seems the need to access custom interfaces is
very low.)

Thommas

Looks good, I think I downloaded ctypes but didn't put in the effort
to wrap the custom COM interface.

I have the type library and full documentation of the interface, so
IMO it _should_ be possible to make a 100% Python application for
testing this particular interface, which would definitely speed up
prototyping, which is the entire goal of my work. I will give it a
spin tomorrow, and I'll get back on the NG once I have tested it. Many
thanks!

It seems most Python uses for COM rely on "simple" automation of COM
interfaces of the IDispatch type, but I need something more (full
client<->server communication and threading, for instance). Hopefully
your module will at least let me get one step further...

My remaining two options, seeing as I would really like to at least
put the data processing and presentation part in Python, are linking
Python with C++ for the COM part and building (Data + Presentation) in
Python, or making a complete standalone C++ part for the COM part and
then _another_ client<->server solution using e.g. named
pipes/sockets.

--
Joakim Persson
M.Sc student, CS/E @ LTH

[Andrew MacIntyre]

On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <th*****@python.net>
wrote:

{...}

(I have released and announced this 3 weeks ago, but haven't got a
single feedback. So it seems the need to access custom interfaces is
very low.)

I have downloaded it and am trying to find the time to play with it
(unsuccessfully so far).

As someone working with a large, complex, COM library with minimal
IDispatch support, I'm really looking forward to this.

However, at the moment I'm limited to Python 2.2 and ctypes 0.6.3 (which
is allowing me to get the job done!!).

Regardless, I thank you for what you have released!

Cheers,
Andrew.

-------------------------------------------------------------------------
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: an*****@bullseye.apana.org.au (pref) | Snail: PO Box 370
an*****@pcug.org.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia

[Joakim Persson]

On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <th*****@python.net>
wrote:

Sounds like a perfect job for comtypes, which is a COM library
implemented in pure Python, based on ctypes. comtypes should make it
easy to access custom (non-dispatch derived) com interfaces, or the
vtable based part of dual interfaces - it would be good however, if you
have a type library for the interfaces.

http://sourceforge.net/projects/comtypes/

No docs yet, but there are tests included which should get you started.

(I have released and announced this 3 weeks ago, but haven't got a
single feedback. So it seems the need to access custom interfaces is
very low.)

Thommas

After some testing today, it does seem to do exactly what I wanted --
I can now access the "custom" but IDispatch-like COM interface that I
couldn't access with win32com (or with Java + jawin). It might be
possible in other ways, but using comtypes was definitely the most
painfree way (everything, including the return values from the
methods, worked as expected). Thank you very much.

Of course, this does not complete my task -- although I can now use my
interface to send messages and commands to the big log tool, I still
need to implement a COM server and pass a pointer to its interface
through one of the messages to the com server to be able to receive
data:

BridgeInterface.StartLogging(filename) <--- works fine, didn't work
before
BridgeInterface.Advise(ptr) <--- Now, I need to create a new
interface for receiving the data sent from the log application, so
that I can (at first) print it

This _shouldn't_ be too difficult -- I know which methods must be
implemented (basically just some kind of event handling to deal with
randomly arriving log points, should be implemented as "onMsg()" on my
COM server side, and some other similar methods), but I don't really
know how. I have tried doing simple COM servers using win32com, but is
it equally possible to implement such a simple thing in comtypes? I
didn't find any server side examples in comtypes, but perhaps there is
a way?

--
Joakim Persson
M.Sc student, CS/E @ LTH

[Thomas Heller]

Andrew MacIntyre <an*****@bullseye.apana.org.au> writes:

On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <th*****@python.net>
wrote:

{...}

(I have released and announced this 3 weeks ago, but haven't got a
single feedback. So it seems the need to access custom interfaces is
very low.)

I have downloaded it and am trying to find the time to play with it
(unsuccessfully so far).

As someone working with a large, complex, COM library with minimal
IDispatch support, I'm really looking forward to this.

However, at the moment I'm limited to Python 2.2 and ctypes 0.6.3
(which is allowing me to get the job done!!).

Regardless, I thank you for what you have released!

I knew there are fans out there ;-)

Thomas

[Thomas Heller]

Joakim Persson <za*****@home.se> writes:

On Fri, 09 Sep 2005 08:36:00 +0200, Thomas Heller <th*****@python.net>
wrote:

Sounds like a perfect job for comtypes, which is a COM library
implemented in pure Python, based on ctypes. comtypes should make it
easy to access custom (non-dispatch derived) com interfaces, or the
vtable based part of dual interfaces - it would be good however, if you
have a type library for the interfaces.

http://sourceforge.net/projects/comtypes/

No docs yet, but there are tests included which should get you started.

(I have released and announced this 3 weeks ago, but haven't got a
single feedback. So it seems the need to access custom interfaces is
very low.)

Thommas

After some testing today, it does seem to do exactly what I wanted --
I can now access the "custom" but IDispatch-like COM interface that I
couldn't access with win32com (or with Java + jawin). It might be
possible in other ways, but using comtypes was definitely the most
painfree way (everything, including the return values from the
methods, worked as expected). Thank you very much.

Of course, this does not complete my task -- although I can now use my
interface to send messages and commands to the big log tool, I still
need to implement a COM server and pass a pointer to its interface
through one of the messages to the com server to be able to receive
data:

BridgeInterface.StartLogging(filename) <--- works fine, didn't work
before
BridgeInterface.Advise(ptr) <--- Now, I need to create a new
interface for receiving the data sent from the log application, so
that I can (at first) print it

This _shouldn't_ be too difficult -- I know which methods must be
implemented (basically just some kind of event handling to deal with
randomly arriving log points, should be implemented as "onMsg()" on my
COM server side, and some other similar methods), but I don't really
know how. I have tried doing simple COM servers using win32com, but is
it equally possible to implement such a simple thing in comtypes? I
didn't find any server side examples in comtypes, but perhaps there is
a way?

There is not yet any server support in comtypes, but it will be added.
In the meantime you could use the ctypes.com package that is included in
ctypes itself, there are even samples.

Thomas

[Joakim Persson]

On Tue, 13 Sep 2005 10:15:18 +0200, Thomas Heller <th*****@python.net>
wrote:

This _shouldn't_ be too difficult -- I know which methods must be
implemented (basically just some kind of event handling to deal with
randomly arriving log points, should be implemented as "onMsg()" on my
COM server side, and some other similar methods), but I don't really
know how. I have tried doing simple COM servers using win32com, but is
it equally possible to implement such a simple thing in comtypes? I
didn't find any server side examples in comtypes, but perhaps there is
a way?

There is not yet any server support in comtypes, but it will be added.
In the meantime you could use the ctypes.com package that is included in
ctypes itself, there are even samples.

Thomas

Thank you, I will try that. Right now I've been trying to use win32com
for the server creation, but I haven't been successful yet (have not
explored it that much yet though).

I have registered a COM server (it should only implement two methods,
"onMsg()" and "onNotify()", which should print whatever is sent to
them directly to stdout), but I need to create an interface which is
not an IDispatch interface and pass it on to the proprietary COM
server. This connection is needed so that the proprietary COM server
knows where to send the log and control data.

Registering the COM object in the registry works fine, but how do I
make it look like a custom interface from the proprietary COM servers
point of view? Oh well, I will look at it some more tomorrow and
hopefully come up with a solution...

--
Joakim Persson
M.Sc student, CS/E @ LTH

[Paul Casteels]

Joakim Persson wrote:

On Tue, 13 Sep 2005 10:15:18 +0200, Thomas Heller <th*****@python.net>
wrote:

This _shouldn't_ be too difficult -- I know which methods must be
implemented (basically just some kind of event handling to deal with
randomly arriving log points, should be implemented as "onMsg()" on my
COM server side, and some other similar methods), but I don't really
know how. I have tried doing simple COM servers using win32com, but is
it equally possible to implement such a simple thing in comtypes? I
didn't find any server side examples in comtypes, but perhaps there is
a way?

There is not yet any server support in comtypes, but it will be added.
In the meantime you could use the ctypes.com package that is included in
ctypes itself, there are even samples.

Thomas

Thank you, I will try that. Right now I've been trying to use win32com
for the server creation, but I haven't been successful yet (have not
explored it that much yet though).

I have registered a COM server (it should only implement two methods,
"onMsg()" and "onNotify()", which should print whatever is sent to
them directly to stdout), but I need to create an interface which is
not an IDispatch interface and pass it on to the proprietary COM
server. This connection is needed so that the proprietary COM server
knows where to send the log and control data.

Registering the COM object in the registry works fine, but how do I
make it look like a custom interface from the proprietary COM servers
point of view? Oh well, I will look at it some more tomorrow and
hopefully come up with a solution...

Hi Joakim,

This server I wrote for the Palm Desktop with win32com, it creates a none IDispatch COMserver.
Maybe it can be of help. I am interested in what you find out on this.

Regards,

Paul Casteels
import pythoncom
import pywintypes
import winerror
import types,time,string
import win32com
import win32ui,win32con
import rfc822
import XmlWriter

from win32com.server.exception import COMException
from win32com.server.util import wrap
from win32com.client import gencache,Dispatch,constants
from win32com import universal

# The following 3 lines may need tweaking for the particular server
# Candidates are pythoncom.Missing and pythoncom.Empty
defaultNamedOptArg=pythoncom.Missing
defaultNamedNotOptArg=pythoncom.Missing
defaultUnnamedArg=pythoncom.Missing

palmApp = 0
outfileName = r'c:\Documents And Settings\paul\My Documents\Python\PDA\cal2.xml'

universal.RegisterInterfaces('{C9B354D8-4A1C-11D5-81D2-00C04FA03755}', 0, 1, 0, ["IDesktopAddin"])
universal.RegisterInterfaces('{C9B354D8-4A1C-11D5-81D2-00C04FA03755}', 0, 1, 0, ["IDesktopCommand"])

def d2s(inDate):
return rfc822.mktime_tz(rfc822.parsedate_tz(inDate))
def s2d(inDate):
return time.asctime(time.localtime(inDate))

class DesktopCommand:
global palmApp

_public_methods_ = ['OnCommand']
_com_interfaces_ = ['IDesktopCommand']

def OnCommand(self):
print "Hello, command"
print palmApp
win32ui.MessageBox("PC-Hello","PC-Title",win32con.MB_OK )

class DesktopAddin:
_public_methods_ = ['GetFrameworkVersion',
'OnInitialize',
'OnUninitialize',
'ShowAboutBox',
'ShowProperties']
_reg_progid_ = "PythonPalm.DesktopAddin"
_reg_clsid_ = "{6F36C1D4-571C-484E-B47B-61297761B3D1}"
_com_interfaces_ = ['IDesktopAddin']

def GetFrameworkVersion(self, plVersion=defaultNamedNotOptArg):
"""method GetFrameworkVersion"""
return 2

def OnInitialize(self, pUnk=defaultNamedNotOptArg, dwCookie=defaultNamedNotOptArg, \
dwDesktopVersion=defaultNamedNotOptArg, lParam=defaultNamedNotOptArg, \
pbSuccess=defaultNamedNotOptArg):
"""method OnInitialize"""
global palmApp
self.cookie = dwCookie
palmApp = Dispatch(pUnk)
# add a command
self.cmd = wrap(DesktopCommand())
addinMgr = palmApp.AddinManager
cmdId = 0
cmdId = addinMgr.AddAddinCommand(self.cookie, "AddressCommand", \
self.cmd, cmdId)
cmdDesc = "&Address Command"
# & vbTab & _
# "Displays a list of 10 contacts." & vbTab & _
# "Displays 10 contacts"

addinMgr.AddAddinCommandDetails(cmdId, 0, 0, cmdDesc, constants.ePimAddressBook)

# no data here yet available
return 1

def OnUninitialize(self):
"""method OnUninitialize"""
global palmApp

of = XmlWriter.XmlFile(outfileName,palmApp.CurrentUser. Name)

dbPim = palmApp.DateBookPim
startDate = '1 Jan 2000 0:0:0'
endDate = '1 Jan 2010 0:0:0'
events = dbPim.Events(d2s(startDate),d2s(endDate))
for j in range(events.Count):
startDate = time.localtime(events[j].StartTime)
day = of.day(startDate)
app = {}
app['description'] = events[j].Description
if events[j].Untimed:
app['untimed'] = 1
else:
app['untimed'] = 0
app['starttime'] = s2d(events[j].StartTime)
app['endtime'] = s2d(events[j].EndTime)
if events[j].Note:
app['note'] = events[j].Note
# app['id'] = 'id%d' % (events[j].RecordId)
of.appointment(app,day)
of.close()

# adPim = palmApp.AddressBookPim
# contacts = adPim.Contacts("All")
# for j in range(contacts.Count):
# print contacts[j].LastName
return None

def ShowAboutBox(self, hWndParent=defaultNamedNotOptArg):
"""method ShowAboutBox"""
print 'ShowAboutBox'
print hWndParent
return None

def ShowProperties(self, hWndParent=defaultNamedNotOptArg):
"""method ShowProperties"""
print 'ShowProperties'
return None

_prop_map_get_ = {
}
_prop_map_put_ = {
}

if __name__ == "__main__":
print "Registering COM server"
import win32com.server.register
win32com.server.register.UseCommandLine(DesktopAdd in)

print "Registering our server with Palm desktop"
palmReg = Dispatch("PalmDesktop.RegistrationServices")
lRegistered = palmReg.IsAddinRegistered ("PythonPalm.DesktopAddin")
if lRegistered == 1:
print "Removing PythonPalm.DesktopAddin"
addinInfo = palmReg.GetAddinRegInfo("PythonPalm.DesktopAddin")
addinInfo.Delete()
print "Registering PythonPalm.DesktopAddin"
palmReg.RegisterAddin ("PythonPalm.DesktopAddin","Python Addin")