msvcp71.dll and Python 2.4 C++ extensions

[Matthias Baas]

Hi,

are there any guidelines about what to do if a Windows extension for
Python 2.4 requires the C++ runtime (msvcp71.dll)? If I want to
distribute a binary installer of an extension that contains C++ code,
should I really include the msvcp71.dll in the package? It doesn't
sound like a good idea to me if every package places another copy of
this dll somewhere in the Python directory.
Python 2.4 does install the C runtime (msvcr71.dll) in the Windows
system32 directory, doesn't it? (btw, shouldn't this be installed in
the Python directory instead?) So would it be possible that future
Python releases would also install the C++ runtime? I think it's
better if the dll would only be installed once by Python instead of
several times by every extension that uses C++.

Currently, I do not package the C++ runtime and leave it up to the
user to install the dll if it isn't already somewhere on his system.
But this really is not a satisfying solution...

- Matthias -

[Martin v. Löwis]

Matthias Baas wrote:

are there any guidelines about what to do if a Windows extension for
Python 2.4 requires the C++ runtime (msvcp71.dll)?
No; it should "just work fine". The standard guidelines apply, of
course: never try to mix different versions of similar DLLs.
If I want to
distribute a binary installer of an extension that contains C++ code,
should I really include the msvcp71.dll in the package?
If you cannot expect your users to have that file, you should distribute
it. Make sure you have read and understood the Microsoft license
concerning the file.
It doesn't
sound like a good idea to me if every package places another copy of
this dll somewhere in the Python directory.
Why not? If your installer follows Windows logo compliance rules, it
should check whether the DLL is already present; if it is, it should
check whether the version installed is newer than the one it would
install, and if so, should skip installation.
Python 2.4 does install the C runtime (msvcr71.dll) in the Windows
system32 directory, doesn't it?
It depends. If this is an allusers installation, it does so. For a
per-user installation, the user might not have sufficient privileges
to install into system32, so the installer won't install it there.
(btw, shouldn't this be installed in
the Python directory instead?)
No. If you install the DLL into the Python directory, Python
will not work anymore because python24.dll (installed into system32)
will not find this DLL. If you then also move python24.dll into
the Python directory, COM components that require python24.dll will
fail to run.
So would it be possible that future
Python releases would also install the C++ runtime?
Why should it? Nothing in the Python distribution requires C++.
I think it's
better if the dll would only be installed once by Python instead of
several times by every extension that uses C++.
Why is that better? The DLL versioning rules, and the shared DLL
refcounting mechanisms will make sure that the installation works
just fine.
Currently, I do not package the C++ runtime and leave it up to the
user to install the dll if it isn't already somewhere on his system.
But this really is not a satisfying solution...

Then you should probably include the DLL, or rewrite your extension
to not use C++.

Regards,
Martin

[Matthias Baas]

On Tue, 01 Feb 2005 00:14:30 +0100, "Martin v. Löwis"
<ma****@v.loewis.de> wrote:

are there any guidelines about what to do if a Windows extension for
Python 2.4 requires the C++ runtime (msvcp71.dll)?

No; it should "just work fine". [...]

I fully agree with that. :) And that was actually the reason why I
posted here, because in my case it did not just work fine...

It doesn't
sound like a good idea to me if every package places another copy of
this dll somewhere in the Python directory.

Why not? If your installer follows Windows logo compliance rules, it
should check whether the DLL is already present; if it is, it should
check whether the version installed is newer than the one it would
install, and if so, should skip installation.

I'm creating the installer via the distutils by calling "setup.py
bdist_wininst". How can I configure distutils to have it create an
installer that does the above things?

How should an installer check if a DLL is already present? Are DLLs
registered with Windows so I can query a database? Or should the
installer search for the DLL in the directories specified in the PATH
variable and see if the DLL is there?
And what's the preferred location for such a DLL? Should it be placed
in a local directory that only belongs to the package or in a
directory where other C++ extension packages might also be able to use
it? But where would this be then? The Windows system directory seems
to be discouraged by Microsoft:

"An application should use and redistribute msvcr71.dll, and it should
avoid placing a copy or using an existing copy of msvcr71.dll in the
system directory. Instead, the application should keep a copy of
msvcr71.dll in its application directory with the program executable."
(see
http://msdn.microsoft.com/library/de..._libraries.asp)
I suppose the same applies for msvcp71.dll.

Python 2.4 does install the C runtime (msvcr71.dll) in the Windows
system32 directory, doesn't it?

It depends. If this is an allusers installation, it does so. For a
per-user installation, the user might not have sufficient privileges
to install into system32, so the installer won't install it there.

Ah, ok. And yes, I did a "all users" installation.

(btw, shouldn't this be installed in
the Python directory instead?)

No. If you install the DLL into the Python directory, Python
will not work anymore because python24.dll (installed into system32)
will not find this DLL. If you then also move python24.dll into
the Python directory, COM components that require python24.dll will
fail to run.

ok, I see.

So would it be possible that future
Python releases would also install the C++ runtime?

Why should it? Nothing in the Python distribution requires C++.

Well, yes, I know. But I don't see Python as a standalone application.
One of the great things about Python is that there are so many
extension modules for every kinds of purposes. So, making available
the C++ runtime would just pave the way for more of those extension
modules by making it easier to distribute binary packages that are
implemented in C++.

I think it's
better if the dll would only be installed once by Python instead of
several times by every extension that uses C++.

Why is that better? The DLL versioning rules, and the shared DLL
refcounting mechanisms will make sure that the installation works
just fine.

I was referring to the possibility that the DLL might get installed
several times at several different locations which would be a waste of
disk space. If there's only one file, then I agree with the above
(however, what happens if I uninstall one package and this package
removes the DLL even when there are other packages installed that
still require the DLL?).

Currently, I do not package the C++ runtime and leave it up to the
user to install the dll if it isn't already somewhere on his system.
But this really is not a satisfying solution...

Then you should probably include the DLL, or rewrite your extension
to not use C++.

I'm afraid the latter will hardly ever be an option... ;)

For one smaller package I was compiling I solved the problem by
linking against the static versions of the C/C++ runtime. This will
also satisfy the "It-Should-Just-Work" constraint... :) (however,
there still could be situations where this is not possible (the
sources might not be available, for example)).

- Matthias -

[Martin v. Löwis]

Matthias Baas wrote:

I'm creating the installer via the distutils by calling "setup.py
bdist_wininst". How can I configure distutils to have it create an
installer that does the above things?
Ah, I see. Unfortunately, bdist_wininst is not capable of doing
a Windows logo compliant installation (with proper versioning checks,
shared dll reference count, etc) of a DLL. You will have to find
a different technology to distribute your application - or make sure
you copy the DLL in a place where it previously did not exist.
How should an installer check if a DLL is already present?
It should check if the file exists. If it exists, it should check
the version number of the file (there are APIs for that available),
and if the language of the installed DLL differs from the language
of the package's DLL. If the new DLL is somehow "better" (newer
version, or same version, different language, and user requested
overwriting), it should replace the DLL.
Are DLLs registered with Windows so I can query a database?
To some degree. If the software is uninstalled, the installer should
remove the file. If the file was already present at installation time,
removal may break other software. Therefor, there is a registry entry
for each path name that is a reference count, under

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Curr entVersion\SharedDLLs

If you expect that the file is already there, you need to use that
mechanism.
Or should the
installer search for the DLL in the directories specified in the PATH
variable and see if the DLL is there?
No. There is no problem at all to install the same DLL multiple times,
in different places. In fact, Microsoft discourages shared DLLs, and
recommends that each package should install its own copy.

Windows will look for a DLL in
- the directory of the EXE
- the current working directory
- system32
- windows
- PATH

In your case, putting the C++ DLL into the directory containing
python.exe might be sufficient. The safest thing, of course, is to
put the DLL into system32 - in which case you *have* to install it
shared.
"An application should use and redistribute msvcr71.dll, and it should
avoid placing a copy or using an existing copy of msvcr71.dll in the
system directory. Instead, the application should keep a copy of
msvcr71.dll in its application directory with the program executable."
Yes. This means to install it into the directory containing python.exe.
Unfortunately, this does not work if your extension is then used in
an embedded Python, e.g. one running as a COM object. In this case,
python.exe will not be the main executable, but, say, outlook.exe will
be. So Microsoft's recommendation to install the DLL into the EXE
directory is flawed for COM and other embedded cases. This is
precisely the reason why the Python installer installs both python24.dll
and msvcr71.dll into system32.
Well, yes, I know. But I don't see Python as a standalone application.
One of the great things about Python is that there are so many
extension modules for every kinds of purposes. So, making available
the C++ runtime would just pave the way for more of those extension
modules by making it easier to distribute binary packages that are
implemented in C++.
I think we agree that building and running such an extension is not
a problem - it just works.

Distributing such an extension is indeed a problem, as bdist_wininst
does not support this case properly.

I'll plan to write a bdist_msi one day, which should handle DLL
installation better as much of the magic required is built into
Windows Installer.
I was referring to the possibility that the DLL might get installed
several times at several different locations which would be a waste of
disk space. If there's only one file, then I agree with the above
(however, what happens if I uninstall one package and this package
removes the DLL even when there are other packages installed that
still require the DLL?).
The other packages will also have added to the shared reference counter.
Uninstalling will only DECREF the counter, and the file will be removed
only when the counter goes to zero.
For one smaller package I was compiling I solved the problem by
linking against the static versions of the C/C++ runtime. This will
also satisfy the "It-Should-Just-Work" constraint... :) (however,
there still could be situations where this is not possible (the
sources might not be available, for example)).

You should make sure to use the DLL version of the C runtime. Otherwise,
you will be mixing multiple C runtimes (one that python24.dll links
with, and one linked statically into your extension); this might cause
troubles (such as memory leaks and crashes).

Regards,
Martin