By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
435,062 Members | 1,358 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 435,062 IT Pros & Developers. It's quick & easy.

Step by step: Compiling extensions with MS Visual C++ Toolkit 2003 - msvccompiler-patch.txt (0/1)

P: n/a
The good news:

Along with Python-2.4 comes really good news to Windows users. Yes,
you now CAN build extension modules yourself using the SAME C++
compiler and linker Python is built with itself. Everything you need
is available at no costs (except download hassle and installation
time). Once your system is set up properly its just a matter of
running 'python setup.py build'. No longer waiting for someone else to
build binaries and a relief to those who kindly used to provide those
binaries.
The bad news:

It's not easy to get there! Gathering information and packages and
finding out about all the little steps and stumbling blocks may be
very time consuming.
Relax:

Here's (much of) what you need to know. With this document I'm trying
to provide the information I would have needed. It's sort of a step by
step instruction and contains some details. Goal: Compile Python-2.4
extensions with the freely available "Microsoft Visual C++ Toolkit
2003". This document DOES NOT address the issue of compiling Python
itself.

The example chosen to build is the "MySQLdb" module with its
_mysql.pyd extension.

Go and check out Mike Fletcher's page first at

http://www.vrplumber.com/programming/mstoolkit/

It's his work (and the need to get a working MySQLdb for Python-2.4 of
course ;-) that motivated me to start and spend some hours. Without
Mike's page I wouldn't even have tried. Thank you so much! Now I'm
prepared.

Two things first - not to forget:

(1) In contrast to what Mike writes I had to use a different registry
key (see step 9)

(2) I don't now what people mean when talking about "msvcr71.lib".
There's no such file on my machine. BUT there IS a "msvcrt.lib" a
directory "C:\Programme\Microsoft Visual Studio .NET 2003\Vc7\lib". I
guess the ".\Vc7\." indicates we are talking of the expected lib.
Correct? I don't know.

Attention - long lines:
Watch out for long lines that may have gotten split in this posting!

See if you've got something already. These are important directories
we're looking for. When "echo %ProgramFiles%" gives "C:\Programme"
there should be:

C:\Programme\Microsoft Platform SDK for Windows XP SP2\Include
C:\Programme\Microsoft Platform SDK for Windows XP SP2\Lib
C:\Programme\Microsoft Visual C++ Toolkit 2003\bin
C:\Programme\Microsoft Visual Studio .NET 2003\Vc7\lib
C:\Programme\Microsoft.NET\SDK\v1.1\include
C:\Programme\Microsoft.NET\SDK\v1.1\Lib
Step By Step
============

1. Set up a Windows machine
1.1 Install Windows XP (Prof. or Home doesn't matter)
1.2 Apply Service Pack 2 (WinXP SP2)
1.3 Perform a Windows update at
http://windowsupdate.microsoft.com
to get the latest patches.
2. Get some tools

This is optional - you don't have to. But it may be instructive and
helpful. And - I would say - fun.

2.1 Get "Windows Resource Kits"
Brings some useful tools like "CMD Prompt Here" to the windows
explorer context menu or "robocopy.exe" - a copy tools really worth to
look at. Seems MS developers need good tools too ...

http://www.microsoft.com/windows/reskits/default.asp

The website says:
"""
Windows Server 2003 Resource Kit Tools
The Resource Kit Tools are a set of software tools for network
administrators, developers, and power users to manage TCP/IP,
networks, the registry, security, remote administration,
configuration, batch files, and many other areas of the Windows Server
2003 operating system. To install the tools, download and run
Rktools.exe.
"""
(rktools.exe, 12.337.752 bytes).

2.2 Add some Unix to Windows. Goto

http://unxutils.sourceforge.net/

and get UnxUtils.zip (4.125 kB)
"""
Here are some ports of common GNU utilities to native Win32. In this
context, native means the executables do only depend on the Microsoft
C-runtime (msvcrt.dll) and not an emulation layer like that provided
by Cygwin tools.
"""
For example, this gives you "diff.exe" and "patch.exe".

3. Get and install the the .NET Framework 1.1.

It's available at no costs from the MicroSoft website. Use Google to
find the download page.

3.1 Install the ".NET Framework 1.1 runtime" first (dotnetfx.exe,
24.277.024 bytes)

3.2 Install the .NET Framework 1.1 SDK" second. (setup(sdk1.1).exe,
117.062.664 bytes).

Attention, observation:

This step should indeed create the badly needed

C:\Programme\Microsoft Visual Studio .NET 2003\Vc7\lib

folder. It didn't on my machine until I first completely deinstalled
on existing installation of .NET Framework 1.1 runtime AND .NET
Framework 1.1 SDK and then resumed with steps 3.1 and 3.2. Only
Redmond will know why - and I doubt it.

3.3 Perform a Windows update at
http://windowsupdate.microsoft.com
to get the latest patches of the .NET Framework.

4. Platform SDK

Get and install the platform SDK "Windows XP SP2 SDK" (about
266.000.00 bytes) from
http://www.microsoft.com/msdownload/...ULLInstall.htm

You better download the whole stuff (several .CAB-files) first instead
of trying to install online. This may be too risky.

5. Get and install free "Visual C++ Toolkit 2003"

This is the C++ compiler and linker. Use Google to find the "Microsoft
VisualC++ Toolkit 2003" download page and get "VCToolkitSetup.exe"
(32.948.392 bytes).

The website says:
"""
Overview
The Microsoft Visual C++ Toolkit 2003 includes the core tools
developers need to compile and link C++-based applications for Windows
and the .NET Common Language

Runtime:
Microsoft C/C++ Optimizing Compiler and Linker. These are the same
compiler and linker that ship with Visual Studio .NET 2003
Professional!

C Runtime Library and the C++ Standard Library, including the Standard
Template Library. These are the same static-link libraries included
with Visual Studio.

Microsoft .NET Framework Common Language Runtime. Visual C++ can
optionally build applications that target the Common LanguageRuntime
(CLR).

Sample code. The toolkit includes four samples designed to showcase
the powerful new features of the 2003 version, including new
optimization capabilities, features to improve code-security and
robustness, enhanced ISO C++ standards support, and the ability to use
the .NET Framework library and target the CLR.
"""
6. Be sure that all components are up to date and do a Windows update
at
http://windowsupdate.microsoft.com
7. Install Python-2.4 to C:\Python24
8. Modify the Python distutils

Make a backup of C:\Python24\Lib\distutils\msvccompiler.py and apply
the patches as described in

http://www.vrplumber.com/programming/mstoolkit/

to this file. You can do this by hand or use "patch.exe" which you've
gotten if you did follow step 2.2. There should be a file
"msvccompiler-patch.txt" appended to this message.

patch.exe msvccompiler.py msvccompiler-patch.txt

should succeed with no error.

9. Verify registry key of platform SDK

Start "regedit.exe" from the commandline and check for the registry
key of the platform SKD.

ON MY MACHINE and in my case it is different from the one listed on
http://www.vrplumber.com/programming/mstoolkit/

It is
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MicrosoftSDK \InstalledSDKs\B44C7E10-89BD-4A32-A3BF-D9D0BC4C9A05

and it is not
....\63DADB24-DC99-45EB-A748-EC93AB8A7497.

Find out about the correct key in your case and adjust the crucial
line 133 in msvccompiler.py:

freeSDK=r"SOFTWARE\Microsoft\MicrosoftSDK\Installe dSDKs\B44C7E10-89BD-4A32-A3BF-D9D0BC4C9A05"

If the key is wrong, you may encounter something like:

"""
C:\> python setup.py build
running build
running build_ext
error: Python was built with version 7.1 of Visual Studio, and
extensions need to be built with the same version of the ompiler, but
it isn't installed.
"""

10. Create a script file "pathsetup.bat" hat helps you set up the
necessary paths. We don't need to add the Python24 lib and include
paths as the distutils will take care for this automatically.

++++++++++++++++
REM START of pathsetup.bat, mb, 2004-12-12

REM add Python-2.4 to path
set Path=C:\Python24;%Path%

REM add the compiler path
set Path=C:\Programme\Microsoft Visual C++ Toolkit 2003\bin;%Path%

REM setup include paths
set Include=%ProgramFiles%\Microsoft Platform SDK for Windows XP
SP2\Include;%Include%
set Include=%ProgramFiles%\Microsoft Visual C++ Toolkit
2003\include;%Include%

REM setup Lib paths
set Lib=C:\Programme\Microsoft Platform SDK for Windows XP
SP2\Lib;%Lib%
set Lib=C:\Programme\Microsoft Visual C++ Toolkit 2003\lib;%Lib%
set Lib=C:\Programme\Microsoft Visual Studio .NET 2003\Vc7\lib;%xxLib%

REM END of pathsetup.bat
++++++++++++++++

11. Source distribution of Python module

Get and unpack the source distribution of a module you want to build.
In our case get "MySQL-python-1.1.7.tar.gz" from
http://sourceforge.net/projects/mysql-python or

http://sourceforge.net/project/showf...ckage_id=34790
und unpack to ...\somehwere\MySQL-python-1.1.7
11.1 Read README and setup.py

Is there a README in "MySQL-python-1.1.7\"? Yes, there is! It tells
you to get additional header and library files from the MySQL website.
Looking for version 4.x at

http://www.mysql.com/downloads

we'll follow version 4.x and be taken to

http://dev.mysql.com/downloads/mysql/4.0.html

Locate "Windows downloads (platform notes)" on that page and download
the binary distribution ("Windows (x86) Without installer (unzip in
C:\) 4.0.22 20.8M"). The file is "mysql-4.0.22-win-noinstall.zip"
(21.346 kB).

Since we're not interested in running MySQL but only in building the
extension we unzip into ...\MySQL-python-1.1.7\mysql-4.0.22-win

11.2
Make sure the MySQL header and library files are accessible by the C++
compiler and linker. The location is hardcoded in 'setup.py' of the
MySQL-python-1.1.7 module.

Search for "win32" in 'setup.py' and change references to
'C:\myslq\...' to '.\mysql-4.0.22-win

### >>>>>> schnipp >>>>>>>
# setup.py of MySQL-python-1.1.7

elif sys.platform == "win32": # Ugh
include_dirs = [r'.\mysql-4.0.22-win\include']
library_dirs = [r'.\mysql-4.0.22-win\lib\opt']
extra_objects = [r'.\mysql-4.0.22-win\lib\opt\mysqlclient.lib']

### <<<<<< schnapp <<<<<<<

12.
Relax. Take a breath. We're ready to go :-)

12.1
Open a command window.

12.2
Run "pathsetup.bat" to setup the environment.

12.3 Change directory and step into ...\MySQL-python-1.1.7, that is,
into the root dir of the module you want to build - the dir that
contains the "setup.py" file.

12.4 Start building the module including the extension using distutils
with the command:

python setup.py build
13.
There aren't fatal errors? Congratulations, you made it! Why not
install then:

python setup.py install

should do the trick. (Yes, I had two or three warnings. But they don't
seem to be important).
14. Earn what you deserve

Start Python and import:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
import MySQLdb
<---------- imported! no error here!


Yes, it does import :-) And it does work :-) And _mysql.pyd is similar
in size to those versions I had before (about 185 kB) :-)

Life goes on. A different time another thread I'm going to ask about
the warnings I got with building MySQLdb ...
mb - Martin

Jul 18 '05 #1
Share this Question
Share on Google+
6 Replies


P: n/a
Cool.... it's nice to see these working. I followed Mike Fletchers
instructions and they worked fine for me.

I used mingw to get the patch tool.
I did the core SDK download by doing a 'full download' (all 13 CAB
files and extraction bat separately) and a local install. This is about
a 350 odd meg download and some people have had issues with the MS
installer.

I've used it to build PyCrypto 2.0 successfully.

Anyway - thanks for the info.

Regards,
Fuzzy
http://www.voidspace.org.uk/atlantib...thonutils.html

Jul 18 '05 #2

P: n/a
Martin Bless wrote:
....
Two things first - not to forget:

(1) In contrast to what Mike writes I had to use a different registry
key (see step 9)

Which is expected (even noted on the page), particularly if you have a
different version of the SDKs. The keys in the patch were extracted
from an English Win2K Platform SDK. Don't know of any *good* way to
figure out the keys in a version-agnostic manner. Suggestions welcome...
(2) I don't now what people mean when talking about "msvcr71.lib".
There's no such file on my machine. BUT there IS a "msvcrt.lib" a
directory "C:\Programme\Microsoft Visual Studio .NET 2003\Vc7\lib". I
guess the ".\Vc7\." indicates we are talking of the expected lib.
Correct? I don't know.

This was sloppy on my part. msvcr71.dll is the DLL used for Visual
Studio 7.1, we want the .lib for that version of the DLL, but as you
note, it is actually named msvcrt.lib (same as the one for Visual Studio
6.0).
Get and install the platform SDK "Windows XP SP2 SDK" (about

....
It is
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MicrosoftSD K\InstalledSDKs\B44C7E10-89BD-4A32-A3BF-D9D0BC4C9A05

and it is not
...\63DADB24-DC99-45EB-A748-EC93AB8A7497.

Find out about the correct key in your case and adjust the crucial
line 133 in msvccompiler.py:

freeSDK=r"SOFTWARE\Microsoft\MicrosoftSDK\Install edSDKs\B44C7E10-89BD-4A32-A3BF-D9D0BC4C9A05"

....
Will have to figure out how to make this work across SDK versions somehow.

Thanks for the report,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

Jul 18 '05 #3

P: n/a
Having done steps 1 to 10, I tried building Numeric-23.6. And got the
following error-msg:

F:\install\Numeric-23.6>python setup.py build
running build
running build_py
running build_ext
building '_numpy' extension
D:\Programme\Microsoft Visual C++ Toolkit 2003\bin\cl.exe /c /nologo
/Ox /MD /W3
/GX /DNDEBUG -I/usr/include/atlas -IInclude -IPackages\FFT\Include
-IPackages\R
NG\Include -ID:\Python24\include -ID:\Python24\PC /TcSrc\_numpymodule.c
/Fobuild
\temp.win32-2.4\Release\Src\_numpymodule.obj
_numpymodule.c
d:\Python24\include\pyconfig.h(30) : fatal error C1083: Cannot open
include file
: 'io.h': No such file or directory
error: command '"D:\Programme\Microsoft Visual C++ Toolkit
2003\bin\cl.exe"' fai
led with exit status 2

why?
Regards
Wolfgang

Jul 18 '05 #4

P: n/a
wj****@web.de wrote:
Having done steps 1 to 10, I tried building Numeric-23.6. And got the
following error-msg:

F:\install\Numeric-23.6>python setup.py build
running build
running build_py
running build_ext
building '_numpy' extension
D:\Programme\Microsoft Visual C++ Toolkit 2003\bin\cl.exe /c /nologo
/Ox /MD /W3
/GX /DNDEBUG -I/usr/include/atlas -IInclude -IPackages\FFT\Include
-IPackages\R
NG\Include -ID:\Python24\include -ID:\Python24\PC /TcSrc\_numpymodule.c /Fobuild
\temp.win32-2.4\Release\Src\_numpymodule.obj
_numpymodule.c
d:\Python24\include\pyconfig.h(30) : fatal error C1083: Cannot open
include file
: 'io.h': No such file or directory
error: command '"D:\Programme\Microsoft Visual C++ Toolkit
2003\bin\cl.exe"' fai
led with exit status 2

why?
Regards
Wolfgang


If you look at the instructions on :
http://www.vrplumber.com/programming/mstoolkit/

It mentions that you probably need the .NET 2.0beta runtime and SDK.

Regards,

Fuzzy
http://www.voidspace.org.uk/atlantib...thonutils.html

Jul 18 '05 #5

P: n/a
....
d:\Python24\include\pyconfig.h(30) : fatal error C1083: Cannot open
include file
: 'io.h': No such file or directory
error: command '"D:\Programme\Microsoft Visual C++ Toolkit
2003\bin\cl.exe"' fai
led with exit status 2

why?


Under :
HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio
\7.1\VC\VC_OBJECTS_PLATFORM_INFO\Win32\Directories

You need to add the strings:
"Include Dirs"(path to toolkit \include; path to platform sdk \include)
"Library Dirs"(path to toolkit \lib; path to platform sdk \lib)
"Path Dirs" (path to toolkit \bin; path to platform sdk \bin)

Jul 18 '05 #6

P: n/a
Is it possible to compile extension modules on windows platforms using
MS Visual C++ Express 2005, or Visual Studio 8? If so, how would one
modify the instructions posted in this thread? [assume I also have
Toolkit 2003 installed, if necessary, for msvcr71.lib]

Marcus
Mike C. Fletcher wrote:
Martin Bless wrote:
...
Two things first - not to forget:

(1) In contrast to what Mike writes I had to use a different registrykey (see step 9)

Which is expected (even noted on the page), particularly if you have

a different version of the SDKs. The keys in the patch were extracted
from an English Win2K Platform SDK. Don't know of any *good* way to
figure out the keys in a version-agnostic manner. Suggestions welcome...
(2) I don't now what people mean when talking about "msvcr71.lib".
There's no such file on my machine. BUT there IS a "msvcrt.lib" a
directory "C:\Programme\Microsoft Visual Studio .NET 2003\Vc7\lib". Iguess the ".\Vc7\." indicates we are talking of the expected lib.
Correct? I don't know.

This was sloppy on my part. msvcr71.dll is the DLL used for Visual
Studio 7.1, we want the .lib for that version of the DLL, but as you
note, it is actually named msvcrt.lib (same as the one for Visual

Studio 6.0).
Get and install the platform SDK "Windows XP SP2 SDK" (about

...
It is


HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MicrosoftSD K\InstalledSDKs\B44C7E10-89BD-4A32-A3BF-D9D0BC4C9A05

and it is not
...\63DADB24-DC99-45EB-A748-EC93AB8A7497.

Find out about the correct key in your case and adjust the crucial
line 133 in msvccompiler.py:


freeSDK=r"SOFTWARE\Microsoft\MicrosoftSDK\Install edSDKs\B44C7E10-89BD-4A32-A3BF-D9D0BC4C9A05"

...
Will have to figure out how to make this work across SDK versions

somehow.
Thanks for the report,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com


Jul 18 '05 #7

This discussion thread is closed

Replies have been disabled for this discussion.