MS VC++ Toolkit 2003, where?

[Alex Martelli]

So, I thought I'd tool up to let me build and test Python extensions on
Windows (as well as Mac and Linux) -- I'm trying out Parallels
Workstation beta on my new Macbook Pro (and so far it seems to work very
well), I bought and installed a Win2000 Pro on it (since according to
the grapevine it works better than XP in various kinds of virtual
machines, and almost all SW supports w2k anyway -- I also found out that
one exception is the Google Pack, which does require XP), did all the
upgrades (why MS forces you to do 8+ rounds of download-install, from an
off-the-CD win2k SP2 to a fully upgraded win2k SP4, rather than offering
a single-round option, I just don't undersand!), got Python 2.4.3 --
phew, so far so good.

So, I go to <http://www.vrplumber.com/programming/mstoolkit/> and start
following the instructions, beginning with the download of Visual C++
Toolkit 2003... oops! I can't find that download any more at
<http://msdn.microsoft.com/visualc/vctoolkit2003/> -- instead, what's at
that URL is a page explaining that
"""
The Visual C++ Toolkit 2003 has been replaced by Visual C++ 2005 Express
Edition.
"""

Hmmm -- I can't build extensions for the standard build of Python 2.4.3
with VC++ 2005, can I? Express or other, they just use separate and
incompatible C-runtime libraries, I believe.

So -- does anybody know if the 2003-level Toolkit is STILL available for
download somewhere, or can suggest other (legal;-) ways in which I can
build and test Python extensions (with an optimizing compiler --
NON-optimizing ones are right out;-) w/o shelling out beeg moolah to MS?
So far my total investment is about $100 ($40 for Parallels Workstation,
which I preordered after briefly checking out the free-for-1-month beta,
since it worked so well; $60 for the cheapest legal CD+license of w2k I
could find around) -- and I'm not complaining, since the current setup
will still let me TEST many diverse things on Windows; to gain the
further ability of *compiling* Python extensions, I'm not averse to
spending a little more if needed, but certainly not as much as another
$100 (I do prefer free to cheap, but for example the $40 Parallels
product is well worth the money, to me, compared to the free 'Q',
because the latter, while fun to hack around with, was absorbing
countless hours of my time in many attempts to get everything working,
while Parallels essentially "just worked" and saved me LOTS of time).
Thanks in advance for any suggestions! Just to show that I'm an
oldstyle Usenet dynosaur: suggestions in private mail are also welcome
(but my real mailbox these days is at gmail, not as yahoo as the 'From'
header above says), and I will undertake to summarize and post any
suggestions thus received (unless the suggestions' authors request me to
not repost).
Alex

[David Rushby]

Alex Martelli wrote:

So -- does anybody know if the 2003-level Toolkit is STILL available for
download somewhere...

http://www.microsoft.com/downloads/d...displaylang=en

[Alex Martelli]

David Rushby <Da*********@gmail.com> wrote:

Alex Martelli wrote:

So -- does anybody know if the 2003-level Toolkit is STILL available for
download somewhere...

http://www.microsoft.com/downloads/d...272BE09D-40BB-
49FD-9CB0-4BFA122FA91B&displaylang=en

Great, thanks, just what I was looking for!!!
Alex

[AIM]

The site comes back with a message saying ...

"The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page."

Does anyone have any other ideas of where to look?

[Alex Martelli]

AIM <An***********@Hotmail.com> wrote:

The site comes back with a message saying ...

"The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page."

Does anyone have any other ideas of where to look?

As suggested to me by David Rushby 10 hours ago,

http://www.microsoft.com/downloads/d...72BE09D-40BB-4
9FD-9CB0-4BFA122FA91B&displaylang=en

does work.

But, there's another problem -- helloworld.c compiles fine, but linking
doesn't find MSVCRT.LIB -- apparently, the 2.0 .NET Framework SDK does
NOT include this file for x86 (there _are_ a couple of versions under
a64 and some other parallel directory, but I doubt they're good to use
with an x86!-). As the already-quoted crucial URL
<http://www.vrplumber.com/programming/mstoolkit/> mentions:

"""
# .NET Framework SDK Version 1.1

Provides the core msvcrt.lib for msvcr71.dll against which to link
your extensions. This is critically important, as without it you are
going to wind up linking against the wrong run-time and will see crashes
whenever a core object such as a file is shared across run-time
barriers.

The 2.0 beta also works according to reports, and may be required
to build Numpy 23.6 (whereas 23.1 seems to work without it).
"""

Whether the 2.0 beta also worked, the current definitive 2.0 seems NOT
to work.

Can anybody suggest where to get a Framework SDK 1.1., or any other
legal way to get "the core msvcrt.lib for msvcr71.dll against which to
link your extensions. This is critically important"...???
And, as an aside...:

It's sure an interesting reflection, that impecunious or thrifty
developers (ones not willing to shell out mucho $$$ to MS for a pro VS
2003) must go through such gyrations as these in order to be able to
build Python extensions on Windows. I'm sure my Windows-loving
colleagues in the PSF (who got several free copies of VS 2003 from
Microsoft, I believe -- at the time, I had zero Windows installations
and zero interest in Windows, so I didn't sign up for one) have fully
considered this recurring drama, and come to the decision of sticking
with VS 2003 (avoiding any free-as-in-beer compilers such as VS 2005 or
mingw) with thorough and wise deliberation.

The optimizer in the C compiler used to build Python in Windows *IS*
impressive: just today, I ran (and posted to it.comp.macintosh) pybench
on Python 2.4.3 on iBook G4 12", Macbook Pro 2.0 GHz, and Parallels
Workstation VM with Win2000 on the same Macbook -- while the Macbook is
4 times as fast as the iBook, the Windows version, despite the slight
overhead of running under Parallels' virtualization, is an impressive
12%+ _faster_ than the "native" MacOSX Python 2.4.3 (I'm not quite sure
about how good Parallels' virtualization IS, but even if it's as
impressive as a mere 3% overhead, this still means that the Windows
version of Python on identical HW must be at least 15% faster than the
MacOSX version, compiled with gcc).

I can see the stance that such a speedup warrants using Microsoft's
costly VS 2003 and imposing endless gyrations on developers trying to
get it on the cheap -- being one of the latter developers, I'm inclined
nevertheless to grumble, right now, of course;-)...
Alex

[Edward Elliott]

Alex Martelli wrote:

the Windows version, despite the slight
overhead of running under Parallels' virtualization, is an impressive
12%+ _faster_ than the "native" MacOSX Python 2.4.3 (I'm not quite sure
about how good Parallels' virtualization IS, but even if it's as
impressive as a mere 3% overhead, this still means that the Windows
version of Python on identical HW must be at least 15% faster than the
MacOSX version, compiled with gcc).

Can you post the complete benchmark results from both systems on the
Macbook? My understanding is that virtualization overhead is not a
constant factor like 3%. Unprivileged instructions run unmodified; only
privileged instructions like hardware accesses get trapped, adding
overhead. Any percentage figure advertised by the manufacturer is most
likely an average over some data set with both types of instructions.

The types of tests mentioned on the pybench site -- function calls,
lookups, exceptions, etc -- shouldn't incur any overhead from
virtualization. It would be helpful to see exactly which tests were faster
and by how much instead of an overall average result.

The differences may be due to factors other than compiler optimizations.
I've heard for instance the OS X virtual memory manager is not particularly
speedy for large data sets. That could give the windows pybench a win on
certain tests regardless of which compiler has better optimizations.

Then you've got issues of whether the compilers are really competing on
equal footing. Were similar options enabled for each build, does each
build represent the best possible optimizations or only the ones the
builder knew about, etc. I think Apple switched to the Intel compiler for
x86 macs, was python built with that or with gcc?

In short, your results are interesting but I'm not sure what to make of
them yet.

[Robert Kern]

Edward Elliott wrote:

I think Apple switched to the Intel compiler for
x86 macs, was python built with that or with gcc?

I'm pretty sure MacTel OS X still uses gcc 4 (although I think there is a beta
version of the Intel compiler available). All of the Python builds floating
around for it certainly use gcc.

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Martin v. Löwis]

Alex Martelli wrote:

Can anybody suggest where to get a Framework SDK 1.1., or any other
legal way to get "the core msvcrt.lib for msvcr71.dll against which to
link your extensions. This is critically important"...???
From

http://www.microsoft.com/downloads/d...displaylang=en
I'm sure my Windows-loving
colleagues in the PSF (who got several free copies of VS 2003 from
Microsoft, I believe -- at the time, I had zero Windows installations
and zero interest in Windows, so I didn't sign up for one) have fully
considered this recurring drama, and come to the decision of sticking
with VS 2003 (avoiding any free-as-in-beer compilers such as VS 2005 or
mingw) with thorough and wise deliberation.

Not sure whether this mark was meant to be sarcastic: but why you don't
want to use mingw to build extensions on Windows, I cannot understand.

Regards,
Martin

[Martin v. Löwis]

Alex Martelli wrote:

As suggested to me by David Rushby 10 hours ago,

http://www.microsoft.com/downloads/d...72BE09D-40BB-4
9FD-9CB0-4BFA122FA91B&displaylang=en

does work.

Can you please try this again: I'm also getting the error message
that AIM is getting.

Regards,
Martin

[JW]

On Sun, 23 Apr 2006 21:15:23 -0700, Alex Martelli wrote:

As suggested to me by David Rushby 10 hours ago,

... < huge URL snipped > ...
Alas, somehow this URL was split in two, and all the kings horses and all
the kings men can't seem to put it back together again (at least in my
browser). Could someone post a tinyurl?
And, as an aside...:

[Those] not willing to shell out mucho $$$ to MS for a pro VS
2003) must go through such gyrations as these in order to be able to
build Python extensions on Windows. I'm sure my Windows-loving
colleagues in the PSF (who got several free copies of VS 2003 from
Microsoft, I believe -- at the time, I had zero Windows installations
and zero interest in Windows, so I didn't sign up for one) have fully
considered this recurring drama, and come to the decision of sticking
with VS 2003 (avoiding any free-as-in-beer compilers such as VS 2005 or
mingw) with thorough and wise deliberation.

Well, so long as there's a way to get the requisite tools. I'm sure
these Windows-loving colleagues confirmed these "free" tools worked under
WINE, else how would us cheap programmers with Windows-less boxes avoid
sending $$$ to the Great Satan?

Jim Wilson
Gainesville, FL

[Alex Martelli]

JW <wi********@bellsouth.net> wrote:
...

As suggested to me by David Rushby 10 hours ago,

... < huge URL snipped > ...

Alas, somehow this URL was split in two, and all the kings horses and all
the kings men can't seem to put it back together again (at least in my
browser). Could someone post a tinyurl?

http://tinyurl.com/gv8wr
Alex

[Alex Martelli]

"Martin v. Löwis" <ma****@v.loewis.de> wrote:

Alex Martelli wrote:

Can anybody suggest where to get a Framework SDK 1.1., or any other
legal way to get "the core msvcrt.lib for msvcr71.dll against which to
link your extensions. This is critically important"...???

From
http://www.microsoft.com/downloads/d...9b3a2ca6-3647-
4070-9f41-a333c6b9181d&displaylang=en

Thanks! For the convenience of tinyurl-preferrers:

http://tinyurl.com/5flob

I'm sure my Windows-loving
colleagues in the PSF (who got several free copies of VS 2003 from
Microsoft, I believe -- at the time, I had zero Windows installations
and zero interest in Windows, so I didn't sign up for one) have fully
considered this recurring drama, and come to the decision of sticking
with VS 2003 (avoiding any free-as-in-beer compilers such as VS 2005 or
mingw) with thorough and wise deliberation.

Not sure whether this mark was meant to be sarcastic: but why you don't
want to use mingw to build extensions on Windows, I cannot understand.

Jocular, but not sarcastic. I have no problems using mingw if that's
what it takes -- the later instructions I saw were those suggesting the
Toolkit instead, so those are the ones I'm trying to follow. What's the
updated URL for the instructions about using mingw instead?
Thanks,

Alex

[Alex Martelli]

"Martin v. Löwis" <ma****@v.loewis.de> wrote:

Alex Martelli wrote:

As suggested to me by David Rushby 10 hours ago,

http://www.microsoft.com/downloads/d...72BE09D-40BB-4
9FD-9CB0-4BFA122FA91B&displaylang=en

does work.

Can you please try this again: I'm also getting the error message
that AIM is getting.

Try tinyurl http://tinyurl.com/gv8wr please.

I've also tinyurl'd your URL for the 1.1 SDK, to
http://tinyurl.com/5flob .

((I suspect the problem has to do with a limitation of 80
characters/line in NNTP messages, which my favorite newsreader enforces
unconditionally)).
Alex

[Alex Martelli]

Robert Kern <ro*********@gmail.com> wrote:

Edward Elliott wrote:

I think Apple switched to the Intel compiler for
x86 macs, was python built with that or with gcc?

I'm pretty sure MacTel OS X still uses gcc 4 (although I think there is a beta
version of the Intel compiler available). All of the Python builds floating
around for it certainly use gcc.

Apple's XCode still uses gcc. The intel compilers are out of beta, and
cost many hundreds of dollars per developer, besides NOT supporting
ObjectiveC, while XCode is still free (as in beer for the GUI parts etc,
as in speech for the underlying commandline tools, though I'm not quite
sure where I'd start looking for the latters' sources if I wanted
them;-).

Edward's request on this thread is eminently reasonable, but I don't
really have time to get and post all the detailed results of pybench
right now - I'll try to get to it this evening.
Alex

[Robert Kern]

Alex Martelli wrote:

Jocular, but not sarcastic. I have no problems using mingw if that's
what it takes -- the later instructions I saw were those suggesting the
Toolkit instead, so those are the ones I'm trying to follow. What's the
updated URL for the instructions about using mingw instead?

Install mingw (an large task in and of itself that I'm not going to go into
here, but look at http://www.mingw.org/) and make sure the bin/ directory is on
your PATH. You will have to edit the gcc specs file to replace "-lmsvcrt" with
"-lmsvcr71". Its filename is lib/mingw32/3.4.2/specs . After that, use the
--compiler=mingw32 option on build_ext when using distutils.

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Robert Kern]

Alex Martelli wrote:

Jocular, but not sarcastic. I have no problems using mingw if that's
what it takes -- the later instructions I saw were those suggesting the
Toolkit instead, so those are the ones I'm trying to follow. What's the
updated URL for the instructions about using mingw instead?

Oh, that's right, you need an import library for Python24.dll . If you build
numpy first, it will automatically build the import library for you. The
implementation is fairly simple if you would rather bodge a script together. The
driver is the function build_import_library() here:

http://svn.scipy.org/svn/numpy/trunk...32ccompiler.py

This uses the following module:

http://svn.scipy.org/svn/numpy/trunk...ils/lib2def.py

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Ron Adam]

Alex Martelli wrote:

"Martin v. Löwis" <ma****@v.loewis.de> wrote:

Alex Martelli wrote:

As suggested to me by David Rushby 10 hours ago,

http://www.microsoft.com/downloads/d...72BE09D-40BB-4
9FD-9CB0-4BFA122FA91B&displaylang=en

does work. Can you please try this again: I'm also getting the error message
that AIM is getting.

Try tinyurl http://tinyurl.com/gv8wr please.

I still get the following with the tinyurl link:

~~~
The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page.
~~~
Pasting the above tinyurl into firefox results in the following link.

http://www.microsoft.com/downloads/d...displaylang=en

Which appears to still be truncated. :-/
I've also tinyurl'd your URL for the 1.1 SDK, to
http://tinyurl.com/5flob .
This one works.
((I suspect the problem has to do with a limitation of 80
characters/line in NNTP messages, which my favorite newsreader enforces
unconditionally)).

Alex

[Martin v. Löwis]

Robert Kern wrote:

Oh, that's right, you need an import library for Python24.dll .

That shouldn't be a problem: that library is included with Python.

Regards,
Martin

[Robert Kern]

Martin v. Löwis wrote:

Robert Kern wrote:

Oh, that's right, you need an import library for Python24.dll .

That shouldn't be a problem: that library is included with Python.

For mingw, too? I.e. a .a not a .lib?

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Alex Martelli]

Ron Adam <rr*@ronadam.com> wrote:
...

I still get the following with the tinyurl link:

~~~
The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page.
~~~
Pasting the above tinyurl into firefox results in the following link.

http://www.microsoft.com/downloads/d...BE09D-40BB-4&d
isplaylang=en

Which appears to still be truncated. :-/

True, but here's the kicker: using the full URL, which I just dblchecked
from Rushby's original message as being (enclosing in <...>)...:

<http://www.microsoft.com/downloads/d...272BE09D-40BB-
49FD-9CB0-4BFA122FA91B&displaylang=en>

NOW gives me the same error page too. When Rushby suggested it
yesterday I immediately used it, and it allowed me to download the
Toolkit 2003 just fine -- so, I guess Microsoft must have killed that
possibility shortly thereafter (who said they can't move fast?-).

As of now, I don't know any more of a URL that's usable to download
this, therefore:-(
Alex

[Andrew Trevorrow]

Ron Adam wrote:

Try tinyurl http://tinyurl.com/gv8wr please.

I still get the following with the tinyurl link:

~~~
The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page.
~~~

Ditto for me. And like Ron, the URL (tiny or long) to the 1.1 SDK
*does* work.

This is really frustrating. I've tried different browsers (Safari and IE
on my Mac, IE on WinXP). I've tried emptying the browser cache.

I'm in Australia, so maybe it depends on where the request is coming from?
Any clues on this would be much appreciated.

Or maybe someone is willing to make their VCToolkitSetup.exe available
temporarily...

Andrew

[Alex Martelli]

Edward Elliott <no****@127.0.0.1> wrote:

Alex Martelli wrote:

the Windows version, despite the slight
overhead of running under Parallels' virtualization, is an impressive
12%+ _faster_ than the "native" MacOSX Python 2.4.3 (I'm not quite sure
about how good Parallels' virtualization IS, but even if it's as
impressive as a mere 3% overhead, this still means that the Windows
version of Python on identical HW must be at least 15% faster than the
MacOSX version, compiled with gcc).
Can you post the complete benchmark results from both systems on the
Macbook? My understanding is that virtualization overhead is not a

OK, I've placed on http://www.aleax.it/Python/ the files that pybench
writes (with the -file option) for each machines, the names are
onmbp.txt and onwin2k.txt -- just 20k each (I'm not sure their format is
documented, but I guess that, worst case, one just needs to study
pybench's sources).
builder knew about, etc. I think Apple switched to the Intel compiler for
x86 macs, was python built with that or with gcc?
The compiler Apple distributes freely is still gcc -- the intel compiler
(rumored to have better optimization) costs hundreds of dollars, so
Apple couldn't possibly distribute it for free with XCode.
In short, your results are interesting but I'm not sure what to make of
them yet.

Consider me available if you need some other tests and don't have other
easy access to OSX and Windows running on the same HW.
Alex

[Alex Martelli]

Andrew Trevorrow <an****@trevorrow.com> wrote:

Ron Adam wrote:

Try tinyurl http://tinyurl.com/gv8wr please.
I still get the following with the tinyurl link:

~~~
The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page.
~~~

Ditto for me. And like Ron, the URL (tiny or long) to the 1.1 SDK
*does* work.

This is really frustrating. I've tried different browsers (Safari and IE
on my Mac, IE on WinXP). I've tried emptying the browser cache.

I'm in Australia, so maybe it depends on where the request is coming from?
Any clues on this would be much appreciated.

Nope: it's stopped working for me, too.

Or maybe someone is willing to make their VCToolkitSetup.exe available
temporarily...

I suggest any such offers be made privately, since I'm pretty sure
they'd be illegal (at least in the US, dunno 'bout Oz law).
Alex

[Andrew Trevorrow]

al*****@yahoo.com (Alex Martelli) wrote:

Andrew Trevorrow <an****@trevorrow.com> wrote:

Or maybe someone is willing to make their VCToolkitSetup.exe available
temporarily...

I suggest any such offers be made privately, since I'm pretty sure
they'd be illegal (at least in the US, dunno 'bout Oz law).

Oh, absolutely -- private offers only please. If it helps ease the
conscience af any prospective benefactors, I *did* have a copy of
VCToolkitSetup.exe at one point. I downloaded it about a year ago
and installed it on Win2000 under Virtual PC, but later trashed
it without making a backup, so that'll teach me.

Failing any offers of help, I guess my options are:

- Wait for Parallels to allow importing VPC disk images (I asked
them about this but it's not going to happen soon).

- Extract all the VC++ 2003 stuff from my VPC Win2000 system and
copy it over to my Parallels system. Sounds like fun...

- Look on ebay for a cheap copy of Visual Studio with VC++ 2003.

- Download VC++ 2005 Express. I'm not planning to build extensions
so I guess I don't really have to use VC++ 2003.

Andrew

[Martin v. Löwis]

Robert Kern wrote:

Oh, that's right, you need an import library for Python24.dll .

That shouldn't be a problem: that library is included with Python.

For mingw, too? I.e. a .a not a .lib?

Right.

Martin

[Robert Kern]

Martin v. Löwis wrote:

Robert Kern wrote:

Oh, that's right, you need an import library for Python24.dll .

That shouldn't be a problem: that library is included with Python.

For mingw, too? I.e. a .a not a .lib?

Right.

Woohoo!

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Fredrik Lundh]

Robert Kern wrote:

Martin v. Löwis wrote:

Robert Kern wrote:

Oh, that's right, you need an import library for Python24.dll .

That shouldn't be a problem: that library is included with Python.

For mingw, too? I.e. a .a not a .lib?

last time I tinkered with mingw, it could link directly against the DLL
file. see the last two minutes in "mingw from scratch in 20 minutes"
post:

http://article.gmane.org/gmane.comp....general/388046

the MinGW FAQ says that you can use reimp in cases like this:

http://www.mingw.org/mingwfaq.shtml#faq-msvcdll

but that didn't seem to be necessary (I assume reimp or something
is now integrated in mingw, or maybe I have a magic computer...)

however, note that the FAQ entry says that you can use an existing
LIB file as well, so Python's standard import library should work.

</F>

[Ron Adam]

Alex Martelli wrote:

Ron Adam <rr*@ronadam.com> wrote:
...

I still get the following with the tinyurl link:

~~~
The download you requested is unavailable. If you continue to see this
message when trying to access this download, go to the "Search for a
Download" area on the Download Center home page.
~~~
Pasting the above tinyurl into firefox results in the following link.

http://www.microsoft.com/downloads/d...BE09D-40BB-4&d
isplaylang=en

Which appears to still be truncated. :-/
True, but here's the kicker: using the full URL, which I just dblchecked
from Rushby's original message as being (enclosing in <...>)...:

Yes, I tried the full url also, but was hoping I did something wrong.
Sigh... guess not.
<http://www.microsoft.com/downloads/d...272BE09D-40BB-
49FD-9CB0-4BFA122FA91B&displaylang=en>

NOW gives me the same error page too. When Rushby suggested it
yesterday I immediately used it, and it allowed me to download the
Toolkit 2003 just fine -- so, I guess Microsoft must have killed that
possibility shortly thereafter (who said they can't move fast?-).

As of now, I don't know any more of a URL that's usable to download
this, therefore:-(

And with a new version of windows due at the end of this year, it's
likely those who upgrade to Vista, will also need to upgrade to visual
studio 2005 or later.

I've seen this pattern before. :-/

Ron

[Edward Elliott]

Alex Martelli wrote:

OK, I've placed on http://www.aleax.it/Python/ the files that pybench
writes (with the -file option) for each machines, the names are
onmbp.txt and onwin2k.txt -- just 20k each (I'm not sure their format is
documented, but I guess that, worst case, one just needs to study
pybench's sources).
Nice, thank you. The files just contain pickled data, but you need pybench
installed to unpickle the classes. I used the command
pybench -s win2k.txt -c osx.txt
to compare the data sets. Here are the results. Left two columns are
Windows times, right column is change from OS X time. I resorted the tests
by percentage change (win2k slowest->fastest) to provide a better picture
of what's going on:

PYBENCH 1.0
Benchmark: /home/ed/python-w2k.txt (rounds=10, warp=20)
measured against: /home/ed/python-osx.txt (rounds=10, warp=20)
Tests: per run per oper. diff
------------------------------------------------------------------------
(slower on win2k)
StringMappings: 309.33 ms 2.46 us +112.53%
ConcatUnicode: 157.47 ms 1.05 us +77.43%
ConcatStrings: 103.57 ms 0.69 us +44.86%
DictWithFloatKeys: 169.10 ms 0.28 us +21.52%
UnicodeProperties: 86.00 ms 0.43 us +6.69%
PythonFunctionCalls: 85.83 ms 0.52 us +1.22%
UnicodeSlicing: 85.64 ms 0.49 us +0.16%
(faster on win2k)
IfThenElse: 70.38 ms 0.10 us -2.45%
SmallLists: 76.98 ms 0.30 us -4.44%
CompareUnicode: 85.22 ms 0.23 us -4.72%
StringPredicates: 87.25 ms 0.31 us -6.13%
NestedForLoops: 38.47 ms 0.11 us -8.07%
CompareInternedStrings: 59.12 ms 0.12 us -8.12%
SimpleListManipulation: 41.93 ms 0.16 us -8.55%
CreateStringsWithConcat: 35.02 ms 0.18 us -9.16%
TryRaiseExcept: 49.62 ms 3.31 us -9.53%
StringSlicing: 63.61 ms 0.36 us -9.77%
SmallTuples: 68.65 ms 0.29 us -12.77%
CompareFloatsIntegers: 72.31 ms 0.16 us -13.72%
Recursion: 45.88 ms 3.67 us -16.20%
CompareFloats: 51.26 ms 0.11 us -17.05%
CompareStrings: 76.14 ms 0.15 us -17.51%
PythonMethodCalls: 60.74 ms 0.81 us -17.75%
CreateUnicodeWithConcat: 90.22 ms 0.45 us -17.76%
BuiltinFunctionCalls: 39.38 ms 0.31 us -17.79%
SimpleFloatArithmetic: 57.72 ms 0.10 us -19.44%
SecondImport: 29.59 ms 1.18 us -19.71%
CreateInstances: 63.02 ms 1.50 us -21.08%
SimpleIntFloatArithmetic: 55.92 ms 0.08 us -21.78%
SimpleComplexArithmetic: 42.40 ms 0.19 us -22.84%
BuiltinMethodLookup: 78.26 ms 0.15 us -24.54%
UnicodePredicates: 56.17 ms 0.25 us -24.75%
SpecialInstanceAttribute: 117.49 ms 0.20 us -24.97%
DictWithStringKeys: 62.30 ms 0.10 us -26.18%
TupleSlicing: 73.63 ms 0.70 us -26.34%
NormalInstanceAttribute: 65.19 ms 0.11 us -26.50%
UnicodeMappings: 65.20 ms 3.62 us -27.68%
SecondPackageImport: 31.25 ms 1.25 us -28.74%
SpecialClassAttribute: 65.94 ms 0.11 us -30.30%
ListSlicing: 50.75 ms 14.50 us -31.37%
SecondSubmoduleImport: 39.39 ms 1.58 us -31.98%
SimpleDictManipulation: 37.88 ms 0.13 us -32.24%
NormalClassAttribute: 66.37 ms 0.11 us -33.40%
DictCreation: 36.66 ms 0.24 us -34.25%
DictWithIntegerKeys: 54.73 ms 0.09 us -35.76%
SimpleLongArithmetic: 27.03 ms 0.16 us -39.80%
TryExcept: 62.54 ms 0.04 us -42.17%
SimpleIntegerArithmetic: 42.88 ms 0.06 us -42.52%
CompareIntegers: 42.81 ms 0.05 us -45.54%
ForLoops: 25.72 ms 2.57 us -53.99%
CompareLongs: 25.45 ms 0.06 us -63.04%
------------------------------------------------------------------------
Average round time: 3847.99 ms -12.61%
None of these tests look to me like anything virtualization would affect.
Beyond that, I'm not sure what to make of it. The fastest gains on win2k
(>39%) seem to come from branching and arithmetic tests. Maybe the ms
compiler has significantly better branch prediction (might explain why
ForLoops and TryExcept have such large gains while NestedForLoops and
TryRaiseExcept are more modest). I can't postulate why some tests show20% gains, like the *Slicing and *InstanceAttributes ones. No idea why the first group of tests came out faster on OS X.

I really don't have enough platform-specific knowledge to make sense of
what's going on. Maybe someone with more experience can comment? There
may just be too many factors at play to tease out causal relationships.

The compiler Apple distributes freely is still gcc -- the intel compiler
(rumored to have better optimization) costs hundreds of dollars, so
Apple couldn't possibly distribute it for free with XCode.
I know ICC is commercial and thus not in XCode, but I thought maybe Apple
was using ICC internally to compile some parts of OSX. I have no firm
basis for that belief, it's probably just a rumor I picked up somewhere.

Consider me available if you need some other tests and don't have other
easy access to OSX and Windows running on the same HW.

Thanks.

[Martin v. Löwis]

Fredrik Lundh wrote:

however, note that the FAQ entry says that you can use an existing
LIB file as well, so Python's standard import library should work.

Right. MingW (GNU ld) was (apparently) changed to support that shortly
after I started including libpython24.a files with the Windows
distributions.

Regards,
Martin

[Ross Ridge]

Martin v. Löwis wrote:

Right. MingW (GNU ld) was (apparently) changed to support that shortly
after I started including libpython24.a files with the Windows
distributions.

A bug in binutils support for short import library records was fixed
about year ago. You need to use MinGW binutils 2.16.91 or newer if you
want to link with any of the ".lib" files included with Python 2.4.

Ross Ridge

[Brian Elmegaard]

Robert Kern <ro*********@gmail.com> writes:

Martin v. Löwis wrote:

Robert Kern wrote:

Oh, that's right, you need an import library for Python24.dll .

That shouldn't be a problem: that library is included with Python.

For mingw, too? I.e. a .a not a .lib?

It is possible to load a .dll in mingw.

--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk

[sturlamolden]

I believe MinGW can link .lib C libraries files from Visual Studio. But
there are no .a for Python24.dll as far as I can tell.

[sturlamolden]

Alex Martelli wrote:

Provides the core msvcrt.lib for msvcr71.dll against which to link
your extensions. This is critically important, as without it you are
going to wind up linking against the wrong run-time and will see crashes
whenever a core object such as a file is shared across run-time
barriers.

You can find msvcr71.dll in the same directory as Python.

The problem is that you cannot redistribute msvcr71.dll unless you by a
copy of Visual Studio 2003 or install VC++ Toolkit 2003. As far as I
can tell, the .NET SDK license does not give you permission to
redistribute msvcr71.dll. So if you are going to use Py2Exe, this is a
dead end. But if you are just going to build a Python extension, you
don't need to redistribute the DLL (it's already in Python). In that
case you can use MinGW insted. Just make sure MinGW links with the
correct CRT. That is, open

c:\mingw\lib\gcc\mingw32\3.4.2\specs

in an editor and change "-lmsvcrt" to "-lmsvcr71"

There is a second advantage with this. MinGW is an optimizing compiler.
The C/C++ compiler you get from the .NET SDK is not. There is a "Visual
C++ 2005 Express" edition which has an optimizing compiler. But it
links yet another version of the CRT, msvcr80.dll, and does not give
you permission to redistribute msvcr71.dll.

[Brian Elmegaard]

"sturlamolden" <st**********@yahoo.no> writes:

I believe MinGW can link .lib C libraries files from Visual Studio. But
there are no .a for Python24.dll as far as I can tell.

But afaik you don't need one.
--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk

[Gerhard Häring]

Brian Elmegaard wrote:

"sturlamolden" <st**********@yahoo.no> writes:

I believe MinGW can link .lib C libraries files from Visual Studio. But
there are no .a for Python24.dll as far as I can tell.

But afaik you don't need one.

Actually, a libpython24.a file was added in Python 2.4.1. The original
2.4.0 release didn't include one.

AFAIR recent MINGW releases don't need this file any longer:
http://mail.python.org/pipermail/pyt...er/057693.html

-- Gerhard

[Chris Mellon]

On 27 Apr 2006 12:06:44 -0700, sturlamolden <st**********@yahoo.no> wrote:

Alex Martelli wrote:

Provides the core msvcrt.lib for msvcr71.dll against which to link
your extensions. This is critically important, as without it you are
going to wind up linking against the wrong run-time and will see crashes
whenever a core object such as a file is shared across run-time
barriers.
You can find msvcr71.dll in the same directory as Python.

The problem is that you cannot redistribute msvcr71.dll unless you by a
copy of Visual Studio 2003 or install VC++ Toolkit 2003. As far as I
can tell, the .NET SDK license does not give you permission to
redistribute msvcr71.dll. So if you are going to use Py2Exe, this is a
dead end. But if you are just going to build a Python extension, you
don't need to redistribute the DLL (it's already in Python). In that
case you can use MinGW insted. Just make sure MinGW links with the
correct CRT. That is, open

c:\mingw\lib\gcc\mingw32\3.4.2\specs

in an editor and change "-lmsvcrt" to "-lmsvcr71"

There is a second advantage with this. MinGW is an optimizing compiler.
The C/C++ compiler you get from the .NET SDK is not.

This is untrue - the MSVC compiler in the VS 2003 Toolkit is exactly
the same compiler that ships with real visual studio, and does
excellent optimization. Modulo all the extremely correct comments in
this thread about how useless it is to make comments about the
optimization capabilities of a compiler, I find that the VS 2003
compiler generally generates faster and (often much) smaller code than
GCC/mingw
There is a "Visual
C++ 2005 Express" edition which has an optimizing compiler. But it
links yet another version of the CRT, msvcr80.dll, and does not give
you permission to redistribute msvcr71.dll.

There are numerous distribution issues with the VS 2005 runtimes (I
don't want to get into them now) besides the legal ones, but it's
useless for building extension modules unless you also re-build Python
(and then your Python can't use any other extension modules). It's
workable for people embedding Python and probably not anyone else.
--
http://mail.python.org/mailman/listinfo/python-list