By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,715 Members | 1,815 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,715 IT Pros & Developers. It's quick & easy.

MinGW and Python

P: n/a
Is there any specific reason for not using MinGW to build the official
distribution of Python for Win32?
A quick Google search did not reveal the answer to my question. If a
link is available, please post it.

Best Regards,
Srijit

Apr 24 '06 #1
Share this Question
Share on Google+
66 Replies


P: n/a
Srijit Kumar Bhadra wrote:
Is there any specific reason for not using MinGW to build the official
distribution of Python for Win32?


What could be the reasons to use MinGW?

As for reasons not to do that:
- there is no build process available to do that
- people building extensions to Python must be able to do so with
Microsoft C++, since some of these extensions are written using MFC.
- developing Python itself in Visual Studio is quite convenient; in
particular, the debugger works "better" than gdb.

Regards,
Martin
Apr 24 '06 #2

P: n/a
> Is there any specific reason for not using MinGW to build the official
distribution of Python for Win32?
A quick Google search did not reveal the answer to my question. If a
link is available, please post it.

You may look at this thread:

* E02 - Support for MinGW Open Source Compiler
http://tinyurl.com/lxfsz

There was a big polemic and it is really long, but there are some useful
posts there.

Regards,
Josef

Apr 24 '06 #3

P: n/a
Martin v. Löwis wrote:
Srijit Kumar Bhadra wrote:
Is there any specific reason for not using MinGW to build the official
distribution of Python for Win32?


What could be the reasons to use MinGW?

As for reasons not to do that:
- there is no build process available to do that
- people building extensions to Python must be able to do so with
Microsoft C++, since some of these extensions are written using MFC.
- developing Python itself in Visual Studio is quite convenient; in
particular, the debugger works "better" than gdb.


- gcc does not optimize particularly well.

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Apr 24 '06 #4

P: n/a
Robert Kern <ro*********@gmail.com> writes:
- gcc does not optimize particularly well.


But well enough for other platforms.

--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk
Apr 24 '06 #5

P: n/a
Brian Elmegaard wrote:
Robert Kern <ro*********@gmail.com> writes:
- gcc does not optimize particularly well.


But well enough for other platforms.


Well, it tends to optimize just as poorly for other platforms, too. It's just
frequently the only compiler widely available on those platforms. The goal of
the gcc project is to provide a portable compiler, not one that generates the
best code for any given platform. And in that goal, it succeeds remarkably well.

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Apr 24 '06 #6

P: n/a
Robert Kern <ro*********@gmail.com> writes:
the gcc project is to provide a portable compiler, not one that
generates the best code for any given platform. And in that goal, it
succeeds remarkably well.


Will a python program be slower on the same machine running windows
compared to linux?

What I don't understand is that it is not possible to distribute a
python compiled with gcc for windows. The main reason I saw in this
thread is that python uses mfc. So python requires api access, I
guess.

Once I asked about distutils here. The answer was that I had access to
the source so I could just extend it. After messing around I found I
couldn't because I don't have msvc.
--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk
Apr 25 '06 #7

P: n/a
Brian Elmegaard wrote:
What I don't understand is that it is not possible to distribute a
python compiled with gcc for windows. The main reason I saw in this
thread is that python uses mfc. So python requires api access, I
guess.
You misunderstood. Python does not use MFC. PythonWin (for example)
does. That has a lot of implications which are hard to understand
if you are not familiar with Windows programming and the C library
mess on Windows.

It would certainly be possible to distribute a gcc-compiled python.
However, what is the point in doing so? Cygwin already includes
a gcc-compiled Python, for Windows:

http://cygwin.com/packages/python/
Once I asked about distutils here. The answer was that I had access to
the source so I could just extend it. After messing around I found I
couldn't because I don't have msvc.


That is simply not true. You can build the entire Python interpreter
with Cygwin (but you don't need to, because there is a precompiled
version), and you can build extensions for the python.org binary
using MingW.

Regards,
Martin
Apr 25 '06 #8

P: n/a
Brian Elmegaard wrote:
Robert Kern <ro*********@gmail.com> writes:
the gcc project is to provide a portable compiler, not one that
generates the best code for any given platform. And in that goal, it
succeeds remarkably well.
Will a python program be slower on the same machine running windows
compared to linux?


Dunno. Depends on the machine. Depends on the program. Depends on how the
interpreter and any extension modules and underlying libraries were built.
Depends on which Linux and which Windows.

I'm sorry, but your question is a non sequitur. I don't understand its relevance
to this thread.
What I don't understand is that it is not possible to distribute a
python compiled with gcc for windows. The main reason I saw in this
thread is that python uses mfc. So python requires api access, I
guess.
It is possible. People have done it.

http://jove.prohosting.com/iwave/ipython/pyMinGW.html
Once I asked about distutils here.
Okay. Again, what's the relevance here?
The answer was that I had access to
the source so I could just extend it. After messing around I found I
couldn't because I don't have msvc.


I'm not sure why this matters since distutils is pure Python.

If you meant writing extension modules for Python instead of extending
distutils, then you're in luck! You can indeed build extension modules with
mingw for the standard Python distribution. Please see my post in the thread "MS
VC++ Toolkit 2003, where?".

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Apr 25 '06 #9

P: n/a
Brian Elmegaard wrote
the gcc project is to provide a portable compiler, not one that
generates the best code for any given platform. And in that goal, it
succeeds remarkably well.


Will a python program be slower on the same machine running windows
compared to linux?


a better optimizer usually results in programs that run faster, not slower.

</F>

Apr 25 '06 #10

P: n/a
"Martin v. Lwis" <ma****@v.loewis.de> writes:
It would certainly be possible to distribute a gcc-compiled python.
However, what is the point in doing so? Cygwin already includes
a gcc-compiled Python, for Windows:
Interesting.
That is simply not true.
Actually, you answered me then too. I misunderstood after reading
http://sebsauvage.net/python/mingw.html.

Is the information on that page not correct? Has it never been?
You can build the entire Python interpreter with Cygwin (but you
don't need to, because there is a precompiled version), and you can
build extensions for the python.org binary using MingW.


Great, then I tend to agree that there is no reason for building it
with mingw.
--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk
Apr 25 '06 #11

P: n/a
Robert Kern <ro*********@gmail.com> writes:
If you meant writing extension modules for Python instead of extending
distutils,
I thought about extending distutils to make non-python installers. I
may have misunderstood the answers I got.
http://groups.google.com/group/comp....f1aea19aa187b6

mingw for the standard Python distribution. Please see my post in the thread "MS
VC++ Toolkit 2003, where?".


Thanks, I will.
--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk
Apr 25 '06 #12

P: n/a
"Fredrik Lundh" <fr*****@pythonware.com> writes:
a better optimizer usually results in programs that run faster, not slower.


Got it the wrong after some editing ;-(
--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk
Apr 25 '06 #13

P: n/a

Martin v. Lwis wrote:
- there is no build process available to do that
In MSYS:

$ ./configure --prefix=/c/mingw
$ make
$ make install

This should be obvious to any with Unix experience.

MinGW actually distribute precompiled Python binaries as well (in
MSYS-DTK).
- people building extensions to Python must be able to do so with
Microsoft C++, since some of these extensions are written using MFC.
MinGW can compile MFC. Download Windows Platform SDK and you get the
MFC source.
- developing Python itself in Visual Studio is quite convenient; in
particular, the debugger works "better" than gdb.


Eclipse is an excellent IDE that works with MinGW.

Apr 25 '06 #14

P: n/a

Robert Kern wrote:
- gcc does not optimize particularly well.


That is beyond BS. The more recent gcc releases optimize as well as any
commercial compiler. GCC 4 may even optimize better than MSVC.

GCC is the compiler used to build the Linux kernel and MacOSX. If it
can deal with this I say it can handle Python as well. My guess is you
will not notice any performance difference for the Python interpreter.
Doesn't Python run well on Linux? GCC is used to build Python on Linux.
MinGW is GCC. MinGW on Windows creates the same binary code as GCC on
Linux.

Apr 25 '06 #15

P: n/a

Robert Kern wrote:
Dunno. Depends on the machine. Depends on the program. Depends on how the
interpreter and any extension modules and underlying libraries were built.
Depends on which Linux and which Windows.

I'm sorry, but your question is a non sequitur. I don't understand its relevance
to this thread.

The relevance: Python is built with GCC on Linux. Do you or do you not
see a performance hit on Linux?

MinGW is GCC. Will you get a performance hit when building Python with
MinGW?

Apr 25 '06 #16

P: n/a

I forgot to mention that C libraries built with Visual C++ and MinGW
are binary compatible. MinGW can link libararies and object files from
Visual C++. Although Python may be build with Visual C++, you can still
compile and link your C extensions with MinGW.

Apr 25 '06 #17

P: n/a
sturlamolden <st**********@yahoo.no> wrote:
Robert Kern wrote:
Dunno. Depends on the machine. Depends on the program. Depends on how
the interpreter and any extension modules and underlying libraries were
built. Depends on which Linux and which Windows.

I'm sorry, but your question is a non sequitur. I don't understand its
relevance to this thread.


The relevance: Python is built with GCC on Linux. Do you or do you not
see a performance hit on Linux?

MinGW is GCC. Will you get a performance hit when building Python with
MinGW?


I cannot predict this, though it would be great if somebody who does
have both VS2003 and mingw could give it a try.

What I did just post on another thread over the last couple of days is
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3
under Win2000 under Parallels Workstation beta, compared to 2.4.3
Universal directly on MacOSX -- the standard build of 2.4.3 in either
cause, i.e., the one built with MS compilers on Windows, vs the one
built with Apple's gcc on MacOSX.
Alex
Apr 25 '06 #18

P: n/a
sturlamolden <st**********@yahoo.no> wrote:
Robert Kern wrote:
- gcc does not optimize particularly well.


That is beyond BS. The more recent gcc releases optimize as well as any
commercial compiler. GCC 4 may even optimize better than MSVC.

GCC is the compiler used to build the Linux kernel and MacOSX. If it
can deal with this I say it can handle Python as well. My guess is you
will not notice any performance difference for the Python interpreter.


That is quite possible, of course: if, for example, the 14% speed
difference I measured with pybench is representative, then you might not
notice it because human beings have a hard time noticing time
differences of that magnitude. But it might still be there, easy to
measure, even though not to notice.

At the same time, if the 14% slowdown is representative, then it's not
true that the compiler responsible for it "optimizes as well" as the
other; indeed, "does not optimize particularly well", under such a
hypothesis, would be far from a "beyond BS" assertion.
Alex
Apr 25 '06 #19

P: n/a
sturlamolden wrote:
Robert Kern wrote:
- gcc does not optimize particularly well. That is beyond BS.

The more recent gcc releases optimize as well as any
commercial compiler.


This is an outrageous claim. Having worked a bit doing
compilers, I no of no commercial compiler that would have
a leg to stand on if it claimed it optimized "as well as
any commercial compiler."

There are compilers and compiler experts out there in the
weeds that can (and regularly do) compile the hell out of code.
I took classes from one of these guys, and now he works in a
little company that does exactly that for particular customers
and problems.

Uncle Timmy, used to work for some of the places that employed
some of these experts. There is no "fully optimized state" for
compiled code (if the resultant instruction stream is greater
than a few hundred instructions); everyone approximates optimization.
It is almost inevitably a grey trade-off of compile time to speed
of resulting code over certain distributions of problems.

To say "gcc does a credible job of optimization" would not be
outrageous. To say "gcc optimization beats tool X on benchmark Y
on hardware Z by 1.237%" is a claim that could be made supportably,
but it would probably have to be accompanied with "using optimization
switches gcc-W for gcc and X-W for tool X."

I expect gcc does a reasonably good job of optimization, but the
best optimizers (as far as I know) are done by a relatively small
number of people who fully understand the workings of the optimizer
and its current workload, the workings of the hardware in question
as well, and have chosen a portion of that workload to become expert
in. Gcc does not have that expertise available to support people
to do things like that (nor does it need to). They don't need (or
want) to become wed to one architecture, but to do the best optimization
you must at some point get married.

The gcc team does a great job at what they do; blind defense of them
discredits the good work they do.

--Scott David Daniels
sc***********@acm.org
Apr 25 '06 #20

P: n/a
Alex Martelli wrote:
At the same time, if the 14% slowdown is representative, then it's not
true that the compiler responsible for it "optimizes as well" as the
other; indeed, "does not optimize particularly well", under such a
hypothesis, would be far from a "beyond BS" assertion.


Maybe someone with access to Windows and some free time could compare
pybench compiled under msvc and mingw. The best way to test the compiler
is to eliminate all other variables.

Apr 25 '06 #21

P: n/a
Alex Martelli wrote:
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3


this is the second time I've seen that 14% figure. OOC, where does it come
from? the data sets you posted show an average 12.6% speedup. 14 is an
odd way to round. :)

I don't think it's very useful to talk about average speedups from a
benchmark of equally-weighted feature tests. the data shows wildly varying
differences in performance for each test. a real-world application could
be much slower or much faster on either platform depending on its feature
mix. not the type of thing that's amenable to expression as a single
value.

Apr 25 '06 #22

P: n/a
Brian Elmegaard wrote:
"Martin v. Löwis" <ma****@v.loewis.de> writes:
You can build the entire Python interpreter with Cygwin (but you
don't need to, because there is a precompiled version), and you can
build extensions for the python.org binary using MingW.


Great, then I tend to agree that there is no reason for building it
with mingw.


wouldn't the gcc/cygwin build of python rely on the cygwin dll? cygwin
implements the standard unix system calls on top of the windows platform.
iirc, some operations run much slower through cygwin than native windows
calls (file and terminal i/o come to mind).

if so, i do see good reason to build with mingw instead of gcc/cygwin.

Apr 25 '06 #23

P: n/a

Edward Elliott wrote:
if so, i do see good reason to build with mingw instead of gcc/cygwin.


MinGW and Cygwin GCC is actually the same compiler. On Cygwin you can
remove the dependency on the cygwin dll by compiling with -mno-cygwin.
But as long as the cygwin dll is there, it creates an overhead for any
system call. The worst overhead is associated with the Unix fork()
system call, which Cygwin must emulate as there are no Win32
equivalent. In particular, a fork() on Unix will be optimized with
copy-on-write semantics in the kernel, but Cygwin cannot emulate that
as it has no access to the Windows kernel.

Another issue is that MinGW and Cygwin implements C++ exceptions with
setjmp/longjmp. Other verions of GCC (including DJGPP for DOS) does not
do this and as a result produces faster code for C++. I don't know if
there are any C++ in Python, but if there is, it will be an issue. it
is not an issue for plain C though.

Finally, MinGW (I am not sure about Cygwin) statically links the C++
standard library. VC++ and GCC on other platforms (e.g. Linux) does not
(normally) do this, and as a result the executables get a smaller
footprint.

Apr 25 '06 #24

P: n/a
sturlamolden wrote:
MinGW and Cygwin GCC is actually the same compiler. On Cygwin you can
remove the dependency on the cygwin dll by compiling with -mno-cygwin.
But as long as the cygwin dll is there, it creates an overhead for any
system call.


Thanks for that very informative post! To clarify, mingw (aka gcc
-mno-cygwin) has no POSIX layer like cygwin. Because your post could also
be (incorrectly) interpreted to mean mingw removes the cygwin dll
dependency by just linking it in statically. But I googled and this is not
the case.

Back to the original point about whether mingw compilation is necessary
given the python compiles under cygwin: Yes, it is. There can't be an
apples-to-apples comparison between gcc and msvc-compiled python with the
cygwin POSIX layer in the way.

Couldn't you compile the msvc-python code under gcc/mingw? If the code
sticks to C standards, and mingw can link to native libs, it should work.
The only problem is if python relies on some msvc-specific weirdness under
msvc. While there's much weirdness in msvc, I would expect a
cross-platform app like python to stay away from it.
Apr 25 '06 #25

P: n/a
Brian Elmegaard wrote:
That is simply not true.


Actually, you answered me then too. I misunderstood after reading
http://sebsauvage.net/python/mingw.html.

Is the information on that page not correct? Has it never been?


It's not correct, to the best of my knowledge. However, since
very few people use it, support for mingw keeps breaking
(inadvertently), so there might be bugs that prevent it from
working *for you*.

At the point this was written (apparently around Python 2.2)
it was possible to build Python extensions with MingW (the page
actually explains you how to do that), so "Under Windows, if you
do not have the costly Microsoft Visual C++, you cannot
install Python extensions written in C" wasn't even true
at the time it was written. You couldn't use distutils, but
you could certainly have built extensions by invoking the
compiler manually.

There were times when cygwin couldn't link with .DLLs unless
they were created with GNU ld, but these times are long past
history.

Regards,
Martin
Apr 25 '06 #26

P: n/a
Edward Elliott wrote:
Thanks for that very informative post! To clarify, mingw (aka gcc
-mno-cygwin) has no POSIX layer like cygwin. Because your post could also
be (incorrectly) interpreted to mean mingw removes the cygwin dll
dependency by just linking it in statically. But I googled and this is not
the case.
Well, you are not compiling with neither mingw, nor cygwin; you are
compiling with gcc in either case.

The question is what C library (and header files) you use: with cygwin,
you use cygwin1.dll (and its header files); with mingw, you use
msvcrt.dll (or some other MS CRT), along with the a GNU version of the
Microsoft header files.
Couldn't you compile the msvc-python code under gcc/mingw? If the code
sticks to C standards, and mingw can link to native libs, it should work.
Well, there is no native C library on Microsoft Windows: the system
simply doesn't include an official C library (I know there is crtdll.dll
and msvcrt.dll, but these aren't "endorsed" system C libraries).
The only problem is if python relies on some msvc-specific weirdness under
msvc. While there's much weirdness in msvc, I would expect a
cross-platform app like python to stay away from it.


For Windows, that would require not to use any of the standard C
functionality, since the system doesn't provide that functionality out
of the box. So Python itself never uses and msvcrt weirdness (well,
some, but that could be dropped easily) - yet still, the Python
binary will depend on a specific version of the MS CRT.

Regards,
Martin
Apr 25 '06 #27

P: n/a
sturlamolden wrote:
- there is no build process available to do that


In MSYS:

$ ./configure --prefix=/c/mingw
$ make
$ make install

This should be obvious to any with Unix experience.

MinGW actually distribute precompiled Python binaries as well (in
MSYS-DTK).


So how does that deal with the various extension modules that
PCbuild/readme.txt mentions? Do I get winsound.pyd? _msi.pyd
(in 2.5)?

Please believe me: there is currently no build process that
gives the same results as the build process used. It might
be fairly easy to create one, but none exists as of today.
- people building extensions to Python must be able to do so with
Microsoft C++, since some of these extensions are written using MFC.


MinGW can compile MFC. Download Windows Platform SDK and you get the
MFC source.


Did you actually try that? Last I tried, GCC would refuse to compile
the MinGW header files.

Even if you succeed, I don't think the SDK license will allow you
to distribute your binary, and it *will* be incompatible the official
MFC binary. MSVC++ has its own object layout, vtable layout, name
mangling, so you can't use GCC to link against MSVC-compiled
C++ libraries.

Regards,
Martin
Apr 25 '06 #28

P: n/a

sturlamolden wrote:
MinGW and Cygwin GCC is actually the same compiler.


Not exactly. They're both GCC, but the MinGW compiler that you can
download from MinGW WWW site is a native Win32 appliction, while the
"MinGW" compiler included with Cygwin and invoked by "-mno-cygwin" is a
Cygwin application.

Ross Ridge

Apr 25 '06 #29

P: n/a

sturlamolden wrote:
MinGW can compile MFC. Download Windows Platform SDK and you get the
MFC source.


Do not do this. The Platform SDK's EULA does not permit you to
redistribute anything you build from the MFC sources included in the
SDK. The only way to get a copy of MFC that you can legitimately use
in anything you distribute you need to buy a Microsoft C++ compiler.

Ross Ridge

Apr 25 '06 #30

P: n/a

Edward Elliott wrote:
Couldn't you compile the msvc-python code under gcc/mingw?


Yes I could, but I cannot compile the code under msvc for comparison. I
only have MinGW. If build the mingw binary then someone else has to
build the msvc binary for comparison. Then we could do pybenches on the
same computer.

Uh .. I actually think it could be an EULA violation to publish mingw
vs. msvc benchmarks without permission from Microsoft. I don't want to
part of anything illegal or have M$ lawyers breathing down my back. If
we are going to do this, then we must do it properly and get the
permissions we need.

Apr 25 '06 #31

P: n/a

Martin v. Lwis wrote:
Well, there is no native C library on Microsoft Windows: the system
simply doesn't include an official C library (I know there is crtdll.dll
and msvcrt.dll, but these aren't "endorsed" system C libraries).


MSVCRT.DLL has been a standard system compent of Windows since at least
Windows 98. Many other system components depend on it. Essentially,
MSVCRT.DLL is an "undocumented" part of the Windows API. It's not
exactly "endorsed", Microsoft would rather you use it's current
compiler and runtime, but it is the standard "official" Windows system
C library.

Ross Ridge

Apr 25 '06 #32

P: n/a
sturlamolden wrote:
But as long as the cygwin dll is there, it creates an overhead for any
system call. The worst overhead is associated with the Unix fork()
system call, which Cygwin must emulate as there are no Win32
equivalent. In particular, a fork() on Unix will be optimized with
copy-on-write semantics in the kernel, but Cygwin cannot emulate that
as it has no access to the Windows kernel.


Cygwin doesn't do COW fork() for historical reasons. Windows 95/98
didn't expose support for such a thing, but NT, XP, etc do (pass NULL
as the SectionHandle parameter to ZwCreateProcess or the older
NtCreateProcess to do a COW duplicate of the calling process).

Cygwin at one point had a fork() implementation using this in the
development tree (or being discussed on the mailing lists) but decided
not to use it since it wasn't much faster than full-copy fork for some
reason, and it would've split the code for NT/95.

Apr 25 '06 #33

P: n/a
sturlamolden wrote:
Edward Elliott wrote:
Couldn't you compile the msvc-python code under gcc/mingw?
Yes I could, but I cannot compile the code under msvc for comparison. I
only have MinGW. If build the mingw binary then someone else has to
build the msvc binary for comparison. Then we could do pybenches on the
same computer.


Sorry, I didn't mean you personally, I meant "you" in the general sense.

Uh .. I actually think it could be an EULA violation to publish mingw
vs. msvc benchmarks without permission from Microsoft. I don't want to
part of anything illegal or have M$ lawyers breathing down my back. If
we are going to do this, then we must do it properly and get the
permissions we need.


Why, because of a "no benchmarks" clause in the EULA? First, I'm fairly
certain that clause is uneforceable. Second, it's not worth MS's time to
come after you. Third, even if they did, they'd have an incredibly tough
time showing that any harm occured from your actions, leaving them with no
recovery. Fourth, if it somehow came to that, you'd have organizations
like the EFF lining up at your door to take the case gratis. I'm assuming
US law here.

Not saying you should do it, just that the risk in my opinion is extremely
low. I am a law student, but this is not legal advice.

Apr 25 '06 #34

P: n/a
"Martin v. Löwis" wrote:
Well, you are not compiling with neither mingw, nor cygwin; you are
compiling with gcc in either case.
touche, mr. pedant. :)
Well, there is no native C library on Microsoft Windows: the system
simply doesn't include an official C library (I know there is crtdll.dll
and msvcrt.dll, but these aren't "endorsed" system C libraries).
don't know what you mean by "endorsed". does it lack features of the C89
ANSI standard?
For Windows, that would require not to use any of the standard C
functionality, since the system doesn't provide that functionality out
of the box.


That would be a problem then. So what happens when you compile python with
msvc, and why can't mingw just replicate that?

Apr 25 '06 #35

P: n/a
Ross Ridge wrote:
MSVCRT.DLL ... It's not
exactly "endorsed", Microsoft would rather you use it's current
compiler and runtime, but it is the standard "official" Windows system
C library.


Does it comply with the ANSI C89 standard? I'm still not seeing why mingw
can't just link python to it.
Apr 25 '06 #36

P: n/a

Edward Elliott wrote:
Sorry, I didn't mean you personally, I meant "you" in the general sense.


OK :-)

I've just tried to build Python 2.4.3 with MinGW (MSYS 1.0.10, GCC
3.4.2):

$ ./configure --prefix=/c/Python243-mingw
$ make

The build then failed on posixmodule.c.
Sturla Molden
../Modules/posixmodule.c:308: warning: function declaration isn't a
prototype
../Modules/posixmodule.c: In function `posix_ttyname':
../Modules/posixmodule.c:1162: warning: implicit declaration of function
`ttyname'
../Modules/posixmodule.c:1162: warning: assignment makes pointer from
integer without a cast
../Modules/posixmodule.c: In function `posix_mkdir':
../Modules/posixmodule.c:1791: error: too many arguments to function
`mkdir'
../Modules/posixmodule.c: In function `posix_execv':
../Modules/posixmodule.c:2199: warning: passing arg 2 of `execv' from
incompatible pointer type
../Modules/posixmodule.c: In function `posix_execve':
../Modules/posixmodule.c:2332: warning: passing arg 2 of `execve' from
incompatible pointer type
../Modules/posixmodule.c:2332: warning: passing arg 3 of `execve' from
incompatible pointer type
../Modules/posixmodule.c: In function `posix_fork':
../Modules/posixmodule.c:2847: warning: implicit declaration of function
`fork'
../Modules/posixmodule.c: In function `posix_openpty':
../Modules/posixmodule.c:2909: error: `O_NOCTTY' undeclared (first use
in this function)
../Modules/posixmodule.c:2909: error: (Each undeclared identifier is
reported only once
../Modules/posixmodule.c:2909: error: for each function it appears in.)
../Modules/posixmodule.c:2912: error: `SIGCHLD' undeclared (first use in
this function)
../Modules/posixmodule.c:2914: warning: implicit declaration of function
`grantpt'
../Modules/posixmodule.c:2919: warning: implicit declaration of function
`unlockpt'
../Modules/posixmodule.c:2924: warning: implicit declaration of function
`ptsname'
../Modules/posixmodule.c:2924: warning: assignment makes pointer from
integer without a cast
../Modules/posixmodule.c:2931: warning: implicit declaration of function
`ioctl'
../Modules/posixmodule.c:2931: error: `I_PUSH' undeclared (first use in
this function)
../Modules/posixmodule.c: In function `posix_getegid':
../Modules/posixmodule.c:2973: warning: implicit declaration of function
`getegid'
../Modules/posixmodule.c: In function `posix_geteuid':
../Modules/posixmodule.c:2986: warning: implicit declaration of function
`geteuid'
../Modules/posixmodule.c: In function `posix_getgid':
../Modules/posixmodule.c:2999: warning: implicit declaration of function
`getgid'
../Modules/posixmodule.c: In function `posix_getppid':
../Modules/posixmodule.c:3121: warning: implicit declaration of function
`getppid'
../Modules/posixmodule.c: In function `posix_getuid':
../Modules/posixmodule.c:3163: warning: implicit declaration of function
`getuid'
../Modules/posixmodule.c: In function `posix_kill':
../Modules/posixmodule.c:3193: warning: implicit declaration of function
`kill'
../Modules/posixmodule.c: In function `posix_wait':
../Modules/posixmodule.c:4970: warning: implicit declaration of function
`wait'
../Modules/posixmodule.c: In function `posix_pipe':
../Modules/posixmodule.c:5511: warning: implicit declaration of function
`pipe'
../Modules/posixmodule.c: At top level:
../Modules/posixmodule.c:382: warning: 'posix_error_with_filename'
defined but not used
../Modules/posixmodule.c:534: warning: 'posix_fildes' defined but not
used
../Modules/posixmodule.c:6223: warning: 'conv_confname' defined but not
used
../Modules/posixmodule.c:7126: warning: 'setup_confname_table' defined
but not used
make: *** [Modules/posixmodule.o] Error 1

Apr 25 '06 #37

P: n/a
sturlamolden:
Uh .. I actually think it could be an EULA violation to publish mingw
vs. msvc benchmarks without permission from Microsoft. I don't want to
part of anything illegal or have M$ lawyers breathing down my back. If
we are going to do this, then we must do it properly and get the
permissions we need.


The EULA attempted to stop disclosing benchmarks of .NET code, not
C++ code. The most recent version of the EULA has replaced that with
permission to disclose benchmarks of .NET code provided some conditions
are met: full disclosure of benchmark code and environment; latest
release of .NET; compliance with 'best practices'. Its still stupid as
it would be in Microsoft's interest to encourage benchmarking .NET but
not quite as stupid as before.

Neil
Apr 25 '06 #38

P: n/a

Martin v. Lwis wrote:
Please believe me: there is currently no build process that
gives the same results as the build process used. It might
be fairly easy to create one, but none exists as of today.


I tried to build with MinGW this eveing and it failed. I believe you We
need to make a build process for MinGW.

Apr 25 '06 #39

P: n/a
Edward Elliott wrote:
Ross Ridge wrote:
MSVCRT.DLL ... It's not
exactly "endorsed", Microsoft would rather you use it's current
compiler and runtime, but it is the standard "official" Windows system
C library.


Does it comply with the ANSI C89 standard? I'm still not seeing why mingw
can't just link python to it.


It can.

--
Robert Kern
ro*********@gmail.com

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Apr 25 '06 #40

P: n/a
Look what I just found:

http://jove.prohosting.com/iwave/ipython/pyMinGW.html

A build process for python 2.4.2 (i.e. not the latest) for MinGW.

Apr 25 '06 #41

P: n/a
Edward Elliott <no****@127.0.0.1> wrote:
Alex Martelli wrote:
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3
this is the second time I've seen that 14% figure. OOC, where does it come
from? the data sets you posted show an average 12.6% speedup. 14 is an
odd way to round. :)


I believe 12.6% is the result if you pick the faster speed as a
denominator (and so it's the right figure for a _slowdown_,
hypothetically moving from fast to slow case), 14% if you pick the
slower speed as a denominator (i.e., for a _speedup_).

Of course if you work with times rather than speeds it's the other way
around.

I don't think it's very useful to talk about average speedups from a
benchmark of equally-weighted feature tests. the data shows wildly varying
differences in performance for each test. a real-world application could
be much slower or much faster on either platform depending on its feature
mix. not the type of thing that's amenable to expression as a single
value.


I believe the single figure is a useful summary. Even the most
sophisticated benchmarks are eventually boiled down to single figures,
as in "so many SPECmarks" etc, because in everyday discourse a scalar is
what you can reasonably discuss. Sure, philosophically speaking it
makes no sense to say that a compiler is better or worse than another at
optimization, without adding a lawyer's brief worth of qualifications
each and every time. In real life, it's a useful and practical
temporary simplification, and engineers (as opposed to mathematicians
and philosophers) have a bias towards practical usefulness.
Alex
Apr 26 '06 #42

P: n/a
Edward Elliott <no****@127.0.0.1> wrote:
Alex Martelli wrote:
At the same time, if the 14% slowdown is representative, then it's not
true that the compiler responsible for it "optimizes as well" as the
other; indeed, "does not optimize particularly well", under such a
hypothesis, would be far from a "beyond BS" assertion.


Maybe someone with access to Windows and some free time could compare
pybench compiled under msvc and mingw. The best way to test the compiler
is to eliminate all other variables.


I assume you mean "compare pybench on Python 2.4.3 compiled under" etc.
Unfortunately, I suspect the ex-officio gcc defenders will only react by
finding yet another quibble (anything to avoid admitting that gcc may
not be as good at optimizing as a _Microsoft_ product -- perish the
thought!), therefore I'm disinclined to invest my scarce free time in
that particular endeavour.
Alex
Apr 26 '06 #43

P: n/a
Alex Martelli wrote:
I assume you mean "compare pybench on Python 2.4.3 compiled under" etc.
Yep
Unfortunately, I suspect the ex-officio gcc defenders will only react by
finding yet another quibble (anything to avoid admitting that gcc may
not be as good at optimizing as a _Microsoft_ product -- perish the
thought!), therefore I'm disinclined to invest my scarce free time in
that particular endeavour.


No problem. I don't expect anyone to follow through, it was more a
suggestion to forestall rampant speculation. :)

Apr 26 '06 #44

P: n/a
Alex Martelli wrote:
I believe the single figure is a useful summary. Even the most
sophisticated benchmarks are eventually boiled down to single figures,
as in "so many SPECmarks" etc, because in everyday discourse a scalar is
what you can reasonably discuss. Sure, philosophically speaking it
makes no sense to say that a compiler is better or worse than another at
optimization, without adding a lawyer's brief worth of qualifications ^^^^^^^^^^^^^^
hey is that a shot at me? ;)
each and every time. In real life, it's a useful and practical
temporary simplification, and engineers (as opposed to mathematicians
and philosophers) have a bias towards practical usefulness.


I agree for benchmarks in general. It's more this particular benchmark I
object to as not being representative. It's like the difference between
SPECmark testing various CPU functions and Winstone measuring real-world
application performance. If you're a CPU designer counting coup, SPECmark
matters. For just about everyone else, Winstone tells you more. And when
you're talking about small (less than an order of magnitude) differences in
SPECmark, platform/system/application issues becomes the dominant factor in
performance. A CPU that's twice as fast won't help my i/o-bound server.

From an engineering standpoint, the pybench number isn't that useful to me.
It tells me little about the practical speed of my application on two
different python interpreters. That's all I'm saying. No need to sic the
philosophers on me (a fate I would not wish on my worst enemy). :)

Apr 26 '06 #45

P: n/a
Alex Martelli wrote:
sturlamolden <st**********@yahoo.no> wrote:
Robert Kern wrote:
> Dunno. Depends on the machine. Depends on the program. Depends on how
> the interpreter and any extension modules and underlying libraries were
> built. Depends on which Linux and which Windows.
>
> I'm sorry, but your question is a non sequitur. I don't understand its
> relevance to this thread.


The relevance: Python is built with GCC on Linux. Do you or do you not
see a performance hit on Linux?

MinGW is GCC. Will you get a performance hit when building Python with
MinGW?


I cannot predict this, though it would be great if somebody who does
have both VS2003 and mingw could give it a try.

What I did just post on another thread over the last couple of days is
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3
under Win2000 under Parallels Workstation beta, compared to 2.4.3
Universal directly on MacOSX -- the standard build of 2.4.3 in either
cause, i.e., the one built with MS compilers on Windows, vs the one
built with Apple's gcc on MacOSX.
Alex


Please when quoting such benchmarks include gcc version. gcc >= 4.1 is
supposed to have a lot of performance improvements. This is the current
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.

Apr 26 '06 #46

P: n/a
Neal Becker wrote:
What I did just post on another thread over the last couple of days is
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3
under Win2000 under Parallels Workstation beta, compared to 2.4.3
Universal directly on MacOSX -- the standard build of 2.4.3 in either
cause, i.e., the one built with MS compilers on Windows, vs the one
built with Apple's gcc on MacOSX.
Please when quoting such benchmarks include gcc version. gcc >= 4.1 is
supposed to have a lot of performance improvements. This is the current
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.


does
"the standard build of 2.4.3 in either case"

do you have trouble understanding ?

(what makes you think that 2.4.3 for windows is compiled with the best possible
compiler for the windows environment?)

</F>

Apr 26 '06 #47

P: n/a
Neal Becker wrote:
What I did just post on another thread over the last couple of days is
about MacOSX, which also uses gcc: 14% faster pybench using Python 2.4.3
under Win2000 under Parallels Workstation beta, compared to 2.4.3
Universal directly on MacOSX -- the standard build of 2.4.3 in either
cause, i.e., the one built with MS compilers on Windows, vs the one
built with Apple's gcc on MacOSX.
Please when quoting such benchmarks include gcc version. gcc >= 4.1 is
supposed to have a lot of performance improvements. This is the current
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.


what part of

"the standard build of 2.4.3 in either case"

do you have trouble understanding ?

(what makes you think that 2.4.3 for windows is compiled with the best possible
compiler for the windows environment?)

</F>

Apr 26 '06 #48

P: n/a
Neal Becker <nd*******@gmail.com> writes:
release. Since mingw is usually current, I haven't checked, but they may
be using 4.1 now.


It is not, it is 3.4.2.
http://www.mingw.org/download.shtml#hdr2

--
Brian (remove the sport for mail)
http://www.et.web.mek.dtu.dk/Staff/be/be.html
http://www.rugbyklubben-speed.dk
Apr 26 '06 #49

P: n/a
Edward Elliott wrote:
Well, there is no native C library on Microsoft Windows: the system
simply doesn't include an official C library (I know there is crtdll.dll
and msvcrt.dll, but these aren't "endorsed" system C libraries).


don't know what you mean by "endorsed". does it lack features of the C89
ANSI standard?


There isn't an import library for these DLLs anywhere, and Microsofts
documents it (msvcrt.dll) as not intended for application use:

http://msdn2.microsoft.com/en-us/lib...yh(VS.80).aspx

'The msvcrt.dll is now a "known DLL," meaning that it is a system
component owned and built by Windows. It is intended for future use only
by system-level components.'
For Windows, that would require not to use any of the standard C
functionality, since the system doesn't provide that functionality out
of the box.


That would be a problem then. So what happens when you compile python with
msvc, and why can't mingw just replicate that?


I link with msvcr71.dll (currently). To distribute Python, I need to
include a copy of msvcr71.dll, which I can, because the MSVC license
allows me to redistribute that DLL.

When I link with mingw, I have a choice of DLLs to link with, including
msvcrt.dll, msvcrt4.dll, msvcr71.dll, and perhaps others - I don't even
need the DLLs on my system to link with them.

However, I cannot redistribute these DLLs when I compile with MingW
(unless I also have a copy of VS.NET - I would have to reread its
license to find out whether it requires that the application is actually
built with MSVC to allow for redistribution).

Regards,
Martin
Apr 26 '06 #50

66 Replies

This discussion thread is closed

Replies have been disabled for this discussion.