New versions breaking extensions, etc.

[Jive]

Can someone explain to me why Python 2.4 on MS Windows has these backward
compatibility problems? What am I missing? Why won't extensions compiled
to run with 2.3 also work with 2.4? Why does it matter whether a component
was compiled with VC++ 6.0 or 7.1? I've been using MS stuff for 6 years.
Before I started using Python, I don't remember ever having a program tell
me I had to use an OLDER version of something to make the program work.
Newer, maybe, but older, never. I had to revert to Python 2.3 because I
have applications that will not run under Python 2.4.

Although I have been a full-time software developer since the late 70's, I
don't know much about Microsoft stuff. (Amazing trick that, eh?) But I've
been reading some things on the MS web pages that say dot-NET and all that
should NOT break existing stuff.
http://msdn.microsoft.com/library/en..._libraries.asp I
don't pretend to understand it all, but certainly one would not expect MS to
screw things up too badly on purpose. The value of MS Windows lies in the
applications that it can run. That's part of the genius of the Evil Genius.
He understood when others didn't that when Joe Blow in his basement develops
a program to run under MS Windows, Joe Blow is making money for the E.G.,
and it doesn't cost the E.G. a penny.

In my own small domain, I cannot even consider coming out with new releases
that are not drop-in replacements for what went before. Customers won't go
for it. It's an absolute deal-breaker.

What puzzles me is that, apparently, the incompatibility of Python 2.4 and
extensions built for 2.3 did not come as a nasty surprise. I get the
impression that it was expected. Can someone explain that to me?

I hope this doesn't sound like I'm complaining. I think Python is great.
And the price can't be beat!

[Robert]

VS7 is a really a vastly different beastie than VS6.
On 12/10/04 9:31 PM, in article YZ********************@news.easynews.com,
"Jive" <so*****@microsoft.com> wrote:

Can someone explain to me why Python 2.4 on MS Windows has these backward
compatibility problems? What am I missing? Why won't extensions compiled
to run with 2.3 also work with 2.4? Why does it matter whether a component
was compiled with VC++ 6.0 or 7.1? I've been using MS stuff for 6 years.
Before I started using Python, I don't remember ever having a program tell
me I had to use an OLDER version of something to make the program work.
Newer, maybe, but older, never. I had to revert to Python 2.3 because I
have applications that will not run under Python 2.4.

Although I have been a full-time software developer since the late 70's, I
don't know much about Microsoft stuff. (Amazing trick that, eh?) But I've
been reading some things on the MS web pages that say dot-NET and all that
should NOT break existing stuff.
http://msdn.microsoft.com/library/en....time_librarie
s.asp I
don't pretend to understand it all, but certainly one would not expect MS to
screw things up too badly on purpose. The value of MS Windows lies in the
applications that it can run. That's part of the genius of the Evil Genius.
He understood when others didn't that when Joe Blow in his basement develops
a program to run under MS Windows, Joe Blow is making money for the E.G.,
and it doesn't cost the E.G. a penny.

In my own small domain, I cannot even consider coming out with new releases
that are not drop-in replacements for what went before. Customers won't go
for it. It's an absolute deal-breaker.

What puzzles me is that, apparently, the incompatibility of Python 2.4 and
extensions built for 2.3 did not come as a nasty surprise. I get the
impression that it was expected. Can someone explain that to me?

I hope this doesn't sound like I'm complaining. I think Python is great.
And the price can't be beat!

[News M Claveau /Hamster-P]

Hi !

But, if Python is as much sensitive to the passage of an external software,
version 6 (obsolete) with a version 7 (obsolete also), it is worrying.

Michel Claveau

[Nick Coghlan]

Jive wrote:

Can someone explain to me why Python 2.4 on MS Windows has these backward
compatibility problems? What am I missing?

The problem is the Python C/API. At the moment, it exposes things directly (like
data structures) that may change size between major version releases. The other
issue is that the interpreter and the extensions may be linked to different
versions of the Microsoft runtime.

This is a solvable problem, but it would require:
1. Major additions to the C/API - a "Python Major Version Agnostic" API that
hides all data structures behind opaque pointers, never passes file descriptors
between the interpreter and the extension and never gets Python to delete memory
allocated by the extension (or vice-versa)
2. Rewriting extensions to use the new API functions instead of the current
major version specific ones.

Such an API is likely to be both harder to work with and slower, simply due to
the limitations of C.

To this date, no-one has cared enough about the problem to put in the effort
required to make the C API version agnostic. Given that the API almost always
remains *source* compatible, within a month or two of a new Python release, the
extension developers have generally tweaked their build environments to also
produce binaries for the new release.

In the current case (Python 2.4), the transition from msvcrt to msvcrt71, and
the poor quality of the free development tools offered by Microsoft has
exacerbated the problem on Windows. The Python developers are looking at
improving the documentation and support for building extensions with Windows
compilers other than Visual Studio .Net 2003 (mainly MinGW and the free MS tools).

Although key extension developers could always try asking the PSF to buy them a
Visual Studio license ;)

Cheers,
Nick.

--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net

[Jive]

"Nick Coghlan" <nc******@iinet.net.au> wrote in message
news:ma**************************************@pyth on.org...

Jive wrote:

Can someone explain to me why Python 2.4 on MS Windows has these backward compatibility problems? What am I missing?
The problem is the Python C/API. At the moment, it exposes things directly

(like data structures) that may change size between major version releases.
Okay. That could be fixed.
The other
issue is that the interpreter and the extensions may be linked to different versions of the Microsoft runtime.
Doesn't Microsoft have an answer for that? There are (at last count) nine
skillion ActiveX
components in the wild. Surely Microsoft didn't blast them into oblivion
with dot-net -- did it?

This is a solvable problem, but it would require:
1. Major additions to the C/API - a "Python Major Version Agnostic" API that hides all data structures behind opaque pointers, never passes file descriptors between the interpreter and the extension and never gets Python to delete memory allocated by the extension (or vice-versa)
I remember having trouble with that last bit once. But I seem to recall
there was an easy
answer. Something to do with the main ap and the DLL using the same heap or
some
such thing.
2. Rewriting extensions to use the new API functions instead of the current major version specific ones.
That's a one-time thing, for those who choose to do it. Certainly the old
API
would still be available.

Such an API is likely to be both harder to work with and slower, simply due to the limitations of C.

To this date, no-one has cared enough about the problem to put in the effort required to make the C API version agnostic. Given that the API almost always remains *source* compatible, within a month or two of a new Python release, the extension developers have generally tweaked their build environments to also produce binaries for the new release.
It would be a Good Thing to put a stop to that requirement, no?

In the current case (Python 2.4), the transition from msvcrt to msvcrt71, and the poor quality of the free development tools offered by Microsoft has
exacerbated the problem on Windows.
No doubt. I don't understand what the deal is with msvcr71 and all that,
but I have
figured out that it's a dirty rotten shame. If you google for msvcr71, you
will
get pages and pages of links to plaintive cries for help. We are not alone.
The Python developers are looking at
improving the documentation and support for building extensions with Windows compilers other than Visual Studio .Net 2003 (mainly MinGW and the free MS tools).

That would be good. But is using VC++ 6.0 right out? As it stands, I can
no longer
build extensions that will run with a standard dot-net Python 2.4 release.
Or at least I don't
know how. And if I build 2.4 under VC++ 6.0 and distribute that around the
company, it's not
clear to me if we can then use new third party extensions. Some
documentation
clearing it all up would be very helpful. Maybe it exists somewhere
already. This is
a Microsoft problem, not a Python problem.

Like I said, dirty rotten shame.

Thinking it over, I'm wondering why the Python crowd went with dot-NET in
the first place.
Surely the world would be a better, happier place without MS jerking
everyone around.
We work our fingerprints to the bone writing code to make their stinking
little OS do something
useful, and what have they ever given us in return? -- Besides the
aquaducts?

Jive

[Jive]

P.s. Does anyone know how to make Outlook Express leave my damned line-ends
alone? If I want line-ends. I know where to find the ENTER key.

[JanC]

Jive schreef:

P.s. Does anyone know how to make Outlook Express leave my damned
line-ends alone? If I want line-ends. I know where to find the ENTER
key.

Google for "oe-quotefix", but the best solution is to use a proper
newsreader. ;-)
--
JanC

"Be strict when sending and tolerant when receiving."
RFC 1958 - Architectural Principles of the Internet - section 3.9

[Mike Meyer]

"Jive" <so*****@microsoft.com> writes:

"Nick Coghlan" <nc******@iinet.net.au> wrote in message
news:ma**************************************@pyth on.org...

To this date, no-one has cared enough about the problem to put in the

effort

required to make the C API version agnostic. Given that the API almost

always

remains *source* compatible, within a month or two of a new Python

release, the

extension developers have generally tweaked their build environments to

also

produce binaries for the new release.

It would be a Good Thing to put a stop to that requirement, no?

Actually, there's a problem on Unix that may not exist on
Windows. Python is installed in <PREFIX>/lib/python<version>/. This
lets us have multiple versions of Python installed at the same time,
which is a good thing.

Now, installed packages and extensions go in
<PREFIX>/lib/python<version>/site-packages. This means that you can't
use *any* of the previously installed packages in the new
version, even if they were pure python. Since I use TMDA, my mail
stopped working when I installed python 2.4.

The real solution is a database of installed packages (which may well
be needed for the apt-get/CPAN functionality that people want) so that
a single python script can fetch and install all the installed
packages.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Martin v. Löwis]

Jive wrote:

Why won't extensions compiled to run with 2.3 also work with 2.4?

I believe nobody has answered *this* question, yet:

Python extensions built for 2.3 link with python23.dll, Python
extensions build for 2.4 link with python24.dll.

pythonxy.dll has global variables, e.g. the pointers to True,
False, None, or the dict type object. If you have two copies
of the Python runtime, you get two copies of each local object.
This cannot work: we really need to rely on having only one
dict type, or else "is" comparisons with the dict type fail.

Regards,
Martin

[Martin v. Löwis]

Jive wrote:

The other
issue is that the interpreter and the extensions may be linked to
different versions of the Microsoft runtime.

Doesn't Microsoft have an answer for that?

Microsoft's answer to this question is: don't do that. Never
mix different versions of the CRT in a single application.
There are (at last count) nine skillion ActiveX
components in the wild. Surely Microsoft didn't blast them into oblivion
with dot-net -- did it?
No, it didn't - but so what? The point is that ActiveX controls normally
don't use the CRT in dangerous ways, since they are constrained to use
Co* function calls only.
I remember having trouble with that last bit once. But I seem to recall
there was an easy
answer. Something to do with the main ap and the DLL using the same heap or
some
such thing.
Something like that, for sure.

[require that extensions be recompiled] It would be a Good Thing to put a stop to that requirement, no?
No. The maintenance cost of such a new API would be significantly
higher, and Python would lose some of its attractiveness for C
developers.
No doubt. I don't understand what the deal is with msvcr71 and all that,
You probably either need to learn what the deal is, or else you have to
trust others that there is no better way.
That would be good. But is using VC++ 6.0 right out? As it stands, I can
no longer build extensions that will run with a standard dot-net Python
2.4 release. Or at least I don't know how.
The most easy way to do this is to use VC7.1 (VS.NET 2003). That will
work out of the box (you need to box first, of course).
Thinking it over, I'm wondering why the Python crowd went with dot-NET in
the first place.
We did not go to .NET. Python 2.4 has nothing to do with .NET. It just
happens that the C compiler we use ships as part of a product that has
..NET in its name - the C compiler itself is Microsoft Visual C 7.1,
aka Microsoft C 13.10.

That compiler is now used because it has a number of improvements over
Microsoft C 12.00, and because VC6 is no longer commercially available.
Many developers have only VS .NET 2003 available, so if Python would
continue to be built with VC6, those people could not build
extensions for it - they could not even buy the compiler they needed,
anymore. OTOH, people who only have VC6 just need to buy VS.NET 2003,
which is still available. Or else they find out what alternative
compiler arrangements can be made.
Surely the world would be a better, happier place without MS jerking
everyone around.

It would surely be better if the operating system shipped with a
compiler, so anybody could rebuild things, and you wouldn't need
binary distributions in the first place :-)

Regards,
Martin

[Jive]

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41**************@v.loewis.de...

OTOH, people who only have VC6 just need to buy VS.NET 2003,
which is still available.

I don't even know how to do that! :-) What's the difference between VC++
..net Standard and Visual Studio .net Pro? (Besides $370?) Is the former
C++ only, but with the IDE, and the later the whole shebang with SourceSafe,
VBASIC, and all that?

OH NO! I've gone seriously off-topic. Please don't call the Spanish
Inquisiton. Allow me to re-phrase the question: What do I need to build
(on-topic) Python extensions?

[John Machin]

Jive wrote:

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41**************@v.loewis.de...

OTOH, people who only have VC6 just need to buy VS.NET 2003,
which is still available.
I don't even know how to do that! :-) What's the difference between

VC++ .net Standard and Visual Studio .net Pro? (Besides $370?) Is the former C++ only, but with the IDE, and the later the whole shebang with SourceSafe, VBASIC, and all that?

OH NO! I've gone seriously off-topic. Please don't call the Spanish
Inquisiton. Allow me to re-phrase the question: What do I need to build (on-topic) Python extensions?

Short answer to Jive's question: (1) free non-MS C compiler (either
MinGW or Borland) (2) inner calm.

I really can't understand what all the screaming and yelling is about.
Windows Python is built using an MS compiler. Those extension
developers who can't/won't buy the MS compiler use either the free
MinGW compiler or the free Borland 5.5 compiler (or both!). Yes, you
have to be careful about mixing the runtimes. An extension that tries
to use a FILE * that was created by Python will crash. Using free() on
a pointer that was malloc()ed by the other party isn't a bright idea
either. There are adequate solutions to these problems, involving
Python-supplied higher-level functions instead of C runtime functions.
Otherwise, not a problem. Distutils has made the process of using MinGW
and bcpp a snap. The documentation is adequate. When a new version of
Python comes out, one rebuilds and tests one's extensions. So ... now
there are THREE compilers that can be used instead of the one that
Python's built with; what's the big deal?

[It's me]

On the other hand, it can be annoying.

I can't use Python 2.4 right now because NumPy won't run. So, I need to
wait for NumPy to get updated.

Of course, one would say: but NumPy is open source, go build it yourself.

My answer is simple: If there are more then 24 hours to a day, I definitely
would...

--
It's me
"John Machin" <sj******@lexicon.net> wrote in message
news:11**********************@f14g2000cwb.googlegr oups.com...
Jive wrote:

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41**************@v.loewis.de...

OTOH, people who only have VC6 just need to buy VS.NET 2003,
which is still available.
I don't even know how to do that! :-) What's the difference between

VC++ .net Standard and Visual Studio .net Pro? (Besides $370?) Is the former C++ only, but with the IDE, and the later the whole shebang with SourceSafe, VBASIC, and all that?

OH NO! I've gone seriously off-topic. Please don't call the Spanish
Inquisiton. Allow me to re-phrase the question: What do I need to build (on-topic) Python extensions?

Short answer to Jive's question: (1) free non-MS C compiler (either
MinGW or Borland) (2) inner calm.

I really can't understand what all the screaming and yelling is about.
Windows Python is built using an MS compiler. Those extension
developers who can't/won't buy the MS compiler use either the free
MinGW compiler or the free Borland 5.5 compiler (or both!). Yes, you
have to be careful about mixing the runtimes. An extension that tries
to use a FILE * that was created by Python will crash. Using free() on
a pointer that was malloc()ed by the other party isn't a bright idea
either. There are adequate solutions to these problems, involving
Python-supplied higher-level functions instead of C runtime functions.
Otherwise, not a problem. Distutils has made the process of using MinGW
and bcpp a snap. The documentation is adequate. When a new version of
Python comes out, one rebuilds and tests one's extensions. So ... now
there are THREE compilers that can be used instead of the one that
Python's built with; what's the big deal?

[Steven Bethard]

It's me wrote:

My answer is simple: If there are more then 24 hours to a day, I definitely
would...

Can we get a patch in for this?

datetime.timedelta(hours=24) + datetime.timedelta(hours=1) datetime.timedelta(1)

would be much preferable to the current:
datetime.timedelta(hours=24) + datetime.timedelta(hours=1)

datetime.timedelta(1, 3600)

;)

Steve

[Jeremy Bowers]

On Sat, 11 Dec 2004 01:28:45 -0600, Mike Meyer wrote:

Actually, there's a problem on Unix that may not exist on
Windows. Python is installed in <PREFIX>/lib/python<version>/. This
lets us have multiple versions of Python installed at the same time,
which is a good thing.
...
The real solution is...

Interestingly, I saw you going somewhere else for this. I expected,

"The real solution is, at least for the pure Python packages, a
system-wide directory for those, reserving site-packages for things that
are truly version specific."

I'm not sure I'm ready to "propose" this yet, but thoughts? Bad idea, good
idea?

Obviously it isn't perfect, but it seems to me that reverse compatibility
has been good enough for a while now that the imperfections are minor in
impact...

[Martin v. Löwis]

Jive wrote:

I don't even know how to do that! :-) What's the difference between VC++
.net Standard and Visual Studio .net Pro? (Besides $370?) Is the former
C++ only, but with the IDE, and the later the whole shebang with SourceSafe,
VBASIC, and all that?
According to

http://msdn.microsoft.com/howtobuy/vstudio/

there is, for VS.NET, "Academic", "Professional", "Enterprise
Developer", and "Enterprise Architect". For Python extensions, all
these releases will work; see

http://msdn.microsoft.com/howtobuy/vstudio/features/

for a checkmark list. For VC++ .NET 2003 Standard, it appears that
you get just the C++ compiler, and the IDE, see

http://msdn.microsoft.com/howtobuy/v...s/default.aspx
OH NO! I've gone seriously off-topic. Please don't call the Spanish
Inquisiton. Allow me to re-phrase the question: What do I need to build
(on-topic) Python extensions?

Either VS.NET 2003 or VC++ .NET 2003 should do (although I don't know
anybody who owns the latter to be sure). The core issue is that it needs
a "native" C++ compiler (ie. not just managed C++), and that it needs
mscvcr71.dll.

Regards,
Martin

[Jive]

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41************@v.loewis.de...

Either VS.NET 2003 or VC++ .NET 2003 should do (although I don't know
anybody who owns the latter to be sure). The core issue is that it needs
a "native" C++ compiler (ie. not just managed C++), and that it needs
mscvcr71.dll.

Sorry if I'm being dense. If I had that DLL, why couldn't I use VC++ 6.0?

It's still not clear to me exactly what's in each package. One would think
there would be a more technical description on the MS web pages.

[Martin v. Löwis]

Jive wrote:

Either VS.NET 2003 or VC++ .NET 2003 should do (although I don't know
anybody who owns the latter to be sure). The core issue is that it needs
a "native" C++ compiler (ie. not just managed C++), and that it needs
mscvcr71.dll.

Sorry if I'm being dense. If I had that DLL, why couldn't I use VC++ 6.0?

Sorry for being imprecise. You don't need just msvcr71.dll, you need an
import library for it, and you need to cause link.exe to use that import
library. VC6 will continue to link your extension modules with
msvcrt40.dll - regardless of what other DLLs you have on your system.

It might be that you also need the header files for msvcr71.dll,
although I believe you could get away with using the "wrong" (i.e.
VC6) header files.

Regards,
Martin

[Jive]

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41***********************@news.freenet.de...

Jive wrote:

Either VS.NET 2003 or VC++ .NET 2003 should do (although I don't know
anybody who owns the latter to be sure). The core issue is that it needs
a "native" C++ compiler (ie. not just managed C++), and that it needs
mscvcr71.dll.
Sorry if I'm being dense. If I had that DLL, why couldn't I use VC++

6.0?
Sorry for being imprecise. You don't need just msvcr71.dll, you need an
import library for it, and you need to cause link.exe to use that import
library. VC6 will continue to link your extension modules with
msvcrt40.dll - regardless of what other DLLs you have on your system.

It might be that you also need the header files for msvcr71.dll,
although I believe you could get away with using the "wrong" (i.e.
VC6) header files.

Regards,
Martin

Well, ain't that enough to gag a maggot? I was aware that DLL's don't
really link dynamically. I was not aware that the crt dll file name was
hard-coded into the linker. But I looked on the link line and, sure enough,
that particular dll was not listed among the others.

Malice or incompetence? You be the judge.

[Martin v. Löwis]

Jive wrote:

Well, ain't that enough to gag a maggot? I was aware that DLL's don't
really link dynamically. I was not aware that the crt dll file name was
hard-coded into the linker. But I looked on the link line and, sure enough,
that particular dll was not listed among the others.

It's not hard-coded in the linker, but hard-coded in the import library.
So if you link with msvcrt.lib (which might not be the precise name
of the import library - I cannot look up the precise name right now),
msvcrt.lib will in turn refer to msvcr71.dll.

In addition, through #pragma comment(lib, foo.lib), the Microsoft C
*header* files force references to import libraries into the object
files, so that the import libraries don't even appear on the linker
line - instead, the linker gets additional command line options from
the object files; use dumpbin to find out what those are.

On the plus side, I believe that the name of the import library for
the CRT was always the same (msvcrt.lib), so different versions of
that file refer to different actual CRTs (msvcrt.dll, msvcrt40.dll,
msvcr70.dll, msvcr71.dll). So just replacing your msvcrt.lib file
of VC6 with the VC71 one might be enough - but then, perhaps there
also where changes to the header files which you need to get.

Regards,
Martin

[Cameron Laird]

In article <11**********************@f14g2000cwb.googlegroups .com>,
John Machin <sj******@lexicon.net> wrote:

Jive wrote:

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41**************@v.loewis.de...

> OTOH, people who only have VC6 just need to buy VS.NET 2003,
> which is still available.

I don't even know how to do that! :-) What's the difference between

VC++

.net Standard and Visual Studio .net Pro? (Besides $370?) Is the

former

C++ only, but with the IDE, and the later the whole shebang with

SourceSafe,

VBASIC, and all that?

OH NO! I've gone seriously off-topic. Please don't call the Spanish
Inquisiton. Allow me to re-phrase the question: What do I need to

build

(on-topic) Python extensions?

Short answer to Jive's question: (1) free non-MS C compiler (either
MinGW or Borland) (2) inner calm.

I really can't understand what all the screaming and yelling is about.
Windows Python is built using an MS compiler. Those extension
developers who can't/won't buy the MS compiler use either the free
MinGW compiler or the free Borland 5.5 compiler (or both!). Yes, you

[Cameron Laird]

In article <ma**************************************@python.o rg>,
Nick Coghlan <nc******@email.com> wrote:

Jive wrote:

Can someone explain to me why Python 2.4 on MS Windows has these backward
compatibility problems? What am I missing?

The problem is the Python C/API. At the moment, it exposes things
directly (like
data structures) that may change size between major version releases. The other
issue is that the interpreter and the extensions may be linked to different
versions of the Microsoft runtime.

This is a solvable problem, but it would require:
1. Major additions to the C/API - a "Python Major Version Agnostic" API that
hides all data structures behind opaque pointers, never passes file descriptors
between the interpreter and the extension and never gets Python to
delete memory
allocated by the extension (or vice-versa)
2. Rewriting extensions to use the new API functions instead of the current
major version specific ones.

Such an API is likely to be both harder to work with and slower, simply due to
the limitations of C.

To this date, no-one has cared enough about the problem to put in the effort
required to make the C API version agnostic. Given that the API almost always

[Cameron Laird]

In article <Ru*********************@news.easynews.com>,
Jive <so*****@microsoft.com> wrote:

[Jive]

"Cameron Laird" <cl****@lairds.us> wrote in message
news:32************@lairds.us...

Tcl went through this epiphany a few years, and has largely,
though not exclusively, committed to use of the once-more-
redirected API called Stubs <URL: http://wiki.tcl.tk/stubs >.
Some Tcl-ers are orgasmic about the wonders of Stubs, some
ignore it, a few don't like it, and most, perhaps, aren't
even aware of its existence. Performance doesn't appear to
be much of an issue, though.

It uses function pointer tables (like COM), but in the "other direction"
apparently.

Bit by bit, I am remembering now just how stupifyingly brain-dead the MS DLL
scheme is.
Back in '87 or so, I wrote a better dynamic linker (for a Unix) in a couple
of days.

[Cameron Laird]

In article <1e**********************@news.easynews.com>,
Jive <so*****@microsoft.com> wrote:

[Jive]

"Martin v. Löwis" <ma****@v.loewis.de> wrote in message
news:41**********************@news.freenet.de...

It's not hard-coded in the linker, but hard-coded in the import library.
So if you link with msvcrt.lib (which might not be the precise name
of the import library - I cannot look up the precise name right now),
msvcrt.lib will in turn refer to msvcr71.dll.

But it makes no difference, no? The problem is that both Python.exe and the
extensions are *compiled* to link with a *particular* crt. (How "dyanamic"
is that?) We could probably kluge around the problem if it were not for the
fact that one crt might perversely define some struct, (FILE, for example),
with difference layouts in different crt DLL's. Right? The header files
contain the poison.

So it would seem that as far as the crt goes, we are at the mercy of the
micro soft ones. They could introduce an incompatible crt at any time.

By the way, I've googled around and found others outside the Python
community who have run into the same problem. In some cases, they have
opted just to stay with VC 6.0. One said, "At least the 6.0 compiler is 2.5
times as fast." Groan.

[Robin Becker]

Cameron Laird wrote:
......

.
Part of the trick is that it demands deep understanding
to detect the antisynergies that arise from the interac-
tions of the DLL, registry, and filesystem schemes. I
know it was only this year that I realized the whole
installation-requires-reboot absurdity is a consequence
of DLL (mis-)design.

That's judgmental of me. What I'm saying is *I* wouldn't
do an OS that way. It's certain, though, that I'll never
extract as much consumer surplus as Microsoft has, so my
evaluations should count for little.

what seems strange is that although the OS hasn't changed we need a
whole new framework just because the compiler underwent a version change.

That would not happen in a *nix style OS because the framework is the
OS. In the M$ world the $ value of causing reams of people to get the
new compiler outways any sense of what would be a decent approach to OS
design etc.

It's sad that people who are otherwise sensible about opensource seem to
be a bit silly about the poisoned apples. There was no rational reason
for me to upgrade to VC 7.x, but now I'm forced to by my preferred language.
--
Robin Becker

[Martin v. Löwis]

Jive wrote:

But it makes no difference, no? The problem is that both Python.exe and the
extensions are *compiled* to link with a *particular* crt.
No, that is not (really) the case. They are compiled to link with
msvcrt.lib, which could, at link time, then become msvcrt.dll,
msvcrt40.dll, or msvcr71.dll.
(How "dyanamic" is that?)
The actual code of the DLL is not provided until run-time - only
at run-time, the precise implementation of the CRT routines is
available - and it indeed may vary from system to system, as different
versions of msvcrt40.dll might be installed (or of python24.dll, for
that matter).
We could probably kluge around the problem if it were not for the
fact that one crt might perversely define some struct, (FILE, for example),
with difference layouts in different crt DLL's. Right?
Wrong. struct FILE binds us the the MS CRT (so the Borland or glibc CRTs
are incompatible), but struct FILE has stayed the same for more than 10
years now, so you can, in principle, mix object files compiled with
different compilers. One would need to check the header files to
determine they are really compatible, but that appears to be the intent,
and Microsoft does not dare to change it (except perhaps for the _ctype
thing, which I don't fully understand).

What really hurts is that the CRT has a number of global variables: the
global FILE objects (FILE 0 to FILE 63, I believe), the global heap,
and the global atexit handlers. If you have multiple CRTs, you get
multiple copies of these globals, and the different routines in the
different CRTs each operate on their own copy of the global variable.
So it would seem that as far as the crt goes, we are at the mercy of the
micro soft ones. They could introduce an incompatible crt at any time.
They could, but they won't. I believe they have long given up changing
or improving the CRT, since that would make things worse in most cases.
The only things they still do is to add new features; the existing ABI
apparently is never touched (again, perhaps except for a _ctype change
I don't understand).
By the way, I've googled around and found others outside the Python
community who have run into the same problem. In some cases, they have
opted just to stay with VC 6.0. One said, "At least the 6.0 compiler is 2.5
times as fast." Groan.

This is what Python did for 2.3. Already at that time, user pressure
was quite high to move to a more recent compiler; with 2.4, it was
clear that we do need some of the new features (in particular,
ongoing commercial availability, and IPv6 support in winsock.h).

Regards,
Martin

[Martin v. Löwis]

Cameron Laird wrote:

Part of the trick is that it demands deep understanding
to detect the antisynergies that arise from the interac-
tions of the DLL, registry, and filesystem schemes. I
know it was only this year that I realized the whole
installation-requires-reboot absurdity is a consequence
of DLL (mis-)design.

Can you elaborate? To me, that problem only originates from
the OS lack of support for deleting open files. If you could
delete a shared libary that is still in use (as you can on
Unix), the put the new version of the DLL in the place, it
would all work fine: new processes would use the new DLL.
Existing processes continue to use the old DLL, which would
be only deleted if the last process using it terminates.
It might be sensible to restart a few long-running processes
so that they also use the new DLL, but apart from that,
installation does need to imply reboot even on a system
that uses DLLs.

Regards,
Martin

[Jive]

"Robin Becker" <ro***@SPAMREMOVEjessikat.fsnet.co.uk> wrote in message
news:41**************@jessikat.fsnet.co.uk...

Cameron Laird wrote:
There was no rational reason
for me to upgrade to VC 7.x, but now I'm forced to by my preferred language. --
Robin Becker

That's the way I feel about it too. Actually, I'm getting pushed toward
Microsoft from other directions, also for no good reason -- and for a whole
lot of computers, not just for the one on my desk.

Open letter to Bill Gates:

Dear Bill,

Okay. You win again. It was silly of me to think Python would help me
escape your clutches. Look at how good I am being now. I am typing this
into an Outlook Express window, using a Microsoft operating system that you
will no doubt find a way to make me replace very soon. You beat me fair and
square. Well, square, anyway. Congratulations. I give up.

Jive

[David Bolen]

"Martin v. Löwis" <ma****@v.loewis.de> writes:

Can you elaborate? To me, that problem only originates from
the OS lack of support for deleting open files. If you could
delete a shared libary that is still in use (as you can on
Unix), the put the new version of the DLL in the place, (...)

Note that at least on NT-based systems, you can at least rename the
existing file out of the way while it is in use in order to install a
new version.

I do think however, that until more recent changes (not sure whether
in 2K versus XP) in how DLL searches work (e.g., permitting local
versions), even with that operation, if a named DLL was available
loaded into memory, it would be used by a subsequent process
attempting to reference it regardless of the state of the filesystem.

-- David