473,847 Members | 1,495 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Building python packages for the correct architecture on OSX 10.5

Hi fellow python enthusiasts.

Having recently acquired a MacBook Pro (Intel Core 2 Duo) which comes
with python2.5, I have been installing some modules that I need (PIL,
psycopg2, PyXML ...).

The problem is that [$python setup.py build] compiles all the binaries
to universal files for i386 and ppc32, but not x86_64 or ppc64. It
does not appear to be a problem when running scripts from the shell
(as python seems to run as a 32 bits problems), but it is a problem
from apache2/mod_python as the included apache2 runs as 64 bits
processes.

This means the modules need to be compiles for at least both i386 and
x86_64 in my case. I have been looking at the setup.py files of
various modules but I cannot see a suitable way to indicate what
architectures I want them compiled for. So far, I have managed by
adding the following lines in setup.py just after the Extension class
is imported:

OrigExtension = Extension
def Extension(*args , **kwargs):
extra_args = ['-arch', 'ppc', '-arch', 'ppc64',
'-arch', 'i386', '-arch', 'x86_64 ']
kwargs['extra_compile_ args'] = extra_args +
kwargs.get('ext ra_compile_args ', [])
kwargs['extra_link_arg s'] = extra_args +
kwargs.get('ext ra_link_args', [])
return OrigExtension(* args, **kwargs)
Obviously this is a dirty hack, and I would like to know how to do
this the right way. How can this be done better?

--
Arnaud

Nov 14 '07 #1
4 3155
Arnaud Delobelle wrote:
Hi fellow python enthusiasts.

Having recently acquired a MacBook Pro (Intel Core 2 Duo) which comes
with python2.5, I have been installing some modules that I need (PIL,
psycopg2, PyXML ...).

The problem is that [$python setup.py build] compiles all the binaries
to universal files for i386 and ppc32, but not x86_64 or ppc64. It
does not appear to be a problem when running scripts from the shell
(as python seems to run as a 32 bits problems), but it is a problem
from apache2/mod_python as the included apache2 runs as 64 bits
processes.

This means the modules need to be compiles for at least both i386 and
x86_64 in my case. I have been looking at the setup.py files of
various modules but I cannot see a suitable way to indicate what
architectures I want them compiled for. So far, I have managed by
adding the following lines in setup.py just after the Extension class
is imported:

OrigExtension = Extension
def Extension(*args , **kwargs):
extra_args = ['-arch', 'ppc', '-arch', 'ppc64',
'-arch', 'i386', '-arch', 'x86_64 ']
kwargs['extra_compile_ args'] = extra_args +
kwargs.get('ext ra_compile_args ', [])
kwargs['extra_link_arg s'] = extra_args +
kwargs.get('ext ra_link_args', [])
return OrigExtension(* args, **kwargs)
Obviously this is a dirty hack, and I would like to know how to do
this the right way. How can this be done better?

--
Arnaud
You may want to post this on the MacPython list--there are plenty of
experts there on building Python mudles for OS X.

--
Kevin Walzer
Code by Kevin
http://www.codebykevin.com
Nov 14 '07 #2
This means the modules need to be compiles for at least both i386 and
x86_64 in my case.
Building Python in 64-bit mode as a universal (fat) binary is not
supported in Python 2.5, period. So any solution you come necessarily
has to be a work-around.

The only solution I can see is to make a plain, non-fat installation
of Python in 64-bit mode, and then use that installation to build
64-bit extension modules.
def Extension(*args , **kwargs):
extra_args = ['-arch', 'ppc', '-arch', 'ppc64',
'-arch', 'i386', '-arch', 'x86_64 ']
This cannot really work, for two reasons:
a) even if your extension module becomes x86_64 with that mechanism,
the Python interpreter itself (i.e. the Python framework) will be
purely 32-bit code. So it should not link correctly.
b) During configure, Python generates a pyconfig.h which has the
computed sizes of data types (such as int, long, size_t). It only
has a single such file, and the file is generated only during
configure. Therefore, the data in it cannot work both for 32-bit
and 64-bit architectures. When you compile for a 64-bit target
using the 32-bit pyconfig.h, the code may work incorrectly
(provided it makes use of the computed values somewhere) (*)

(*) It is surprising that pyconfig.h actually works for both
big-endian (ppc) and little-endian (i386) systems, even though
it computes the endianness during configure only once. This is
due to an OSX-specific hack in pyconfig.h, which hides the
definition of the computed endianness value, and uses the
value that the compiler provides as a macro instead.

Regards,
Martin
Nov 14 '07 #3
On Nov 14, 7:40 pm, "Martin v. Löwis" <mar...@v.loewi s.dewrote:
This means the modules need to be compiles for at least both i386 and
x86_64 in my case.

Building Python in 64-bit mode as a universal (fat) binary is not
supported in Python 2.5, period. So any solution you come necessarily
has to be a work-around.

The only solution I can see is to make a plain, non-fat installation
of Python in 64-bit mode, and then use that installation to build
64-bit extension modules.
def Extension(*args , **kwargs):
extra_args = ['-arch', 'ppc', '-arch', 'ppc64',
'-arch', 'i386', '-arch', 'x86_64 ']

This cannot really work, for two reasons:
a) even if your extension module becomes x86_64 with that mechanism,
the Python interpreter itself (i.e. the Python framework) will be
purely 32-bit code. So it should not link correctly.
My machine disagrees:

marigold:~ arno$ file /System/Library/Frameworks/Python.framewor k/
Python
/System/Library/Frameworks/Python.framewor k/Python: Mach-O universal
binary with 4 architectures
/System/Library/Frameworks/Python.framewor k/Python (for architecture
ppc7400): Mach-O dynamically linked shared library ppc
/System/Library/Frameworks/Python.framewor k/Python (for architecture
ppc64): Mach-O 64-bit dynamically linked shared library ppc64
/System/Library/Frameworks/Python.framewor k/Python (for architecture
i386): Mach-O dynamically linked shared library i386
/System/Library/Frameworks/Python.framewor k/Python (for architecture
x86_64): Mach-O 64-bit dynamically linked shared library x86_64

b) During configure, Python generates a pyconfig.h which has the
computed sizes of data types (such as int, long, size_t). It only
has a single such file, and the file is generated only during
configure. Therefore, the data in it cannot work both for 32-bit
and 64-bit architectures. When you compile for a 64-bit target
using the 32-bit pyconfig.h, the code may work incorrectly
(provided it makes use of the computed values somewhere) (*)

(*) It is surprising that pyconfig.h actually works for both
big-endian (ppc) and little-endian (i386) systems, even though
it computes the endianness during configure only once. This is
due to an OSX-specific hack in pyconfig.h, which hides the
definition of the computed endianness value, and uses the
value that the compiler provides as a macro instead.
Thanks for the details.
I have had no problems with the modules I have compiled so far, they
have been working in 32 and 64 bits. Maybe I was just lucky? I'll
have to look into this more, then. Python on OSX 10.5 has been a
challenge so far :(
Regards,
Martin
Thanks

--
Arnaud
Nov 14 '07 #4
>a) even if your extension module becomes x86_64 with that mechanism,
> the Python interpreter itself (i.e. the Python framework) will be
purely 32-bit code. So it should not link correctly.

My machine disagrees:
I see. I guess Apple has implemented that somehow; the official Python
release does not support such an operation.

Regards,
Martin
Nov 14 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
2336
by: ketulp_baroda | last post by:
Hi Does python support MVC architecture? Java has register & notify obsever methods in javax.util . Does python has these functions. If not then how to register the views with the models & how to notify the views that the model has been updated??
0
434
by: Mike Maxwell | last post by:
First, apologies if this is not the appropriate newsgroup (and let me know what would be a better group). I downloaded the Python 2.3.3, did the ./configure and make routine. 'configure' seems to run OK, but 'make' produces a large number of errors, including "undefined reference to 'PyInt_FromLong'", "undefined reference to 'PyErr_SetString'", and a host of others. Clearly something is wrong, but I'm not sure where to start. One...
1
1755
by: Seo Sanghyeon | last post by:
For those who may be concerned: this mail is sent to comp.lang.python, debian-python, freebsd-python, and catalog-sig. I created a table of Python packages in Debian, FreeBSD, and Gentoo. http://sparcs.kaist.ac.kr/~tinuviel/pypackage/list.cgi The table is created as follow: first I made a list of all packages in Debian Python section, Freebsd Python category, and Gentoo dev-python. And I checked equivalence by hand, alphabetically...
7
2427
by: Erik Johnson | last post by:
I am trying to upgrade my Python installation. After downloading sources and building Python 2.3.4, I am unable to use the command history editing feature in the interactive interpreter (where the up-arrow would previously give you the last command line to edit, it now just prints "^[[A".) This is a feature I use often, and it kinda nullifies that warm fuzzy feeling you get when things are otherwise working as expected. Python 2.2.2 was...
5
2701
by: Fuzzyman | last post by:
Python 2.4 is built with Microsoft Visiual C++ 7. This means that it uses msvcr7.dll, which *isn't* a standard part of the windows operating system. This means that if you build a windows installer using distutils - it *requires* msvcr7.dll in order to run. This is true even if your package is a pure python package. This means that when someone tries to use a windows installer created with Python 2.4, on a machine with only python 2.3 - it...
14
2048
by: Kenneth McDonald | last post by:
At the moment I'm doing most of my stuff on a Mac, but I've been considering also getting a Windows laptop. One of the reasons is that I've found it very difficult to get certain Python related things running on the Mac; for example, if one wants to use the most up-to-date Python on the mac, rather than the one installed by Apple, things can get a bit hairy, and then if one wants to go PyQT/wxPy/etc, it can get even worse with
1
1095
by: redefined.horizons | last post by:
Is it possible to store "non-python" files in a directory that serves as a Python Package? (Like an image file or an XML file.) Is this allowed for sub-directories that are not Python Packages? In other words, can I have a Python Package that contains other Python Packages and also folders that aren't Python Packages? Thanks, Scott Huey
3
1401
by: kyosohma | last post by:
I am trying to figure out how to build binaries for Python packages and I've done it with MinGW. However, in my research, I've noticed that some of the programmers out there think that you should use the compiler that is used to compile the official Python distributions, which appears to be Visual Studio 2003. None of these programmers has really given their reasons for this though. Does compiling with a different compiler introduce...
0
1311
by: Chris | last post by:
Hi, I'm trying to build Python on Windows using Visual Studio 2005. I have no trouble compiling tcl/tk 8.5, but when I subsequently build Python it doesn't find tcl/tk (I get a message about being unable to find tcl.h). I'm using the source distribution of Python 2.5.2, not the svn version. I'm trying to follow the instructions in PCbuild8/readme.txt, but I'm not sure I understand them correctly. Here is the part of the
0
9889
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10988
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10652
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10344
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7886
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7058
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5914
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4536
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4126
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.