By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,156 Members | 1,004 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,156 IT Pros & Developers. It's quick & easy.

SWIG overhead

P: n/a
Hi,
Im looking for some benchmarks comparing SWIG generated modules with
modules made directly with C/Python API. Just how much overhead does
SWIG give? Doing profile of my code I see, that it spends quiet some
time in functions like _swig_setattr_nondinamic, _swig_setattr,
_swig_getattr.

Feb 1 '07 #1
Share this Question
Share on Google+
10 Replies


P: n/a
On 1 Feb 2007 02:21:35 -0800, Bart Ogryczak <B.********@gmail.comwrote:
Hi,
Im looking for some benchmarks comparing SWIG generated modules with
modules made directly with C/Python API. Just how much overhead does
SWIG give? Doing profile of my code I see, that it spends quiet some
time in functions like _swig_setattr_nondinamic, _swig_setattr,
_swig_getattr.
Before you decide to go "low level" consider to use Boost.Python.
According to this( http://tinyurl.com/322d3p ) post it gives pretty
good performance.

P.S. The post does not contain numbers

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
Feb 1 '07 #2

P: n/a
On Thursday 01 February 2007 10:21 am, Bart Ogryczak wrote:
Hi,
Im looking for some benchmarks comparing SWIG generated modules with
modules made directly with C/Python API. Just how much overhead does
SWIG give? Doing profile of my code I see, that it spends quiet some
time in functions like _swig_setattr_nondinamic, _swig_setattr,
_swig_getattr.
There was a EuroPython paper describing some benchmarks. It's from 2004 so
things have probably changed a bit (SIP is now fully documented for example).

http://people.web.psi.ch/geus/talks/...n2004_geus.pdf

Phil
Feb 1 '07 #3

P: n/a
On Feb 1, 12:12 pm, Phil Thompson <p...@riverbankcomputing.co.uk>
wrote:
On Thursday 01 February 2007 10:21 am, Bart Ogryczak wrote:
Hi,
Im looking for some benchmarks comparing SWIG generated modules with
modules made directly with C/Python API. Just how much overhead does
SWIG give? Doing profile of my code I see, that it spends quiet some
time in functions like _swig_setattr_nondinamic, _swig_setattr,
_swig_getattr.

There was a EuroPython paper describing some benchmarks. It's from 2004 so
things have probably changed a bit (SIP is now fully documented for example).

http://people.web.psi.ch/geus/talks/...n2004_geus.pdf
Yeah, found that one googling around. But I havent fund anything more
up to date. I imagine, that the performance of all of these wrappers
has been improved since then. But the performance of Python/C API
would too?
Anyways, its not about exact number, its more about taking decision
if doing rewrite is worth its time.

Feb 1 '07 #4

P: n/a
Yeah, found that one googling around. But I haven´t fund anything more
up to date. I imagine, that the performance of all of these wrappers
has been improved since then. But the performance of Python/C API
would too?
Anyways, it´s not about exact number, it´s more about taking decision
if doing rewrite is worth it´s time.
The wrappers essentially create the boilerplate-code that invokes the Python
C-API. So whatever improvements the latter has been developed, the wrappers
will benefit from it. I doubt that there are major performance penalties
associated with any of them.

More important for a wrapper-decision is the question how easy they are to
use.
Diez

Feb 1 '07 #5

P: n/a
On Feb 1, 12:48 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Yeah, found that one googling around. But I havent fund anything more
up to date. I imagine, that the performance of all of these wrappers
has been improved since then. But the performance of Python/C API
would too?
Anyways, its not about exact number, its more about taking decision
if doing rewrite is worth its time.

The wrappers essentially create the boilerplate-code that invokes the Python
C-API. So whatever improvements the latter has been developed, the wrappers
will benefit from it.
Without doubt its true in case of SWIG, but if I understand
Python.Boost documentation correctly, it does *not* use Python/C API.
I doubt that there are major performance penalties associated with any ofthem.
Take a look at pages 23 and 24 of http://people.web.psi.ch/geus/talks/
europython2004_geus.pdf
More important for a wrapper-decision is the question how easy they are to use.
Well, SWIG is easy to use. But Ive gotta make hundreds of millions of
calls, which do tasks as simple, as getting one int from an array and
returning it. With functions that simple SWIGs overhead seems to be a
problem.
Feb 1 '07 #6

P: n/a
Bart Ogryczak wrote:
On Feb 1, 12:48 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Yeah, found that one googling around. But I haven´t fund anything more
up to date. I imagine, that the performance of all of these wrappers
has been improved since then. But the performance of Python/C API
would too?
Anyways, it´s not about exact number, it´s more about taking decision
if doing rewrite is worth it´s time.

The wrappers essentially create the boilerplate-code that invokes the
Python C-API. So whatever improvements the latter has been developed, the
wrappers will benefit from it.

Without doubt it´s true in case of SWIG, but if I understand
Python.Boost documentation correctly, it does *not* use Python/C API.
It has to. In the end, marshalling data between e.g. python datastructures
and C++ is done that way.
>I doubt that there are major performance penalties associated with any of
them.

Take a look at pages 23 and 24 of http://people.web.psi.ch/geus/talks/
europython2004_geus.pdf
Ok, I see. I really wonder what SWIG does.
Well, SWIG is easy to use. But I´ve gotta make hundreds of millions of
calls, which do tasks as simple, as getting one int from an array and
returning it. With functions that simple SWIG´s overhead seems to be a
problem.
Still I think you should first use wrappers for ease of use. Then when you
hit a performance bottleneck, it might be worth wrapping that class
manually. However, it _might_ of course be that this isn't integrating too
seamless with the wrapped classes, but I can't say anything about that
really.

Diez
Feb 1 '07 #7

P: n/a
On 2/1/07, Diez B. Roggisch <de***@nospam.web.dewrote:
Bart Ogryczak wrote:
On Feb 1, 12:48 pm, "Diez B. Roggisch" <d...@nospam.web.dewrote:
Yeah, found that one googling around. But I havent fund anything more
up to date. I imagine, that the performance of all of these wrappers
has been improved since then. But the performance of Python/C API
would too?
Anyways, its not about exact number, its more about taking decision
if doing rewrite is worth its time.

The wrappers essentially create the boilerplate-code that invokes the
Python C-API. So whatever improvements the latter has been developed, the
wrappers will benefit from it.
Without doubt its true in case of SWIG, but if I understand
Python.Boost documentation correctly, it does *not* use Python/C API.

It has to. In the end, marshalling data between e.g. python datastructures
and C++ is done that way.
As I understand it, part of the Boost.Python internals is a C++
wrapper over the Python C api, and there's no separate code generation
phase because it uses template magic to generate the wrappers. So
while obviously the C API is used at some level, it's not visible to
the wrapper author.
I doubt that there are major performance penalties associated with anyof
them.
Take a look at pages 23 and 24 of http://people.web.psi.ch/geus/talks/
europython2004_geus.pdf

Ok, I see. I really wonder what SWIG does.
SWIG generates a low-level C module, and then a Python one on top of
it. It allows for quite some versatility in wrapping (because you can
have (almost) arbitrary Python code generated). Most of the other
tools generate a .pyd which you import directly.
Well, SWIG is easy to use. But Ive gotta make hundreds of millions of
calls, which do tasks as simple, as getting one int from an array and
returning it. With functions that simple SWIGs overhead seems to be a
problem.

Still I think you should first use wrappers for ease of use. Then when you
hit a performance bottleneck, it might be worth wrapping that class
manually. However, it _might_ of course be that this isn't integrating too
seamless with the wrapped classes, but I can't say anything about that
really.
Personally, I'd favor *correctness* first, then ease of use, and then speed..
Diez
--
http://mail.python.org/mailman/listinfo/python-list
Feb 1 '07 #8

P: n/a
On Feb 1, 3:21 am, "Bart Ogryczak" <B.Ogryc...@gmail.comwrote:
Hi,
Im looking for some benchmarks comparing SWIG generated modules with
modules made directly with C/Python API. Just how much overhead does
SWIG give? Doing profile of my code I see, that it spends quiet some
time in functions like _swig_setattr_nondinamic, _swig_setattr,
_swig_getattr.

This isn't exactly what you're looking for, but I tried SWIG a while
ago, back when I still had a C compiler, and then decided to use c-
types.

The reason: I've got an aversion to anything that generates huge
steaming piles of incomprehensible code. Especially when I found a
couple of bugs in a SWIG-generated module I'd gotten from someone
else. This wasn't SWIG's fault, it was the author's for not
understanding the API he was wrapping properly, but I had to dig into
the SWIG code before I could figure out what was going on.

Bottom line: the c-types module was a lot smaller, in Python, and
completely comprehensible. And while I didn't measure the performance,
I doubt if it was slower.

John Roth

Feb 1 '07 #9

P: n/a

JohnBottom line: the c-types module was a lot smaller, in Python, and
Johncompletely comprehensible. And while I didn't measure the
Johnperformance, I doubt if it was slower.

One advantage SWIG (or Boost.Python) has over ctypes is that it will work
with C++.

Skip

Feb 1 '07 #10

P: n/a
On 2/1/07, Chris Mellon <ar*****@gmail.comwrote:
As I understand it, part of the Boost.Python internals is a C++
wrapper over the Python C api,
That's true.
>and there's no separate code generation
phase because it uses template magic to generate the wrappers.
Well, actually it depends on the size of the project. If project is
small you can use
Boost.Python without a code generator. For big projects you have to use the code
generator. The advantage is that generated code is pretty readable.
So
while obviously the C API is used at some level, it's not visible to
the wrapper author.
Boost.Python hides it pretty well :-)

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
Feb 1 '07 #11

This discussion thread is closed

Replies have been disabled for this discussion.