471,578 Members | 1,531 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,578 software developers and data experts.

Using python for _large_ projects like IDE

Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?

Also, any suggestions on reducing memory usage of big applications
written in python?
Jul 18 '05 #1
8 1972
Sridhar R:
I am a little experienced python programmer (2 months). [...]
I am planning (now in design stage) to write an IDE
I've never developed an IDE, but it wouldn't be my first attempt to write
an app. It's huge and complex.

I started with prime numbers, tic-tac-toe, towers of hanoi, traveling
sales man, merge sort, and little things like that :-)
Are there any ways to forsee the performance critical parts?
Sure, no problem. Post the design please.
Also, any suggestions on reducing memory usage of big applications
written in python?


Only the obvious: don't keep more data alive than strictly necessary.

--
René Pijlman
Jul 18 '05 #2
| Sridhar R said |
1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.
Well, it depends on your definition of "high memory usage." I had no
problems running GUI python apps on my old pentium 100mhz with 74 megs of
ram. So, unless your program should be able to run on smaller systems, I
wouldn't worry much.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?


Data structures are easier to handle than you'd think when extending
python. I recommend either boost or pyrex. I've used pyrex, but most
people seem to use boost.

There are great python profiling tools. It's not at all difficult to find
the performance critical portions.

As far as advice on forseeing performance critical parts, keep in mind
that there are different standards for different portions of a program.
An IDE would have many parts where interactivity is a concern. Noone
likes to see their keystrokes appear two seconds after they have been
typed. Things like syntax highlighting can form a bottleneck, but I've
seen enough apps that manipulate data like this that I know it can be
done.

There are other parts, such as search-and-replace that people are a bit
more patient about. Try to separate the two types of functionality so
that they can be recoded independently.

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #3
Sridhar R wrote:
I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following


Don't bother, there are many good IDEs that can be used with python,
there are even more excellent text editors that (IMO) are better
than all IDEs that I tried. Why waste you time?

i.
Jul 18 '05 #4
If you do write this IDE, please include a compiler warning that catches
the missing () at the end of a function that doesn't have arguments. I've
sunk into a dark world of a diabolically mind-destroying hysteria while
trying to troubleshoot some scripts, and then realizing that a python
function was not executing because I was calling it without '()'. But
Python doesn't complain, or give you a message or anything,... it simply
doesn't execute that function!

Big reason I forget the empty parenthesis: I do Delphi also, and it doesn't
require it.

(Meanwhile, for Python I'm using Komodo, which is really good. Funny how I
keep upgrading UltraEdit when I go ahead and register more language-specific
tools anyway )

"Sridhar R" <sr*************@yahoo.com> wrote in message
news:93**************************@posting.google.c om...
Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?

Also, any suggestions on reducing memory usage of big applications
written in python?

Jul 18 '05 #5
djw
I would take a look at Eric. Its written in Python, is fully featured
and is plenty fast. In fact, there are plenty of IDEs written in Python
or fo Python, why would you want to create yet another one? Also, as a
beginning Python programmer, I fear you may be getting in over your head
(I'm a 2+ year Python and 10+ year C/C++ developer and would be hesitant
to tackle an IDE).

Anyway, Eric is here:

http://www.die-offenbachs.de/detlev/eric3.html

-Don

Sridhar R wrote:
Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?

Also, any suggestions on reducing memory usage of big applications
written in python?


Jul 18 '05 #6
| DilbertFan said |
If you do write this IDE, please include a compiler warning that catches
the missing () at the end of a function that doesn't have arguments. I've
sunk into a dark world of a diabolically mind-destroying hysteria while
trying to troubleshoot some scripts, and then realizing that a python
function was not executing because I was calling it without '()'. But
Python doesn't complain, or give you a message or anything,... it simply
doesn't execute that function!


Try PyChecker. It should warn you about such things.
http://pychecker.sourceforge.net/

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #7
Thanks Sam
"Samuel Walters" <sw*************@yahoo.com> wrote in message
news:pa****************************@yahoo.com...
| DilbertFan said |
If you do write this IDE, please include a compiler warning that catches the missing () at the end of a function that doesn't have arguments. I've sunk into a dark world of a diabolically mind-destroying hysteria while
trying to troubleshoot some scripts, and then realizing that a python
function was not executing because I was calling it without '()'. But
Python doesn't complain, or give you a message or anything,... it simply
doesn't execute that function!


Try PyChecker. It should warn you about such things.
http://pychecker.sourceforge.net/

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #8
Sridhar R wrote:
Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
It's quite hard to tell, since I've never seen anybody writing a significant
application *both* in C/C++ and Python to compare their memory usage... I'd say
memory usage depends a lot more on the design than on the language used. Python
does introduce an extra cost in memory consumption, but the larger the
application, the less you'll notice it.
2. if pure python is used, the IDE will be slower.
An application coded in Python will actually be slower than the same application
coded in C/C++. I won't repeat what others have already said about what *looks*
slow to a user.
I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
We did just that for quite a big application and it works like a charm: the
critical code is usually one that does many calculations and it's usually quite
easy to isolate this part in a module handling only base types. And even if it's
not true, as somebody already said, it's easier to handle Python objects at C
level that it may seem at first sight.
Are there any ways to forsee the performance critical parts?
I really think you shouldn't care; remember, "premature optimization is the root
of all evil" ;-)
Also, any suggestions on reducing memory usage of big applications
written in python?


Design them keeping this problem in mind if you really have to. But...
"premature optimization ...". You'll always be able to make compromises on a
well designed application afterwards to reduce memory usage; it's indeed a lot
easier than to maintain an application originally badly designed for bad reasons
(e.g. reduce memory consumption).

HTH
--
- Eric Brunel <eric dot brunel at pragmadev dot com> -
PragmaDev : Real Time Software Development Tools - http://www.pragmadev.com

Jul 18 '05 #9

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

36 posts views Thread by Andrea Griffini | last post: by
1 post views Thread by Dark Cowherd | last post: by
8 posts views Thread by wen | last post: by
2 posts views Thread by Carl J. Van Arsdall | last post: by
27 posts views Thread by John J. Lee | last post: by
reply views Thread by leo001 | last post: by
reply views Thread by lumer26 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.