473,320 Members | 2,146 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Using python for _large_ projects like IDE

Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?

Also, any suggestions on reducing memory usage of big applications
written in python?
Jul 18 '05 #1
8 2075
Sridhar R:
I am a little experienced python programmer (2 months). [...]
I am planning (now in design stage) to write an IDE
I've never developed an IDE, but it wouldn't be my first attempt to write
an app. It's huge and complex.

I started with prime numbers, tic-tac-toe, towers of hanoi, traveling
sales man, merge sort, and little things like that :-)
Are there any ways to forsee the performance critical parts?
Sure, no problem. Post the design please.
Also, any suggestions on reducing memory usage of big applications
written in python?


Only the obvious: don't keep more data alive than strictly necessary.

--
René Pijlman
Jul 18 '05 #2
| Sridhar R said |
1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.
Well, it depends on your definition of "high memory usage." I had no
problems running GUI python apps on my old pentium 100mhz with 74 megs of
ram. So, unless your program should be able to run on smaller systems, I
wouldn't worry much.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?


Data structures are easier to handle than you'd think when extending
python. I recommend either boost or pyrex. I've used pyrex, but most
people seem to use boost.

There are great python profiling tools. It's not at all difficult to find
the performance critical portions.

As far as advice on forseeing performance critical parts, keep in mind
that there are different standards for different portions of a program.
An IDE would have many parts where interactivity is a concern. Noone
likes to see their keystrokes appear two seconds after they have been
typed. Things like syntax highlighting can form a bottleneck, but I've
seen enough apps that manipulate data like this that I know it can be
done.

There are other parts, such as search-and-replace that people are a bit
more patient about. Try to separate the two types of functionality so
that they can be recoded independently.

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #3
Sridhar R wrote:
I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following


Don't bother, there are many good IDEs that can be used with python,
there are even more excellent text editors that (IMO) are better
than all IDEs that I tried. Why waste you time?

i.
Jul 18 '05 #4
If you do write this IDE, please include a compiler warning that catches
the missing () at the end of a function that doesn't have arguments. I've
sunk into a dark world of a diabolically mind-destroying hysteria while
trying to troubleshoot some scripts, and then realizing that a python
function was not executing because I was calling it without '()'. But
Python doesn't complain, or give you a message or anything,... it simply
doesn't execute that function!

Big reason I forget the empty parenthesis: I do Delphi also, and it doesn't
require it.

(Meanwhile, for Python I'm using Komodo, which is really good. Funny how I
keep upgrading UltraEdit when I go ahead and register more language-specific
tools anyway )

"Sridhar R" <sr*************@yahoo.com> wrote in message
news:93**************************@posting.google.c om...
Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?

Also, any suggestions on reducing memory usage of big applications
written in python?

Jul 18 '05 #5
djw
I would take a look at Eric. Its written in Python, is fully featured
and is plenty fast. In fact, there are plenty of IDEs written in Python
or fo Python, why would you want to create yet another one? Also, as a
beginning Python programmer, I fear you may be getting in over your head
(I'm a 2+ year Python and 10+ year C/C++ developer and would be hesitant
to tackle an IDE).

Anyway, Eric is here:

http://www.die-offenbachs.de/detlev/eric3.html

-Don

Sridhar R wrote:
Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
2. if pure python is used, the IDE will be slower.

I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
Are there any ways to forsee the performance critical parts?

Also, any suggestions on reducing memory usage of big applications
written in python?


Jul 18 '05 #6
| DilbertFan said |
If you do write this IDE, please include a compiler warning that catches
the missing () at the end of a function that doesn't have arguments. I've
sunk into a dark world of a diabolically mind-destroying hysteria while
trying to troubleshoot some scripts, and then realizing that a python
function was not executing because I was calling it without '()'. But
Python doesn't complain, or give you a message or anything,... it simply
doesn't execute that function!


Try PyChecker. It should warn you about such things.
http://pychecker.sourceforge.net/

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #7
Thanks Sam
"Samuel Walters" <sw*************@yahoo.com> wrote in message
news:pa****************************@yahoo.com...
| DilbertFan said |
If you do write this IDE, please include a compiler warning that catches the missing () at the end of a function that doesn't have arguments. I've sunk into a dark world of a diabolically mind-destroying hysteria while
trying to troubleshoot some scripts, and then realizing that a python
function was not executing because I was calling it without '()'. But
Python doesn't complain, or give you a message or anything,... it simply
doesn't execute that function!


Try PyChecker. It should warn you about such things.
http://pychecker.sourceforge.net/

HTH

Sam Walters.

--
Never forget the halloween documents.
http://www.opensource.org/halloween/
""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

Jul 18 '05 #8
Sridhar R wrote:
Hi,

I am a little experienced python programmer (2 months). I am somewhat
experienced in C/C++. I am planning (now in design stage) to write an
IDE in python. The IDE will not be a simple one. I had an idea of
writing the IDE in C/C++, as it is a big project, bcoz of the
following

1. if python is used, then the memory required for running the IDE
will be high.
It's quite hard to tell, since I've never seen anybody writing a significant
application *both* in C/C++ and Python to compare their memory usage... I'd say
memory usage depends a lot more on the design than on the language used. Python
does introduce an extra cost in memory consumption, but the larger the
application, the less you'll notice it.
2. if pure python is used, the IDE will be slower.
An application coded in Python will actually be slower than the same application
coded in C/C++. I won't repeat what others have already said about what *looks*
slow to a user.
I'm not sure whether they are true.

Then I thought of using python for 90% of the code and using C for
other performance critical part. But the problem is in identifying
the performance critical code. People suggest to start with pure
python and then _recode_ the performance critical code after profiling
the original code. But I fear whether there will be a conflit in say
data structures. Not yet expert in extending/embedding python.
We did just that for quite a big application and it works like a charm: the
critical code is usually one that does many calculations and it's usually quite
easy to isolate this part in a module handling only base types. And even if it's
not true, as somebody already said, it's easier to handle Python objects at C
level that it may seem at first sight.
Are there any ways to forsee the performance critical parts?
I really think you shouldn't care; remember, "premature optimization is the root
of all evil" ;-)
Also, any suggestions on reducing memory usage of big applications
written in python?


Design them keeping this problem in mind if you really have to. But...
"premature optimization ...". You'll always be able to make compromises on a
well designed application afterwards to reduce memory usage; it's indeed a lot
easier than to maintain an application originally badly designed for bad reasons
(e.g. reduce memory consumption).

HTH
--
- Eric Brunel <eric dot brunel at pragmadev dot com> -
PragmaDev : Real Time Software Development Tools - http://www.pragmadev.com

Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

11
by: Brett C. | last post by:
For my thesis (once the bloody thing stops throwing bugs at me) I am going to need to collect stats on the frequency that atomic types in local variables are applied to various opcodes and methods....
36
by: Andrea Griffini | last post by:
I did it. I proposed python as the main language for our next CAD/CAM software because I think that it has all the potential needed for it. I'm not sure yet if the decision will get through, but...
1
by: Dark Cowherd | last post by:
The Python language is at ver 2.4 and a thing of beauty. As a development environment IMHO it is probably 0.4 I really like what I read when I say "import this" in Python. But as a development...
8
by: wen | last post by:
due to the work reason, i have to learn python since last month. i have spent 1 week on learning python tutorial and felt good. but i still don't understand most part of sourcecode of...
18
by: spiffo | last post by:
The Main Issue in a nutshell I am a corporate developer, working for a single company. Got a new project coming up and wondering if I should stay with Python for this new, fairly large project,...
10
by: TokiDoki | last post by:
Hello there, I have been programming python for a little while, now. But as I am beginning to do more complex stuff, I am running into small organization problems. It is possible that what I...
2
by: Carl J. Van Arsdall | last post by:
I'm aware of a couple python projects for embedded systems. I am currently considering using Python on an embedded platform to develop a simple application as a personal project, mostly to see if...
27
by: John J. Lee | last post by:
I'm surprised to read this: http://en.wikipedia.org/wiki/Python_3 """Note that while there is no explicit requirement that code be able to run unmodified in both versions, in practice it is...
4
by: kj | last post by:
I'm looking for "example implementations" of small projects in Python, similar to the ones given at the end of most chapters of The Perl Cookbook (2nd edition, isbn: 0596003137). (Unfortunately,...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.