473,837 Members | 1,603 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Python for large projects

Hello,

I am beginning to work on a fairly large project and I'm considering to use python for most of the coding, but I need to make sure first that it is reliable enough.

I need to make sure that I won't have surprises when my program runs on different real-world systems. So far I wrote a little script with python using urllib, and on one computer it failed completely because of a problem in getting the proxies (in my opinion this is a bug). How likely are such things to happen and how often, and to what extent are they more prevalent in python in comparison to C++?

If python is indeed suitable for large projects, how common is it to actually use if for such purposes? Is there perhaps a list of examples of real projects using python?

Thanks,

Bob

-----------------------------------------------------------------------
Walla! Mail, Get Your Private, Free E-mail from Walla! at:
http://mail.walla.co.il
Jul 18 '05 #1
23 3423
as*****@walla.c om wrote:
If python is indeed suitable for large projects, how common is it to
actually use if for such purposes? Is there perhaps a list of examples of
real projects using python?


I would say that Python is very suitable for use in large projects, just the
benefit of automatic memory management is enough to convince me that it
beats C++. The powerful builtin types are another big win - what you would
spend ages coding in C++ is often a no brainer in Python.

Obviously there are downsides, I find the lack of static type checking to
cause some bugs to hide in my code that in other languages would be found
at compile time. In general (and IMHO) though, the pros far outway cons.

There is a decent sized list of companies that use Python at
http://pythonology.org/success.

HTH,
Andrew
Jul 18 '05 #2
as*****@walla.c om writes:
I am beginning to work on a fairly large project and I'm considering
to use python for most of the coding, but I need to make sure first
that it is reliable enough.

I need to make sure that I won't have surprises when my program runs
on different real-world systems. So far I wrote a little script with
python using urllib, and on one computer it failed completely
because of a problem in getting the proxies (in my opinion this is a
bug). How likely are such things to happen and how often, and to
what extent are they more prevalent in python in comparison to C++?
The Python language is in general well-designed and much more concise
than C++. A big program in C++ may map to a much smaller Python
program, turning a large project into a small project, and making it
less relevant whether Python works for large projects.

Python's library does have a lot of small gaps like the one you found
in urllib. As the Twisted Matrix documentation puts it, you find
yourself re-inventing the wheel a lot, because you discover that the
existing wheels are often square and made of glue.
If python is indeed suitable for large projects, how common is it to
actually use if for such purposes? Is there perhaps a list of
examples of real projects using python?


The canonical example of a complex Python project is Zope
(www.zope.com). It's medium sized by the standards of big C++
projects, but as mentioned, a medium amount of Python code can
implement functionality that would take a much larger amount of C++.

Because Python is interpreted and highly dynamic, Python programs tend
to run slower than comparable C++ programs. Whether that's a problem
for you depends on your application. If you have specific Python
functions that are bottlenecks, you can re-implement them in C and
call them through Python's C API. There's also a semi-experimental
native-code Python compiler called Psyco that produces a considerable
speedup at the cost of increased memory consumption. The
next-generation Python implementation (PyPy or Python in Python) will
reportedly use Psyco or something simliar, in a more fundamental way.

In short, there's not a quick and simple answer to your question of
whether Python is right for what you're doing. It's great for lots of
things, not so hot for some others, and is still evolving rather
quickly, so an unsuitable application today may become suitable in a
forthcoming release.
Jul 18 '05 #3
In article <7x************ @ruckus.brouhah a.com>,
Paul Rubin <http://ph****@NOSPAM.i nvalid> wrote:
as*****@walla. com writes:
I am beginning to work on a fairly large project and I'm considering
to use python for most of the coding, but I need to make sure first
that it is reliable enough.

I need to make sure that I won't have surprises when my program runs
on different real-world systems. So far I wrote a little script with
python using urllib, and on one computer it failed completely
because of a problem in getting the proxies (in my opinion this is a
bug). How likely are such things to happen and how often, and to
what extent are they more prevalent in python in comparison to C++?


The Python language is in general well-designed and much more concise
than C++. A big program in C++ may map to a much smaller Python
program, turning a large project into a small project, and making it
less relevant whether Python works for large projects.

Python's library does have a lot of small gaps like the one you found
in urllib. As the Twisted Matrix documentation puts it, you find
yourself re-inventing the wheel a lot, because you discover that the
existing wheels are often square and made of glue.
If python is indeed suitable for large projects, how common is it to
actually use if for such purposes? Is there perhaps a list of
examples of real projects using python?


The canonical example of a complex Python project is Zope
(www.zope.com). It's medium sized by the standards of big C++
projects, but as mentioned, a medium amount of Python code can
implement functionality that would take a much larger amount of C++.

Because Python is interpreted and highly dynamic, Python programs tend
to run slower than comparable C++ programs. Whether that's a problem
for you depends on your application. If you have specific Python
functions that are bottlenecks, you can re-implement them in C and
call them through Python's C API. There's also a semi-experimental
native-code Python compiler called Psyco that produces a considerable
speedup at the cost of increased memory consumption. The
next-generation Python implementation (PyPy or Python in Python) will
reportedly use Psyco or something simliar, in a more fundamental way.

In short, there's not a quick and simple answer to your question of
whether Python is right for what you're doing. It's great for lots of
things, not so hot for some others, and is still evolving rather
quickly, so an unsuitable application today may become suitable in a
forthcoming release.


You might also have occasion to learn about Pyrex and
.... well, I'm in the Trotskyite wing on this question.
It's not just that Python can work for large projects.
I sincerely regard it as an even *better* comparative
choice on large projects; C++ and Java, the usual com-
petition, show all sorts of blemishes when one scales
the size of the project. Python remains usable, even
at the high end.
--

Cameron Laird <cl****@phaseit .net>
Business: http://www.Phaseit.net
Jul 18 '05 #4
cl****@lairds.c om (Cameron Laird) writes:
You might also have occasion to learn about Pyrex and ... well, I'm
in the Trotskyite wing on this question. It's not just that Python
can work for large projects. I sincerely regard it as an even
*better* comparative choice on large projects; C++ and Java, the
usual com- petition, show all sorts of blemishes when one scales the
size of the project. Python remains usable, even at the high end.


I keep hearing this, but I don't see any large (much less very large)
applications that have been done in Python; Zope is medium sized.

Suppose you wanted to write any of the following:

1) Optimizing C/C++ compiler, like GCC
2) Full featured web browser, like MSIE or Mozilla
3) Full featured office suite, like MS Office or Open Office or KDE
4) Avionics for the space shuttle
5) Internals of a large telephone/data switch
6) Tax processing software for the IRS
7) Operating system kernel (Linux: the next generation)
8) Accounting software for a big bank
9) Full featured database like Oracle or Postgres
10) ... well you get the idea.

Which, if any, would you write in Python? By "write in Python" I mean
the central framework and most of the code is written in Python,
though you're allowed to use the C API as needed. Using Python to
provide some functions around the edges (e.g. the operator UI for the
phone switch) doesn't count. Note that these are all supposed to be
used in production and not just as technology demos, so speed matters
(nobody wants a compiler that's 10x slower than GCC. GCC is slow
enough already).

I think I'd spend some time at least considering Python in each of the
above cases, but I'm not sure I could convincingly make it fly for any
of them.
Jul 18 '05 #5
Paul Rubin wrote:
1) Optimizing C/C++ compiler, like GCC
2) Full featured web browser, like MSIE or Mozilla
3) Full featured office suite, like MS Office or Open Office or KDE
4) Avionics for the space shuttle
5) Internals of a large telephone/data switch
6) Tax processing software for the IRS
7) Operating system kernel (Linux: the next generation)
8) Accounting software for a big bank
9) Full featured database like Oracle or Postgres
10) ... well you get the idea.

Which, if any, would you write in Python? By "write in Python" I mean


Which, if any, would you write in JAVA or C#? I think for a project in
"userland" thats not on the number crunching or machine architecture side
like a kernel2, python often would be a viable alternative.

And the spaceshuttle runs on a 68K processor - I bet a modern 3GHz processor
running python can beat that :)
--
Regards,

Diez B. Roggisch
Jul 18 '05 #6
On 22 Mar 2004 15:37:27 -0800,
Paul Rubin <http://ph****@NOSPAM.i nvalid> wrote:
1) Optimizing C/C++ compiler, like GCC
2) Full featured web browser, like MSIE or Mozilla
3) Full featured office suite, like MS Office or Open Office or KDE
4) Avionics for the space shuttle
5) Internals of a large telephone/data switch
6) Tax processing software for the IRS
7) Operating system kernel (Linux: the next generation)
8) Accounting software for a big bank
9) Full featured database like Oracle or Postgres
10) ... well you get the idea. Which, if any, would you write in Python? ... Note that these
are all supposed to be used in production and not just as
technology demos, so speed matters ...


With the "speed counts" caveat, that's a bit of a trick question,
trivially answered with None Of The Above (okay, so maybe most
bankers (application 8) are in less of a hurry than most user
processes (application 7)).

With sufficiently powerful hardware and sufficiently patient
clients, I'd take Python over C for any of those projects except:

Life/death applications, like 4, and maybe 5, and maybe 7
(depends on who's using the OS and for what). Python is not
well-specified enough (too many things depend on the
underlying C library) for this type of application.

Hardware-oriented applications, like maybe parts of 4 and 5
(depending on what the interfaces to the hardware look like),
and the low-level pieces of 7.

Specifically, I can't think of a single advantage C has over
Python (except speed and footprint) for applications 1, 2, 3, 5,
6, 8, and 9.

What advantages *do* C and Java have for such large projects? How
many bug-free, meets-the-full-spec, on-time, under-budget examples
of applications written in those languages do we have?

Regards,
Heather

--
Heather Coppersmith
That's not right; that's not even wrong. -- Wolfgang Pauli
Jul 18 '05 #7
Paul Rubin wrote:
cl****@lairds.c om (Cameron Laird) writes:
You might also have occasion to learn about Pyrex and ... well, I'm
in the Trotskyite wing on this question. It's not just that Python
can work for large projects. I sincerely regard it as an even
*better* comparative choice on large projects; C++ and Java, the
usual com- petition, show all sorts of blemishes when one scales the
size of the project. Python remains usable, even at the high end.

I keep hearing this, but I don't see any large (much less very large)
applications that have been done in Python; Zope is medium sized.

Suppose you wanted to write any of the following:

1) Optimizing C/C++ compiler, like GCC
2) Full featured web browser, like MSIE or Mozilla
3) Full featured office suite, like MS Office or Open Office or KDE
4) Avionics for the space shuttle
5) Internals of a large telephone/data switch
6) Tax processing software for the IRS
7) Operating system kernel (Linux: the next generation)
8) Accounting software for a big bank
9) Full featured database like Oracle or Postgres
10) ... well you get the idea.

Which, if any, would you write in Python?


6 and 8, maybe 3...and 10 of course

Take care,
Jay

Jul 18 '05 #8
Heather Coppersmith <me@privacy.net > writes:
What advantages *do* C and Java have for such large projects? How
many bug-free, meets-the-full-spec, on-time, under-budget examples
of applications written in those languages do we have?


It was a real question, not a rhetorical one, and your answers are
reasonable.

We may not have C or Java examples of those apps that are on-time and
under budget, but at least we have examples now that are deployed and
that work, even if they were delivered late and over budget. We don't
have ANY examples for programs like that in Python, whether delivered
on time or not. On the other hand, we haven't been writing stuff like
that in Python for as long.

I'd like to think that a suitable implementation of Python, or
something pretty closely resembling Python as we know it, could do
pretty much anything Common Lisp can do, and CL has been used for some
very big apps. However, the current Python implementations seem aimed
at smaller projects.
Jul 18 '05 #9
Jay O'Connor <ja********@ear thlink.net> writes:
1) Optimizing C/C++ compiler, like GCC
2) Full featured web browser, like MSIE or Mozilla
3) Full featured office suite, like MS Office or Open Office or KDE
4) Avionics for the space shuttle
5) Internals of a large telephone/data switch
6) Tax processing software for the IRS
7) Operating system kernel (Linux: the next generation)
8) Accounting software for a big bank
9) Full featured database like Oracle or Postgres
10) ... well you get the idea.
Which, if any, would you write in Python?


6 and 8, maybe 3...and 10 of course


I think 6 and 8 would depend on decimal arithmetic; some modules for
that have been proposed but nothing is released yet, afaik.

I can't see answering 3 without also answering 2.
Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

42
3733
by: Bicho Verde | last post by:
I have now free time and money to do what I want :-) I have some basic skills in programming (C, Pascal, Macromedia Actionscript) but don't know exactly what to do in the world of programming. And also I don't know exactly why would I learn Python rather than C#, C++ or Perl. Basicaly I don't know where to start, if there is much to do or if it is has it seems and there is software to everything nowadays and so doesn't make sense to spend...
42
4133
by: Fred Ma | last post by:
Hello, This is not a troll posting, and I've refrained from asking because I've seen similar threads get all nitter-nattery. But I really want to make a decision on how best to invest my time. I'm not interested on which language is better in *general*, just for my purpose. My area of research is in CAD algorithms, and I'm sensing the need to resort to something more expedient than C++, bash scripting, or sed scripting.
9
2430
by: limor | last post by:
Hi, I am considering using Python in a new testing tool application we intend to build for out product. I must get references before starting develope in this language , since although lots of good things are said about this language , I still have my doubts how can it compete with languages like C++ and Java. I have found te sytax and some features if the language not as problematic in maintenance prospective (although I keep reading...
7
2409
by: Ixokai | last post by:
Hello all. :) I've been a long time Python fan, and have fairly recently (with the support of a coworker who surprised me by mentioning my pet language one day) convinced my company to begin the colossal task of basically rewriting all of our software in Python. Woohoo. Previously we used a few different development environments, mostly Borland, for different products in our 'system' of thick clients sort of operating with eachother as...
36
6423
by: Andrea Griffini | last post by:
I did it. I proposed python as the main language for our next CAD/CAM software because I think that it has all the potential needed for it. I'm not sure yet if the decision will get through, but something I'll need in this case is some experience-based set of rules about how to use python in this context. For example... is defining readonly attributes in classes worth the hassle ? Does duck-typing scale well in complex
1
2602
by: Adriaan Renting | last post by:
I think the point you want to make is that Python needs vastly less lines of code as a similar application written in C++. I think Python might on average be 50-60% of comparable C++ code, but not the 1-2% you seem to want to claim. LOC is a somewhat arbitrairy measurement, but it gives an idea of what I'm talking about. Your single example compares a simple use in Python with a much more elaborate framework in one of the worst C++ IDE's...
10
9931
by: TokiDoki | last post by:
Hello there, I have been programming python for a little while, now. But as I am beginning to do more complex stuff, I am running into small organization problems. It is possible that what I want to obtain is not possible, but I would like the advice of more experienced python programmers. I am writing a relatively complex program in python that has now around 40 files.
267
10906
by: Xah Lee | last post by:
Python, Lambda, and Guido van Rossum Xah Lee, 2006-05-05 In this post, i'd like to deconstruct one of Guido's recent blog about lambda in Python. In Guido's blog written in 2006-02-10 at http://www.artima.com/weblogs/viewpost.jsp?thread=147358
162
10350
by: Sh4wn | last post by:
Hi, first, python is one of my fav languages, and i'll definitely keep developing with it. But, there's 1 one thing what I -really- miss: data hiding. I know member vars are private when you prefix them with 2 underscores, but I hate prefixing my vars, I'd rather add a keyword before it. Python advertises himself as a full OOP language, but why does it miss one of the basic principles of OOP? Will it ever be added to python?
0
9852
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9696
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
10642
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9420
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7824
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7014
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5680
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5863
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4481
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.