Dynamic features used

[bearophileHUGS]

I often use Python to write small programs, in the range of 50-500
lines of code. For example to process some bioinformatics data,
perform some data munging, to apply a randomized optimization
algorithm to solve a certain messy problem, and many different things.
For that I often use several general modules that I have written, like
implementation of certain data structures, and small general "utility"
functions/classes, plus of course several external modules that I keep
updated.

Compared to other languages Python generally allows me to write a
correctly working program in the shorter time (probably because of the
Python shell, the built-in safeties, the doctests, the clean and short
and handy syntax, the quick write-run-test cycle, the logical design,
its uniformity, the availability of standard data structures and many
standard or external modules, and the low number (compared to other
languages) of corner cases and tricky semantic corners).

Today Python is defined a dynamic language (and not a scripting
language, a term that few languages today seem to want attached to
them) but being dynamic isn't a binary thing, it's an analog quality,
a language can be less or be more dynamic. For example I think Ruby is
more dynamic than Python, Python is more dynamic than CLisp, CLips
seems more dynamic than C#/Java, Java is more dynamic than D, and D is
more dynamic than C++. Often such differences aren't sharp, and you
can find ways to things more dynamically, even with a less nice syntax
(or creating less idiomatic code). (In C#4 they have even added a
dynamic feature that may make languages like IronPython/Boo faster and
simpler to write on the dotnet).

In the last two years I have seen many answers in the Python
newsgroups, and I have seen that some of the dynamic features of
Python aren't much used/appreciated:
- Metaclasses tricks
- exec and eval
- monkey patching done on classes
- arbitrary cmp among different types removed from Python 3
While some new static-looking/related features are being introduced:
- ABCs and function signatures added
- More tidy exception tree

So it seems that while C#/D are becoming more dynamic, Python/
JavaScript are becoming a little less dynamic looking (and this I
think this is positive, because too much dynamism turns code into a
swamp, and too much rigid systems lead to bloat and other problems.
Note that there are another orthogonal solution: to use an advanced
flexible and handy static type system, as in Haskell).

I have seen that in lot of those little programs of mine, or in a
significant percentage of their lines, I often don't use the dynamic
features of Python (this means that the same code can be written with
static types, especially if you can use templates like in C++/D, or a
flexible type system like in Haskell, and it also means that lot of
those small programs can be compiled by ShedSkin/Cython, with usually
a sharp decrease of running time).

What are the dynamic features of Python more used in your programs?
(From this set try to remove the things that can be done with a
flexible static template system, like the D one, that for some things
is more handy and powerful than the C++ template system, and for other
things less powerful).

If very little or no dynamic features are used in a program it may
seem a "waste" to use Python to write the code, because the final
program may be quite slow with no gain from the other features of
Python. (I think in such situations Python can be a good choice
anyway, because it's good to write working prototypes in a short
time). (The large number of solution of this page shows how a certain
class of Python programmers want more speed from their programs:
http://scipy.org/PerformancePython and note that page misses many
other solutions, like SIP, Boost Python, ShedSkin, Cinpy, Cython,
RPython, and so on).

In the last year I have found two situations where exec/eval is a way
to reduce a lot of the complexity of the code, so if used with care
the dynamic features can be quite useful.

Before ending this partially incoherent post, I'd also like to briefly
remind how the dynamic features are used in the Boo language: Boo
programs are generally statically typed, but duck types are used once
in a while to reduce the "pressure" of the static type system. You can
find more info on this on the Boo site. (Note that I have never seen a
good set of speed benchmarks to compare the performance of CPython to
Boo).

Bye,
bearophile

[Rafe]

On Nov 21, 4:17*pm, [e-mail] wrote:

I often use Python to write small programs, in the range of 50-500
lines of code. For example to process some bioinformatics data,
perform some data munging, to apply a randomized optimization
algorithm to solve a certain messy problem, and many different things.
For that I often use several general modules that I have written, like
implementation of certain data structures, and small general "utility"
functions/classes, plus of course several external modules that I keep
updated.

Compared to other languages Python generally allows me to write a
correctly working program in the shorter time (probably because of the
Python shell, the built-in safeties, the doctests, the clean and short
and handy syntax, the quick write-run-test cycle, the logical design,
its uniformity, the availability of standard data structures and many
standard or external modules, and the low number (compared to other
languages) of corner cases and tricky semantic corners).

Today Python is defined a dynamic language (and not a scripting
language, a term that few languages today seem to want attached to
them) but being dynamic isn't a binary thing, it's an analog quality,
a language can be less or be more dynamic. For example I think Ruby is
more dynamic than Python, Python is more dynamic than CLisp, CLips
seems more dynamic than C#/Java, Java is more dynamic than D, and D is
more dynamic than C++. Often such differences aren't sharp, and you
can find ways to things more dynamically, even with a less nice syntax
(or creating less idiomatic code). (In C#4 they have even added a
dynamic feature that may make languages like IronPython/Boo faster and
simpler to write on the dotnet).

In the last two years I have seen many answers in the Python
newsgroups, and I have seen that some of the dynamic features of
Python aren't much used/appreciated:
- Metaclasses tricks
- exec and eval
- monkey patching done on classes
- arbitrary cmp among different types removed from Python 3
While some new static-looking/related features are being introduced:
- ABCs and function signatures added
- More tidy exception tree

So it seems that while C#/D are becoming more dynamic, Python/
JavaScript are becoming a little less dynamic looking (and this I
think this is positive, because too much dynamism turns code into a
swamp, and too much rigid systems lead to bloat and other problems.
Note that there are another orthogonal solution: to use an advanced
flexible and handy static type system, as in Haskell).

I have seen that in lot of those little programs of mine, or in a
significant percentage of their lines, I often don't use the dynamic
features of Python (this means that the same code can be written with
static types, especially if you can use templates like in C++/D, or a
flexible type system like in Haskell, and it also means that lot of
those small programs can be compiled by ShedSkin/Cython, with usually
a sharp decrease of running time).

What are the dynamic features of Python more used in your programs?
(From this set try to remove the things that can be done with a
flexible static template system, like the D one, that for some things
is more handy and powerful than the C++ template system, and for other
things less powerful).

If very little or no dynamic features are used in a program it may
seem a "waste" to use Python to write the code, because the final
program may be quite slow with no gain from the other features of
Python. (I think in such situations Python can be a good choice
anyway, because it's good to write working prototypes in a short
time). (The large number of solution of this page shows how a certain
class of Python programmers want more speed from their programs:http://scipy.org/PerformancePython*and note that page misses many
other solutions, like SIP, Boost Python, ShedSkin, Cinpy, Cython,
RPython, and so on).

In the last year I have found two situations where exec/eval is a way
to reduce a lot of the complexity of the code, so if used with care
the dynamic features can be quite useful.

Before ending this partially incoherent post, I'd also like to briefly
remind how the dynamic features are used in the Boo language: Boo
programs are generally statically typed, but duck types are used once
in a while to reduce the "pressure" of the static type system. You can
find more info on this on the Boo site. (Note that I have never seen a
good set of speed benchmarks to compare the performance of CPython to
Boo).

Bye,
bearophile

http://scipy.org/PerformancePython is loading to slowly for my
connection. How ironic (though it probably isn't Pythons fault).

[Almar Klein]

<snip>

You're right (I think), but I fail to see the point you're trying
to make or the question you're asking... :)

I use python for scientific research too, and for me speed can be
an issue too sometimes. By using numpy and scipy I have
an environment similar to Matlab in terms of speed and
functionality (but with a language thats much easier to code in).
C# or C++ etc would be faster, yes...
But what I find even more important is that my code is interpreterd,
so I can run some code from my editor, check some results,
run another piece of code, or make changes to the code and try
again. I've tried using C# for doing my stuff, but then the
compile-run step takes ages just too long... Plus in python you
can introspect all your varialbes using the python prompt.

Cheers,
Almar

[bearophileHUGS]

Almar Klein:

but I fail to see the point you're trying
to make or the question you're asking... :)

It's not easy to define what my point was :-) I try again, but the
following questions don't cover all the points:
- What are the dynamic features of Python that you use in your code?
(excluding ones that can can be done with a good static template
system).
- Are them worth the decrease in running speed?
- Is it good for Python to become two languages in one, a fast
statically typed one and a dynamically one, like pypy shows to like
with RPython, or is it better to go the way of the Boo language, that
(while being mostly static) is mixing dynamic and static typing in the
same code, but that must rely on a very complex virtual machine to
work?
- Or maybe is it better to find other ways like CLips ones, that allow
to mix dynamic and static features, generally keeping programs fast
enough (Lisp-like syntax can lead to high performance too, as shown by
the Stalin Scheme compiler http://en.wikipedia.org/wiki/Stalin_...implementation)
).

Bye,
bearophile

[pruebauno]

On Nov 21, 4:17 am, [e-mail] wrote:

What are the dynamic features of Python that you use in your code?

The main ones is using configuration files that are plain Python
instead of XML and not having to wait 5 minutes to compile larger
programs. I also prefer structural typing over nominative typing and
duck typing is closer to structural than nominative typing with the
difference that compilation doesn’t take forever but the disadvantage
that the checking doesn’t happen until runtime. Hopefully they will
keep improving pylint to generate warnings for all those.

As far as speed goes, there might be 5% of my code that I would be
willing to run through a slow compiler to speed it up. For the rest it
is probably not worth it.

[George Sakkis]

On Nov 21, 7:55*am, [e-mail] wrote:

It's not easy to define what my point was :-) I try again, but the
following questions don't cover all the points:
- What are the dynamic features of Python that you use in your code?
(excluding ones that can can be done with a good static template
system).

Off the top of my head, getattr/setattr are the most frequent dynamic
features I use.

- Are them worth the decrease in running speed?
- Is it good for Python to become two languages in one, a fast
statically typed one and a dynamically one, like pypy shows to like
with RPython, or is it better to go the way of the Boo language, that
(while being mostly static) is mixing dynamic and static typing in the
same code, but that must rely on a very complex virtual machine to
work?
- Or maybe is it better to find other ways like CLips ones, that allow
to mix dynamic and static features, generally keeping programs fast
enough (Lisp-like syntax can lead to high performance too, as shown by
the Stalin Scheme compilerhttp://en.wikipedia.org/wiki/Stalin_(Scheme_implementation)

Very valid points, and I also often think that dynamic typing is
overrated; most programs don't need to add or remove attributes at
will or change the class hierarchy. I don't know which of the
alternatives you mention would be better but I would welcome changes
towards the "static by default" direction, provided that (1) it *is*
still possible to write dynamic code if necessary and (2) the extra
effort in writing and reading it is not off-putting (e.g. no C++
template metaprogramming atrocities)

George