Status of Numeric

[Travis Oliphant]

Numarray is making great progress and is quite usable for many
purposes. An idea that was championed by some is that the Numeric code
base would stay static and be replaced entirely by Numarray.

However, Numeric is currently used in a large installed base. In
particular SciPy uses Numeric as its core array. While no doubt
numarray arrays will be supported in the future, the speed of the less
bulky Numeric arrays and the typical case that we encounter in SciPy of
many, small arrays will make it difficult for people to abandon Numeric
entirely with it's comparatively light-weight arrays.

In the development of SciPy we have encountered issues in Numeric that
we feel need to be fixed. As this has become an important path to
success of several projects (both commercial and open) it is absolutely
necessary that this issues be addressed.

The purpose of this email is to assess the attitude of the community
regarding how these changes to Numeric should be accomplished.

These are the two options we can see:
* freeze old Numeric 23.x and make all changes to Numeric 24.x still
keeping Numeric separate from SciPy
* freeze old Numeric 23.x and subsume Numeric into SciPy essentially
creating a new SciPy arrayobject that is fast and lightweight. Anybody
wanting this new array object would get it by installing scipy_base.
Numeric would never change in the future but the array in scipy_base would.

It is not an option to wait for numarray to get fast enough as these
issues need to be addressed now. Ultimately I think it will be a wise
thing to have two implementations of arrays: one that is fast and
lightweight optimized for many relatively small arrays, and another that
is optimized for large-scale arrays. Eventually, the use of these two
underlying implementations should be automatic and invisible to the user.

A few of the particular changes we need to make to the Numeric
arrayobject are:

1) change the coercion model to reflect Numarray's choice and eliminate
the savespace crutch.
2) Add indexing capability to Numeric arrays (similar to Numarray's)
3) Improve the interaction between Numeric arrays and scalars.
4) Optimization:

Again, these changes are going to be made to some form of the Numeric
arrays. What I am really interested in knowing is the attitude of the
community towards keeping Numeric around. If most of the community
wants to see Numeric go away then we will be forced to bring the
Numeric array under the SciPy code-base and own it there.

Your feedback is welcome and appreciated.

Sincerely,

Travis Oliphant and other SciPy developers