Discussion about arrays and matrices in Maxima

Hi Maxima developers and users!

Since we have no 'benovelent dictator' or Maxima roadmap for future  
changes of Maxima, I've decided to start this discussion - in order  
to achive a sort of consense among Maxima developers about how the  
matrix/arrays situation should be treated and also to come up with  
new ideas that haven't been mentioned here perhaps.

I've seen the inefficiency of Maxima's development model from the  
user's standpoint of view - I've written a function $table, both  
feature rich and 'correct' in order to improve upon the makelist/ 
create_list situation. However, though there have been some positive  
comments about it from users/developers, it all stopped with the  
argument, that there are already makelist/create_list functions, but  
no real solution is presented to me as a contributor. In effect,  
Maxima stays as it is (with inferior solutions), just because  
implementing something new would mean some complications (not from a  
practical point of view)... So from my experience - it's hard to  
improve something in Maxima, though some Maxima's facilities  
(makelist/create_list and current array handling pop to mind) are  
clearly inefficient and/or inflexible and/or harder to use than they  
should be.

Now to the topic of discussion - it has been mentioned many times on  
this list, that matrix/array situation should be solved. Current  
array handling in Maxima is a disaster - I would be surprised if  
someone used it a lot, and I think most people on this mailing list  
agree.
A solution has been mentioned many times: unification of matrices and  
arrays in Maxima. However we have to come to detailed description how  
to do this in order that someone can work on this. I have some  
opinions on this matter and I hope a discussion will produce a well  
formed description of how arrays should be handled in Maxima:

*) Matrices should internally represented as n-dimensional lisp  
arrays. So a matrix might look like ((mmatrix simp) #2A(...))  
internally. I stress 'n-dimensional lisp arrays', because I wouldn't  
like to limit array usage to two dimensional cases, but also one  
dimensional and 3-dimensional cases of arrays could be useful. Since  
Common Lisp provides arrays of any 'sensible' dimensionality (at  
least up to 9 I think), this generalisation comes naturally from the  
underlying implementation.

*) Matrices should support advanced indexing possibilities, both for  
retrieval of values and assigment (symetric). A good start is  
'amatrix' package. I would like to improve upon Maxima list indexing  
also to support things like my_list[[1,3,4]]. In general Maxima lists  
and Maxima matrices/arrays should try to behave in the same way if  
possible - this makes for easier learning of Maxima.

*) Lisp arrays for holding matrix elements should be of general type  
for general matrices containing Maxima expressions, but perhaps we  
could think about a special case of 'double-float cases, where a  
matrix would internally represented by a 'double-float array. I'm  
mentioning this, because I wish for faster floating point performance  
in Maxima.
For instance - it's often with big matrices (500x500 and up) that all  
elements are floating point numbers. If we have two 'double-float  
matrices - then the algorithms can be much faster - using lisp's '+'  
for addition and so on. Ofcourse this is additional complication, but  
it would mean very good floating point performance  
(lu,svd,multiplication,eigenvalues and so on) if it were correctly  
implemented. Lisp however makes things simple - since a lisp functon  
doesn't need to distinguish between 'double-float matrices and  
general matrices - 'aref' always works for retrieval of values.  
However when additional performance is needed, 'double-float matrices  
would help to get that MATLAB and Mathematica performance.

*) Entering matrices would probably be implemented as it is now -  
with matrix function.

*) Make matrices/arrays good citizens of Maxima: besides indexing and  
assigment, also functions like 'map' could be defined to work on both  
(elementwise for lists and arrays). If it is possible for a function  
to work on lists and arrays in somewhat the same way, it should be  
implemented. If it is not - two functions should be written and named  
approprietly, for instance:
makelist()
makematrix()

*) The only place a word 'array' should be mentioned in Maxima  
documentation is where implementation of matrices is explained,  
ideally (if current array facilities are some day removed). I think  
matrix (in Maxima language) is a suitable word for an array(its  
meaning in computer science).

I'm hoping this discussion will result in changes in Maxima. I'm not  
experienced enough to write any 'core implementation' of Maxima  
matrices with arrays. I'm hoping veteran Maxima developers will do  
that. I would, however, gladly contribute code for functions I would  
know how to write (for instance write a $makematrix modeled after  
$table) after the core implementation of array matrices is done.

Regards,
Ziga Lenarcic