Maxima: simplification function error message.

There are two ways you are probably running out of space:

1) In Maxima's current string implementation (using Lisp symbols),
strings are never garbage collected (!!).  This means that once you've
taken space for a given string, that space is never reclaimed.  When
we eventually move to a LIsp-string based implementation, this problem
will go away.  In the meantime, you should probably evaluate
expression complexity in some other way, such as tree-nodes (see
below).

2) You may be keeping around a very large number of candidate
expressions in indexlist.  You need to think about how to reduce that.

By the way, if you really want to try hard to create simpler forms,
another thing to try is factoring all *subexpressions*....  Of course,
that would use a great deal of runtime....

Here's a simple complexity function:

  expr_nodes(ex):=
     block([inflag:true],           /* to avoid reformatting in maplist */
       if numberp(ex) then 1 else
       if atom(ex) then 3 else
       if subvarp(ex) then 5 else
       2 + lreduce("+", maplist(expr_nodes, ex)))

Obviously this can be made more elaborate in many ways, e.g. taking
into account nesting structure, or considering -x to be equally
complex as x, considering exponentiation as more expensive than
addition, etc. etc.

             -s