What do you mean about this data structure

> I try to solve a system of equation and maxima take more than
> 450MB and stop with error. And this after more than 15 hours of calculus

There are many possible reasons for this:

1) Your system is inherently hard to solve, e.g. the solution is huge.
 Fortunately, it is often possible to reformulate the problem (e.g.
specialize to the cases of interest, use appropriate approximations,
etc.) so that the solution is tractable.
2) The algorithms Maxima uses are not efficient, e.g. Maxima uses an
n^6 algorithm where an n*log(n) one is known.  There are certainly
cases where Maxima is not using the best known algorithm.
3) The implementation of the algorithms is not efficient, e.g. writing
in Lisp rather than C exacts a speed penalty.
4) There is a bug in Maxima, e.g. there is an infinite loop in some
case.  This is alas more common than it should be.
5) You are solving the system in an inefficient way, e.g. you are
performing a matrix inversion when you only need an eigenvalue.

As Fateman points out, an improvement in the representation can only
address (3), and only give a small linear improvement in it.

In my experience, (1), (4), and (5) are more common than (2) and (3). 
Perhaps you could share with the list what sort of equations these are
and why you are solving them.

          -s