Questions for using Lisp language to built the computer algebra systems.

I think the most drawback of the computer algebra system built on the Lisp system 
is that since the Lisp system makes so much pointer references
to make access to the lisp cells it is slow on the pipelined and cached systems.
(Even the cdr-coding could reduce the pointer references, tagged pointer access
means every reference of pointer can causes the branch or the pipeline jam to make
the computation slow.
Even today's procedural languages like Fortran 9x have the recursive function
and pointers and record-type data definition mechanism. (It is static and 
lacks the dynamic type as the language which Lisp has.) Do you think writing
generic computer algebra system like Maxima using for example Fortran9x or Java 
is too difficult ? 

Another drawback is that there are so many Lisp dialects.
Common Lisp system as the language contains too many functions defined 
and against the kind of ideas of the "reduced instruction sets" system.
(Only to have the instructions which are frequently used or unable to be eliminated. )
Like portable common lisp for Reduce, I think it might be better that for the maxima 
to define the minimum required set of lisp functions, and then other functions 
can be defined using lisp.

How do you think lisp is so essential or not? (I might be wrong.)