[sage-devel] compiling Maxima by ECL



 

> -----Original Message-----
> From: Juan Jose Garcia-Ripoll 
> [mailto:juanjose.garciaripoll at googlemail.com] 
> Sent: Monday, April 28, 2008 8:14 AM
> To: fateman at EECS.Berkeley.EDU
> Cc: maxima at math.utexas.edu; Michael. Abshoff
> Subject: Re: [Maxima] [sage-devel] compiling Maxima by ECL
> 
> On Mon, Apr 28, 2008 at 4:25 PM, Richard Fateman
> <fateman at cs.berkeley.edu> wrote:
> >  Thank you for this information. Unfortunately Michael said 
> the sourceforge
> >  page was mostly out of date and  pointed me to
> >  this: http://ecls.wiki.sourceforge.net/space/showimage/eclm2008.pdf
> 
> What is the problem with that PDF? It contains accurate information.
> The web pages have some out of date regions, in particular around the
> manual introduction and details about the C compiler, because this
> manual organically evolved from EcoLisp's one.

What is wrong with it, is that it is vague with respect to such issues as:
Do you compile special code for fixnums? floats? 

This information is in the manual at sourceforge, which I was told by
Michael
to ignore because it was out of date.


> 
> >  rather than some useful manual, which I just found via Google at
> >  uh, sourceforge, the place I was told not to look. Thanks Michael.
> >  http://ecls.sourceforge.net/new-manual/pr01s02.html




> 
> What is the problem with this description? It is correct. The rest of
> the frontpage is as well: http://ecls.sourceforge.net/ ECL has a
> bytecodes interpreter, provides two different garbage collectors,
> implements a foreign function interface, etc, etc.

There may be nothing wrong with this. Just that Michael said not to look at
it.


>

.....

> >  It is my understanding that you are referring to an ANSI 
> test suite which
> >  is a conformance test, not an efficiency test.
> 
> I refer to both. 

  Both? I did not see two tests.
 The Dietz test from the powerpoint presentation is a conformance test.


> This test suite does all of the things that the
> Maxima compilation does, in terms of using COMPILE a lot and consing
> and destroying a lot of data as well.

I don't see how you could tell that.  If you wanted to use compilation of
Maxima
as a test suite, that would be interesting to the Maxima people.
  Running Maxima would be another test, which apparently ECL does not quite
pass.

>  My experience is that this is
> also a stress test of the garbage collector, as the ANSI test suite
> may end up using close to a hundred Mb depending on the platform. And
> to this respect, the more we learn about the Boehm-Weiser GC the
> better these numbers (speed and consing) get.

100Mb is a very very very small amount of memory to use. 1/10 of a Gigabyte.
If I needed only 100Mb I would not have to do any garbage collection at all.

 This is not a serious stress test. A stress test might be 5 Gigabytes,
churned over
repeatedly,say 100 times, on a machine with 2 Gigabytes of RAM. 


> 
> >  It is also not clear what
> >  you mean by "strong"  when it is pretty clear that GC has 
> traditionally
> >  been a problem with lisp until the deployment of better GCs.
> 
> Excuse my poor use of English, which is not my mother tongue, but by
> "strong" I meant both robust and well performing.

Your English is quite clear. No need to apologize.
What you are saying is, your GC does not have bugs, as far as you can tell.
Since it has not been stressed very much, maybe it has bugs that you have
not
yet encountered.

> Your statements also
> do not contradict mine: the fact that GC has been traditionally a weak
> point in implementations does not preclude the existence of a good GC
> library.

I think that strapping on a GC as a library pretty much does preclude a
modern 
"generation scavenging" copying garbage collector. Perhaps someone who is
clever can figure out a way around this, but as far as I know, the
generation
scavenging marking has to be made part of the generated code.
> 
> >  This may be your view, but the benchmarks you offer show that it is
> >  inferior to other lisps with the exception of one aspect, 
> which is that
> >  it is (alleged) to meet the SAGE criterion of easily 
> compilable from
> >  source.  That is not to say that it cannot be made better, 
> and perhaps
> >  that will happen.
> 
> Which benchmarks? 

The one benchmark you offer (Dietz) shows CLISP faster. There are other
benchmarks,
say the "Gabriel" benchmarks, that offer more variety.

>I have only seen the previous email about how ECL
> run 10x slower with the current build of Maxima. But to blame ECL
> alone is just as innocent as to say Maxima is the perfect software
> that should run anywhere unchanged.

I would say 10X slower is pretty significant, but a compiler can
make a 20X difference.

> Different implementations may have
> different behavior and features. A port is rarely just load and run.

> 
> And which sense of inferiority do you use. As mentioned before: "I
> understand that GCL is not  ANSI compliant." Is this not an
> inferiority?

Only if you need these features. Macsyma was written originally in
Maclisp, which was not ANSI compliant.  It was ported to GCL by
Bill Schelter, before ANSI CL existed. So original Maxima code does
not require ANSI compliance. More significant for some users are the
set of extensions available that are not part of the ANSI standard.

> 
> I will stop discussing performance here, but anyone will understand
> the three following facts:
> - GCL has been traditionally the host of Maxima. As such, performance
> has been tuned in both programs so as to get it run well and fast.

Perhaps, but Maxima runs OK in other lisps too.


> - Maxima expects a certain behavior of the common lisp. Namely,
> COMPILE has to be cheap.

This does not strike me as true. Where did you got this idea?
Compilation, is quite expensive in CMU-CL, when I last tried it.
And in fact compilation is expensive in GCL compared to (say) Allegro CL.

> It should be possible to dump memory images.

That is handy, but loading the core system from "fasl" files can be done, if
necessary.  I think it takes about 15 seconds on my machine.



> Running interpreted code is slow...
> - ECL has not exhausted its potential for speed up as it has not been
> a concern so far. But rather getting a better in the sense of
> maintainable, portable and standard implementation.
> 
> I do not care whether Maxima is ported or not to ECL. I definitely
> would like to use it to see what performance bottlenecks there are and
> to see to what extent this is a problem of Maximas assumption on the
> lisp backend or to ECL's own limitations.

There have been many implementations of Lisp. Many have essentially no
users. Writing and maintaining another one is interesting to the author
for whatever reasons he has.  For users of Lisp, a new implementation
is interesting only if it is significantly superior to previous
implementations
in some respect, and not much worse than previous systems in most others.
According to the ALU wiki, there are 8 commercial vendors and 9 other
open-source projects
(+ ECL). 

Is ECL going to be better than the others for anyone?  I cannot say.

> 
> But it seems you are having more fun making this a war of lisps.

Diversion of resources is an issue.

...  snip

> >  to have its own set of optimized bytecodes?  Doesn't ECL 
> have its own set of
> >  bytecodes? If so, why are they not "optimized"? Whatever 
> that means.
> 
> Why are you so picky? I just stated something everybody working with
> CLISP knows, which is that it has a lot of special purpose bytecodes
> to make bytecompiled code run fast. In ECL I chose to keep the set of
> bytecodes small so as to make the interpreter more easily maintainable
> and to to enforce people write certain kind of code for efficiency.

OK, so what you meant to say is that ECL has fewer distinct byte codes than
CLISP.
Whether this is good or bad, I don't know.  Why it is a good idea to enforce
people to write certain kind of code, I do not know.  Some of the people
who wrote code for Maxima are no longer alive, so it is hard to enforce them
to write particular way.

Bill Rowan (under my supervision, at Berkeley)
wrote a byte compiler for Maclisp that ran most of the Macsyma
system in 1980. (Conference on Lisp and Functional Programming), and the
Interlisp byte code system dates back to 1978 or earlier. 

> 
> >  I suppose that coming from a background in compiling conventional
> >  programming languages it seems odd to use a list based interpreter,
> >  but that is not necessarily an advantage. Consider the 
> debugging advantage
> >  of actually interpreting the lisp lists as programs.  Of 
> course, after
> >  the programs are debugged you might choose to speed them 
> up by compiling.
> 
> First, list interpreting is not such a weird experience for me. You
> should understand I have been working with Common lisp for over 10
> years, most of it developing ECL, from the very first sources which
> did look a lot like GCL looks nowadays
> 
> So my experience with list-based interpreters is not that of a C guy
> who is scared by all the lispy things. List based interpreters have
> serious speed issues, and also semantic issues, related to how changes
> in macros affect code.

> Besides list based debugging is not really that
> powerful. I have implemented stepper and debuggers using the ECL
> bytecodes and in my experience it has not been that painful. OTOH,
> having forms being bytecompiled provides a well defined semantics,
> which is not far away from that of ordinary compiled code.

The two paragraphs above are somewhat backwards, I think. The semantics
of common lisp are usually viewed operationally as embodied in the
interpreter and its treatment of
code as lisp.  

Using the byte code as the semantics sounds backwards.

> 
> >  Making GCL into an ANSI compliant Common Lisp would make 
> it a better
> >  host for Maxima, in some ways. It is speculative at best 
> that improving
> >  ECL will help Maxima. But you are of course free to spend your time
> >  any way you wish.
> 
> Doesn't sound very encouraging. Really, it seems all this email
> exchange is just playing the scarecrow game: scare all possible
> innovation away.
> 

 If ECL cannot stand closer examination, it should not absorb
peoples' valuable time (other than the author, who presumably has
his own reasons independent of any users.)


> >  > So, to the interested parties, if you so much care about Maxima
> >  > running on just one computer, then stick to sbcl and 
> cmucl which are
> >  > pretty superior implementations, but please do not scare 
> people from
> >  > porting useful software to other platforms and environments.
> >
> >  My impression is that CMUCL and SBCL cannot run on Windows 
> directly, which
> >  makes GCL part of the Maxima world.
> 
> Ok, I must admit this is kind of mind twisting for me. First it was
> enough to have one computer and one compiler to run your software, now
> Windows is a requirement.

Actually, my fastest and most convenient computer is a Pentium running
windows.
That is GCL's advantage.


>
RJF