Re: [Axiom-developer] Re: FeynCalc -> MAXIMA

>>> How else can you show that the code does what it is intended to do?
>> That is a very hard problem in general.

> Test cases.
> 
> While this does not *prove* the correctness of the code in a 
> mathematical sense, it does in a monte-carlo sense.  I note 
> that both maxima and axiom have test suites.
> 
> I do not think new mathematical code or subsystems should be 
> accepted unless they are accompanied by a thorough test suite.

I certainly agree that test suites are a very important part of any
software system, *even* when the code has been supposedly proven
correct, because so many things can go wrong in so many ways.

I am not sure, however, what it means to say that a test suite proves
the correctness of the code in a monte-carlo sense.  Software behavior
is not continuous, so you can't use variational arguments.
Combinatorial explosion makes it impossible to do anything near
exhaustive search.  Even with 20+ years of use, users find new bugs,
some of them surprisingly shallow (so you would expect that someone had
reported them before).  And actual usage is not random at all.  If a
mathematician asks Maxima to factor a polynomial, that polynomial
probably has some interesting properties, and it may be precisely those
properties which cause Maxima to fail.

Sure, in the best of all worlds, the test suite would at least exercise
every single line of code (though of course not all the combinations)
and would include all past bugs as well as a large selection of standard
cases and corner cases.  But it is a lot of work to maintain this, and
it would quickly become very time-consuming to execute as well....

       -s