[Fwd: maxima-cvs-20020408: validation test results]

-----Forwarded Message-----

From: Nelson H. F. Beebe <beebe at math>
To: James Amundson <amundson@fnal.gov>
Cc: beebe@math.utah.edu
Subject: maxima-cvs-20020408: validation test results
Date: 11 Apr 2002 09:35:52 -0600

The validation test results for maxima-cvs-20020408 for all of the
previously-reported successful maxima builds appear to be okay:

	% grep -h errors, Typescript.* | sort | uniq -c
	   1081 0 errors, 0 warnings
	     88 0 errors, 1 warning
	     56 0 errors, 2 warnings
	      8 0 errors, 3 warnings
	      8 0 errors, 32 warnings
	      8 0 errors, 5 warnings
	      8 0 errors, 7 warnings
	      8 0 errors, 8 warnings

The summary tests, except for one, look like this:

	*** Summary results for tests recorded in
	*** log file tests-gcl.log:
	No Errors Found
	*** end of summary for tests-gcl.log

On Sun Solaris 2.8, I got this instead:

	*** Summary results for tests recorded in
	*** log file tests-clisp.log:
	Unknown error in log file
	*** end of summary for tests-clisp.log

There is a lot of output in the log file, and I cannot spot anything
that is obviously responsible.

Ultimately, it would be better to design the maxima validation tests
to show nothing but their names on success, such as this output from
one of my own packages:

	% make check
	========================================================================
	Test of ndiff: there should be no output, other than the test and
	command names. The tests take about 10 sec on a fast workstation.
	========================================================================
	============ check001: ndiff -?
	============ check002: ndiff --author
	============ check003: ndiff --copyright
	============ check004: ndiff --copyleft
	============ check005: ndiff --help
	============ check006: ndiff --version
	============ check007: ndiff --absolute-error 1.e-14
	============ check008: ndiff -fields 2--3,4,1
	============ check009: ndiff --minimum-width 4
	============ check010: ndiff -quiet
	============ check011: ndiff --relative-error 2
	============ check012: ndiff --relerr 8
	============ check013: ndiff --relative-error 1.e-10
	============ check014: ndiff --relative-error 1.e-14 --quiet
	============ check015: ndiff --relative-error 1.e-14 --silent
	============ check016: ndiff --relative-error 1.e-14 --quiet --silent
	============ check017: ndiff --relative-error 1.e-14
	============ check018: ndiff --relative-error 1.e-15
	============ check019: ndiff --relative-error 1.23e-300
	============ check020: ndiff
	============ check021: ndiff
	============ check022: ndiff
	========================================================================

For large test suites, a summary is useful:

	Test summary: 423 passes, 2 failures

That way, grep can easily extract the test result from a possibly
large typescript of the build.

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- Center for Scientific Computing       FAX: +1 801 585 1640, +1 801 581 4148 -
- University of Utah                    Internet e-mail: beebe@math.utah.edu  -
- Department of Mathematics, 110 LCB        beebe@acm.org  beebe@computer.org -
- 155 S 1400 E RM 233                       beebe@ieee.org                    -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe  -
-------------------------------------------------------------------------------