On Tue, 2004-10-05 at 10:40, Raymond Toy wrote:
> James> I would also like to remind everyone that we can add tests that fail.
> James> The test suite handles expected failures smoothly.
>
> How does it know about expected failures?
Look in tests/testsuite.lisp. The list of filenames contains one entry
with a number, i.e.,
("rtest15.mac" 4)
Problem 4 in rtest15.mac is a known error. To see the known errors, use
run_testsuite(true)
Any problem numbers entered after a filename are considered to be known
errors.
The existence of this "known errors" mechanism is the reason I used the
lawyer-like statement "No unexpected errors found." when the test suite
finishes.
> This would be an excellent way of adding the examples from the bug
> database.
That was the idea when I introduced the known errors mechanism.
> They're known bugs, and we are constantly reminded about
> them. :-)
Unfortunately, you have to go out of your way ("run_testsuite(true)") to
be reminded.
--Jim