Hunter Monroe <hmonroe.maxima at huntermonroe.com> wrote:
Could you give me any guidance on how to go about making the Mockmma parser
case sensitive? I am aware that ANSI standard Common Lisp is not case
sensitive, and in particular GCL does not implement "(setf (readtable-case
*readtable*) :preserve)". How does Maxima achieve case sensitivity in
reading user input and would it make sense for Mockmma to employ the same
approach?
There are two mistaken assumptions in the above paragraph. First,
Common Lisp is actually case sensitive. Symbols will always be distinct
if they have any difference in case. It is the Common Lisp _reader_
(the function cl:read and friends) that manipulates case in ways
intended to emulate an IBM 026 keypunch. But read-char, read-line,
etc. never manipulate case. They don't use the readtable at all.
The second mistake is much more subtle -- everyone has made it from time
to time. It is true that the readtable permits customization of the
behavior of cl:read, in some ways very extensivce customization. But
this capability misleads programmers to the assumption that readtable
customization and cl:read is the right way to build almost any sort of
tokenizer and/or parser. It isn't. Attempts to overreach what the
readtable can accomplish always leads to massive contusions and eventual
failure.
(In more detail, the CL reader consists of two phases. The first phase
deals with collecting tokens and/or dispatching on reader macro
characters to call functions to accumulate and interpret a token from
the stream. This is the part that is customized by the readable. The
second phase is the interpretation of an accumulated token. It is not
at all under control of the readtable -- it's behavior is entirely fixed
except for minor effects by readtable-case and *read-base*. For
example, there is no way to alter the behavior of the colon character
except with horrendous kludgery.)
If you have a non-trivial language to parse, and can write something
approxiumating a BNF for it, the tool of choice is to use a LALR parser
such as YACC. There is a perfectly usable open source CL-YACC which
works in pretty mich all implementations. Using YACC effectively is
something of a rare art, but once you know it you will never be afraid
of writig a parser that is both reliable and readable.