Accuracy and error analysis (was Re: [Maxima] primes)

--- Albert Reiner  wrote:

> OK, that is certainly much more ambitious than what Mma gives you
> with its bigfloats.  Moreover, if it can be made to work, this would
> actually be useful to many.

That's the ultimate goal, of course :-).

> And given the need to actually specify the uncertainty associated
> with some input, this analysis is certainly something the user has to
> ask for explicitly, by calling some command; Mma instead tries to
> read the user's mind by counting digits upon input.

Right - for "normal" mathematical use, unless one is studying the
issue, I think things like the checking of the accuracy of %PI are
probably the limit of what is either needed or interesting.  I like the
idea that all digits shown to the user are meaningful, but that goal is
different from error analysis as such - or, at least, it can (probably)
be handled with purely mechanical checking instead of a full error
analysis.  Also, the problem arises that if one supplies an input of
1.11111+ 1.222222, there is no explicit statement of whether 1.11111 is
1.111110 exactly or 1.111110+-(0.000005).  The answer of 2.333332 is of
course not surprising, but how meaningful is that last two?  For most
numerical work, it is assumed that 2.333332 is what one wanted, but
when dealing with physical quantities it is an altogether different
situation.  1.10 and 1.1 imply different things, in such a case.

I think the issue should be invoked only when the user makes it clear,
by specifying an input with a user supplied error, that the user cares
about the uncertainty in the outcome.  In such problems, even with
small numbers, the limits of calculations by the machine can be handled
as above, and the fprec error component either ignored or included
based on how large it is relative to the user error.  The handling of
user error, on the other hand, is indeed a difficult problem since it
requires a full on error analysis and not just a few cycles of
numerical limit testing.  Such an analysis should be user requested
only, since only then can it be presumed to be worth the effort.

> This more ambitous task might also reduce some of the problems when
> mixing numerical and symbolical calculations in Mma.  It seems that
> you are after correctness, whereas Mma (in, e.g., the N... functions)
> often just decides to pretend that some quantity is known with
> greater accuracy than it is, which cannot possibly be correct.

Exactly.  If such an analysis is not correct it is worse than useless,
since the user will likely trust the answer whether or not it is valid.
 But for numerical calculations, at least ones that to not take such a
long time that the user notices, empirical determination of the error
should be sufficient, and if Maxima decides it needs greater accuracy
it has the option of either computing it out, or acknowledging the
uncertainty introduced and keeping track of it. 

> > The user should be able to denote the correlations between the
> > arguements.  I have no good vision for a syntax for this, probably
> > because I have a ways to go to understand the problem properly, but
> > I like the idea.
> 
> Well the obvious notation is to just take the joint probability
> distribution of all independent quantities (including those that do
> not enter the ``pure'' computation) as an additional argument to the
> error-propagating function.

Yes, that's the general case.  However, there will be many "standard"
cases where this could be anticipated (after all, much of physical
science works with the same measurement tools, measuring the same
quantities)  so I would prefer it that if standard situations are
described, Maxima is able to reuse previously defined nitty gritty in
order to lessen the grunt work.  This, of course, would also be
optional - the user would always have the option of taking matters
entirely into his/her own hands if necessary or desirable.

Another possibility would be establishing "guiding" questions, based on
previous input for an analysis - these at least can help the user ask
him/herself the right questions (which they would have had to answer
regardless of using a CAS, right?)  A more experienced user might wish
to disable this, but it would be both an excellent learning tool and a
good self checking tool for the researcher.

> Of course this places quite a burden on the user, but it also means
> that the user still has to make the difficult choices, and in
> particular that she has to decide on the precise distribution to use:
> Maxima will then only (try to) perform the routine part of the
> analysis.  And for this, the uncertainty-affected quantities need not
> even be numeric at all.

Difficult (or more precisely informed and intelligent) choices have to
be made regardless - the best that can be hoped for is those choices
can be made once and reused for other similar systems.

> Whether to call this automated error analysis is another question - I
> would view it more as providing a set of definitions for doing some
> specific tasks that happen to be useful for error analysis.

Maybe.  It comes down to how you define error analysis.  I gather the
normal definition includes both learning things about your system and
the mechanical process of doing the mathematics involved - I would
argue that a system which a) accepts (and perhaps even prompts for and
guides the user to) the full general set of error analysis options
relevant to a particular system b) is able to perform analysis of the
uncertainty of the system correctly as a result and c) is able to
return a reliably correct answer is pretty much automated error
analysis.  It isn't reasonable to expect Maxima to be able to handle
error analysis calculations without all relevant information, any more
than one can expect a to a+b without first telling Maxima what a and b
are.  One might (and would want to) define profiles relevant to
specific physical situations/equations/whatever, which would be even
more automated in an indiviual case.

So I guess it is automated in the sense that solving an equation for
forces in a mechanical system is automated - Maxima can solve the
equations (probably) but you have to understand the system in order to
write the equations.  Maxima can't understand the system for you.  But
for many such systems I'd much rather have Maxima solve the equation,
even if I still have to understand the system first in order to
describe it accurately.  Or perhaps one might look at it as needing to
define boundary conditions, as one needs to do in a variety of PDE and
ODE problems in order to get the desired answer.

CY


		
__________________________________ 
Do you Yahoo!? 
Read only the mail you want - Yahoo! Mail SpamGuard. 
http://promotions.yahoo.com/new_mail