Subject: Maxima gives incorrect results with ceiling
From: Stavros Macrakis
Date: Fri, 7 Dec 2007 17:18:37 -0500
On Dec 7, 2007 4:36 PM, Daniel Lakeland <dlakelan at street-artists.org> wrote:
> I guess I'm suggesting a refinement of the current method
> which might have better properties, if not perfect ones.
>
You seemed to suggest that the correct precision to calculate in can be
determined by the size of the result estimated by the current fpprec
precision. I don't see how this helps. After all, ( 1.0b0-10.0b0^-100) is
exactly equal to 1.0b0 for fpprec < 100. By your criterion, since the base
2 exponent is 0 << fpprec, we don't need to calculate more digits.
The other option is to double the precision and recalculate, until the
> abs error between successive values is smaller than 1.
Again, why is this a good criterion? If the correct value is int+epsilon,
what matters to the floor/ceiling calculation is the sign of epsilon, which
may be arbitrarily small and arbitrarily badly behaved, and an arbitrarily
small fraction of int.
> The doubling strategy should find that bf precision in log(n) time,
!!! Only if calculating in fpprec=2*N costs the same as calculating in
fpprec=N, which is far from true!
> and the "jump to it" strategy should work in constant time for any case
> where
> cancellation is not a major issue.
Well, of course in the cases where it works, it is very fast. So is
calculating in floating point. But it gets the wrong answer in general.
-s