exact decimal type, was: 12.3*70.95 a little bit weird
Subject: exact decimal type, was: 12.3*70.95 a little bit weird
From: Raymond Toy
Date: Wed, 30 Nov 2011 20:19:03 -0800
On 11/30/11 9:21 AM, Stavros Macrakis wrote:
> How about rat(12.345678)? Of course, ratepsilon should be 2e-15 or so
> (which we agreed on in February 2005, but it's still 1e-8!) and you
> probably want to turn off ratprint.
Just remembered there's one problem with using rat. The number is
converted to a floating-point number before rat gets it. So you get FP
roundoff. With r notation, it truly is exact and you can have way more
than 15 significant digits. Using bfloats is a workaround, which only
works if fpprec is bigger than the number of digits you've entered.
Ray