Setting precision; [Was] A puzzling arithmetic



Richard Fateman <fateman@cs.berkeley.edu> writes:

> Perhaps someone is also using FPPREC for some other purpose, maybe
> for controlling formatted output.

Yes. EXPLODEN (in commac.lisp) passes it to FORMAT.

With FPPREC:16,

1. Trace: (EXPLODEN '251.7999999999999)
2. Trace: (FORMAT 'NIL '"~vf" '17 '251.7999999999999)
2. Trace: FORMAT ==> "251.7999999999999"
1. Trace: EXPLODEN ==> (#\2 #\5 #\1 #\. #\7 #\9 #\9 #\9 #\9 #\9 #\9 #\9 #\9 #\9 #\9 #\9 #\9)

With FPPREC:12,

1. Trace: (EXPLODEN '251.7999999999999)
2. Trace: (FORMAT 'NIL '"~vf" '13 '251.7999999999999)
2. Trace: FORMAT ==> "        251.8"
1. Trace: EXPLODEN ==> (#\2 #\5 #\1 #\. #\8)

Should this behaviour be changed?

Wolfgang