newbie question, forcing numeric result

• Subject: newbie question, forcing numeric result
• From: Barton Willis
• Date: Sun, 24 Dec 2006 14:02:11 -0600

Notice 177.828b0 not 177.828:

(%i4) bfloat(gain(100,%i*2*%pi*177.828b0)), fpprec : 123;
(%o4) 1.1471348945529250782891741337[65
digits]5635237103496777477455719898b0

I'm not sure about this, but I think your calculation
runs into problems with floating point overflow. Subtractive
cancellation might not be the real problem.

Barton

• Prev by Date: newbie question, forcing numeric result
• Next by Date: newbie question, forcing numeric result
• Previous by thread: newbie question, forcing numeric result
• Next by thread: newbie question, forcing numeric result
• Index(es):
  • Date
  • Thread