Python - Number of Significant Digits in results of division

Posted by russ on Stack Overflow See other posts from Stack Overflow or by russ
Published on 2010-04-01T10:56:55Z Indexed on 2010/04/01 11:03 UTC
Read the original article Hit count: 304

Filed under:
|
|
|

Newbie here. I have the following code:

myADC = 128
maxVoltage = 5.0
maxADC = 255.0
VoltsPerADC = maxVoltage/maxADC
myVolts = myADC * VoltsPerADC
print "myADC = {0: >3}".format(myADC)
print "VoltsPerADC =  {0: >7}".format(VoltsPerADC)
print VoltsPerADC
print "myVolts = {0: >7}".format(myVolts)
print myVolts

This outputs the following:

myADC = 128
VoltsPerADC =  0.0196078
0.0196078431373
myVolts =  2.5098
2.50980392157

I have been searching for an explanation of how the number of significant digits is determined by default, but have had trouble locating an explanation that makes sense to me. This link link text suggests that by default the "print" statement prints numbers to 10 significant figures, but that does not seem to be the case in my results.

How are the number of significant digits/precision determined?

Can someone shed some light on this for me.

Thanks in advance for your time and patience.

© Stack Overflow or respective owner

Related posts about python

Related posts about division