Floating point inaccuracy examples

Posted by David Rutten on Stack Overflow See other posts from Stack Overflow or by David Rutten
Published on 2010-01-20T10:10:30Z Indexed on 2010/04/04 0:33 UTC
Read the original article Hit count: 658

Filed under:

floating-accuracy

|

floating-point

|

perennial-question

How do you explain floating point inaccuracy to fresh programmers and laymen who still think computers are infinitely wise and accurate?
Do you have a favourite example or anecdote which seems to get the idea across much better than an precise, but dry, explanation?
How is this taught in Computer Science classes?

© Stack Overflow or respective owner

Related posts about floating-accuracy

Floating point inaccuracy examples

as seen on Stack Overflow - Search for 'Stack Overflow'
How do you explain floating point inaccuracy to fresh programmers and laymen who still think computers are infinitely wise and accurate? Do you have a favourite example or anecdote which seems to get the idea across much better than an precise, but dry, explanation? How is this taught in Computer… >>> More
Why does (int)(33.46639 * 1000000) return 33466389?

as seen on Stack Overflow - Search for 'Stack Overflow'
(int)(33.46639 * 1000000) returns 33466389 Why does this happen? >>> More
Fixed Point to Floating Point and Backwards

as seen on Stack Overflow - Search for 'Stack Overflow'
Is converting Fixed Pt. (fixed n bit for fraction) to IEEE double safe ? ie: does IEEE double format can represent all numbers a fixed point can represent ? The test: a number goes to floating pt format then back to it's original fixed pt format. >>> More
C++ floating point precision

as seen on Stack Overflow - Search for 'Stack Overflow'
double a = 0.3; std::cout.precision(20); std::cout << a << std::endl; result: 0.2999999999999999889 double a, b; a = 0.3; b = 0; for (char i = 1; i <= 50; i++) { b = b + a; }; std::cout.precision(20); std::cout << b << std::endl; result: 15.000000000000014211 So… >>> More
Is JavaScript's math broken?

as seen on Stack Overflow - Search for 'Stack Overflow'
0.1 + 0.2 == 0.3 returns false 0.1 + 0.2 = 0.30000000000000004 Any ideas on why this happens? >>> More

Related posts about floating-point

Comparing floating point values

as seen on Stack Overflow - Search for 'Stack Overflow'
I just read a statement about the floating point value comparison Floating point values shall not be compared using either the == or != operators. Most floating point values have no exact binary representation and have a limited precision. If so what is the best method for comparing two floating… >>> More
understanding floating point variables

as seen on Stack Overflow - Search for 'Stack Overflow'
There is some problem, i can't understand anyway. look at this code please <script type="text/javascript"> function math(x) { var y; y = x*10; alert(y); } </script> <input type="button" onclick="math(0.011)"> What must be alerted after i click on button? i think 0.11… >>> More
floating point hex octal binary

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am working on a calculator that allows you to perform calculations past the decimal point in octal, hexadecimal, binary, and of course decimal. I am having trouble though finding a way to convert floating point decimal numbers to floating point hexadecimal, octal, binary and vice versa. The… >>> More
What is a Bias Value of Floating Point Numbers?

as seen on Stack Overflow - Search for 'Stack Overflow'
In learning how floating point numbers are represented in computers I have come across the term "bias value" that I do not quite understand. The bias value in floating point numbers has to do with the negative and positiveness of the exponent part of a floating point number. The bias value of a… >>> More
Why are floating point values so prolific?

as seen on Stack Overflow - Search for 'Stack Overflow'
So, title says it all. Why are floating point values so prolific in computer programming. Due to problems like rounding errors, and not being able to even accurately represent numbers such as 0.1, I really can't see how they got as far as they did. I understand that the computation is faster… >>> More