Rejigging a floating point equation ...

Posted by Jamie on Stack Overflow See other posts from Stack Overflow or by Jamie
Published on 2009-05-27T18:51:24Z Indexed on 2010/12/27 23:54 UTC
Read the original article Hit count: 231

Filed under:

floating-point

|

precision

|

linear-algebra

I'd like to know if there is a way to improve the accuracy of calculating a slope. (This came up a few months back here).

It seems by changing:

float get_slope(float dXa, float dXb, float dYa, float dYb) {
    return (dXa - dXb)/(dYa - dYb);
}

to

float get_slope(float dXa, float dXb, float dYa, float dYb) {
    return  dXa/(dYa - dYb) - dXb/(dYa - dYb);
}

might be an improvement. Suggestions?

Edit: It's precision I'm after, not efficiency.

© Stack Overflow or respective owner

Related posts about floating-point

understanding floating point variables

as seen on Stack Overflow - Search for 'Stack Overflow'
There is some problem, i can't understand anyway. look at this code please <script type="text/javascript"> function math(x) { var y; y = x*10; alert(y); } </script> <input type="button" onclick="math(0.011)"> What must be alerted after i click on button? i think 0.11… >>> More
Comparing floating point values

as seen on Stack Overflow - Search for 'Stack Overflow'
I just read a statement about the floating point value comparison Floating point values shall not be compared using either the == or != operators. Most floating point values have no exact binary representation and have a limited precision. If so what is the best method for comparing two floating… >>> More
floating point hex octal binary

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am working on a calculator that allows you to perform calculations past the decimal point in octal, hexadecimal, binary, and of course decimal. I am having trouble though finding a way to convert floating point decimal numbers to floating point hexadecimal, octal, binary and vice versa. The… >>> More
What is a Bias Value of Floating Point Numbers?

as seen on Stack Overflow - Search for 'Stack Overflow'
In learning how floating point numbers are represented in computers I have come across the term "bias value" that I do not quite understand. The bias value in floating point numbers has to do with the negative and positiveness of the exponent part of a floating point number. The bias value of a… >>> More
Why are floating point values so prolific?

as seen on Stack Overflow - Search for 'Stack Overflow'
So, title says it all. Why are floating point values so prolific in computer programming. Due to problems like rounding errors, and not being able to even accurately represent numbers such as 0.1, I really can't see how they got as far as they did. I understand that the computation is faster… >>> More

Related posts about precision

SQL SERVER – Precision of SMALLDATETIME – A 1 Minute Precision

as seen on SQL Authority - Search for 'SQL Authority'
I am myself surprised that I am writing this post today. I am going to present one of the very known facts of SQL Server SMALLDATETIME datatype. Even though this is a very well-known datatype, many a time, I have seen developers getting confused with precision of the SMALLDATETIME datatype. The precision… >>> More
Using Taylor Series to Avoid Loss of Precision

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to use Taylor series to develop a numerically sound algorithm for solving a function. I've been at it for quite a while, but haven't had any luck yet. I'm not sure what I'm doing wrong. The function is f(x)=1 + x - sin(x)/ln(1+x) x~0 Also: why does loss of precision even occur in… >>> More
Delphi - Frac function losing precision.

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a TDateTime variable which is assigned a value at runtime of 40510.416667. When I extract the time to a TTime type variable using the Frac function, it sets it to 0.41666666666. Why has it changed the precision of the value and is there a workround to retain the precision from the original… >>> More
SQL Server cast fails with arithmetic overflow

as seen on Stack Overflow - Search for 'Stack Overflow'
According to the entry for decimal and numeric data types in SQL Server 2008 Books Online, precision is: p (precision) The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision must be a value from 1 through the maximum… >>> More
Why don't computers store decimal numbers as a second whole number?

as seen on Programmers - Search for 'Programmers'
Computers have trouble storing fractional numbers where the denominator is something other than a solution to 2^x. This is because the first digit after the decimal is worth 1/2, the second 1/4 (or 1/(2^1) and 1/(2^2)) etc. Why deal with all sorts of rounding errors when the computer could have… >>> More