c++ float subtraction rounding error

Posted by Volkan Ozyilmaz on Stack Overflow See other posts from Stack Overflow or by Volkan Ozyilmaz
Published on 2014-08-21T19:17:28Z Indexed on 2014/08/21 22:20 UTC
Read the original article Hit count: 138

Filed under:
|

I have a float value between 0 and 1. I need to convert it with -120 to 80. To do this, first I multiply with 200 after 120 subtract. When subtract is made I had rounding error. Let's look my example.

    float val = 0.6050f;
    val *= 200.f;

Now val is 121.0 as I expected.

    val -= 120.0f;    

Now val is 0.99999992

I thought maybe I can avoid this problem with multiplication and division.

    float val = 0.6050f;
    val *= 200.f;
    val *= 100.f;
    val -= 12000.0f;    
    val /= 100.f;

But it didn't help. I have still 0.99 on my hand.

Is there a solution for it?

Edit: After with detailed logging, I understand there is no problem with this part of code. Before my log shows me "0.605", after I had detailed log and I saw "0.60499995946884155273437500000000000000000000000000" the problem is in different place.

© Stack Overflow or respective owner

Related posts about c++

Related posts about floating-point