Search Results

Search found 1 results on 1 pages for 'samchandra'.

Page 1/1 | 1 

  • Wrong Ruby Float and BigDecimal subtraction result

    - by SamChandra
    I am running: [~/ruby/rails/sas]$ ruby --version ruby 1.8.7 (2009-06-08 patchlevel 173) [universal-darwin10.0] on Mac Snow Leopard 10.6.3 Can anyone help to explain why the Float and BigDecimal subtraction can be this wrong. [~/ruby/rails/sas]$ console Loading development environment (Rails 2.1.1) >> num = 30.0 => 30.0 >> num.class => Float >> ex = 28.04.to_d => #<BigDecimal:105367e40,'0.284E2',8(8)> >> ex.class => BigDecimal >> num - ex => 1.6 >> _.class => Float >> I was hoping that the result should be 1.96, I know that perhaps doing an arithmetic operation using 2 different data types are not recommended, but this behavior is so strange. It seems to be wise that from now on, I have to check the variables data type before doing any arithmetic operation. Hopefully somebody can give me an insight on what was happening.

    Read the article

1