Search Results

Search found 218 results on 9 pages for 'rounding'.

Page 2/9 | < Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >

  • Truncating double without rounding in C

    - by Coder
    Lets consider we have a double R = 99.999999; (which may be obtained by a result of some other computation),now the desired output is 99.99 I tried using printf("%.2lf",R); but it's rounding off the value.How to get the desired output ? (preferably using printf)

    Read the article

  • Round-twice error in .NET's Double.ToString method

    - by Jeppe Stig Nielsen
    Mathematically, consider for this question the rational number 8725724278030350 / 2**48 where ** in the denominator denotes exponentiation, i.e. the denominator is 2 to the 48th power. (The fraction is not in lowest terms, reducible by 2.) This number is exactly representable as a System.Double. Its decimal expansion is 31.0000000000000'49'73799150320701301097869873046875 (exact) where the apostrophes do not represent missing digits but merely mark the boudaries where rounding to 15 resp. 17 digits is to be performed. Note the following: If this number is rounded to 15 digits, the result will be 31 (followed by thirteen 0s) because the next digits (49...) begin with a 4 (meaning round down). But if the number is first rounded to 17 digits and then rounded to 15 digits, the result could be 31.0000000000001. This is because the first rounding rounds up by increasing the 49... digits to 50 (terminates) (next digits were 73...), and the second rounding might then round up again (when the midpoint-rounding rule says "round away from zero"). (There are many more numbers with the above characteristics, of course.) Now, it turns out that .NET's standard string representation of this number is "31.0000000000001". The question: Isn't this a bug? By standard string representation we mean the String produced by the parameterles Double.ToString() instance method which is of course identical to what is produced by ToString("G"). An interesting thing to note is that if you cast the above number to System.Decimal then you get a decimal that is 31 exactly! See this Stack Overflow question for a discussion of the surprising fact that casting a Double to Decimal involves first rounding to 15 digits. This means that casting to Decimal makes a correct round to 15 digits, whereas calling ToSting() makes an incorrect one. To sum up, we have a floating-point number that, when output to the user, is 31.0000000000001, but when converted to Decimal (where 29 digits are available), becomes 31 exactly. This is unfortunate. Here's some C# code for you to verify the problem: static void Main() { const double evil = 31.0000000000000497; string exactString = DoubleConverter.ToExactString(evil); // Jon Skeet, http://csharpindepth.com/Articles/General/FloatingPoint.aspx Console.WriteLine("Exact value (Jon Skeet): {0}", exactString); // writes 31.00000000000004973799150320701301097869873046875 Console.WriteLine("General format (G): {0}", evil); // writes 31.0000000000001 Console.WriteLine("Round-trip format (R): {0:R}", evil); // writes 31.00000000000005 Console.WriteLine(); Console.WriteLine("Binary repr.: {0}", String.Join(", ", BitConverter.GetBytes(evil).Select(b => "0x" + b.ToString("X2")))); Console.WriteLine(); decimal converted = (decimal)evil; Console.WriteLine("Decimal version: {0}", converted); // writes 31 decimal preciseDecimal = decimal.Parse(exactString, CultureInfo.InvariantCulture); Console.WriteLine("Better decimal: {0}", preciseDecimal); // writes 31.000000000000049737991503207 } The above code uses Skeet's ToExactString method. If you don't want to use his stuff (can be found through the URL), just delete the code lines above dependent on exactString. You can still see how the Double in question (evil) is rounded and cast.

    Read the article

  • Hard-coded 8191 10485 values in JavaScript rounding function

    - by Matthew Hegarty
    I've seen the following (bizarre) Javascript rounding function in some legacy code. After googling for it I can see that it crops up in a number of places online. However I can't work out why the hard-coded values 8191 and 10485 are present. Does anyone know if there's any sensible reason why these values are included? If not, hopefully we can kill off the meme! function roundNumber(num,dec) { var newnumber = 0; if (num > 8191 && num < 10485) { num = num-5000; newnumber = Math.round(num*Math.pow(10,dec))/Math.pow(10,dec); newnumber = newnumber+5000; } else { newnumber = Math.round(num*Math.pow(10,dec))/Math.pow(10,dec); } return newnumber; }

    Read the article

  • iphone float vs integer rounding?

    - by Rob
    Okay, from what I understand, an integer that is a fraction will be rounded one way or the other so that if a formula comes up with say 5/6 - it will automatically round it to 1. I have a calculation: xyz = ((1300 - [abc intValue])/6) + 100; xyz is defined as an NSInteger, abc is an NSString that is chosen via a UIPicker. I want the calculation (1300 - [abc intValue]) to add 1 to 100 for each 6 units below 1300. For example, 1255 should result in xyz having a value of 100 and 1254 should result in a value of 101. Now, I understand that my formula above is wrong because of the rounding principles, but I am getting some CRAZY results from the program itself. When I punched in 1259 - I got 106. When I punched in 1255 - I got 107. Why would it behave that way?

    Read the article

  • Why does 99.99 / 100 = 0.9998999999999999

    - by the-locster
    Whereas 99.99 * 0.01 = 0.99 Clearly this is the age old floating point rounding issue, however the rounding error in this case seems quite large to me; what I mean is I might have expected a result of 0.99990000001 or some similar 'close' result. And for the record I got the same answer in a JavaVM and in a .Net environment.

    Read the article

  • datediff rounding

    - by derekcohen
    I have a db table in SQL Server which contains a start date for a project. On a web status page I want to show how many days/weeks/months the project has run, the units depending on the duration. So under 21 days I'd show days, under 7 weeks I'd show weeks, otherwise show completed months. So I get the days, weeks and months values and can then use some code to decide which one to display. Suppose the project starts on 30 Dec 2010 and I'm checking today (27 Feb 2011). select datediff(d,'30 Dec 2010',getdate()) as days, datediff(wk,'30 Dec 2010',getdate()) as weeks , datediff(m,'30 Dec 2010',getdate())as months produces days: 59 weeks: 9 months: 2 But in fact the difference is 8 whole weeks and some rounding takes place. I've tried doing it in ASP as well, getting the start date and then doing the datediff() but it's no better. Is there a better way? thanks

    Read the article

  • C# rounding DateTime objects

    - by grenade
    I want to round dates/times to the nearest interval for a charting application. I'd like an extension method signature like follows so that the rounding can be acheived for any level of accuracy: static DateTime Round(this DateTime date, TimeSpan span); The idea is that if I pass in a timespan of ten minutes, it will round to the nearest ten minute interval. I can't get my head around the implementation and am hoping one of you will have written or used something similar before. I think either a floor, ceiling or nearest implementation is fine. Any ideas? Edit: Thanks to @tvanfosson & @ShuggyCoUk, the implementation looks like this: public static class DateExtensions { public static DateTime Round(this DateTime date, TimeSpan span) { long ticks = (date.Ticks / span.Ticks) + (span.Ticks / 2) + 1; return new DateTime(ticks * span.Ticks); } public static DateTime Floor(this DateTime date, TimeSpan span) { long ticks = (date.Ticks / span.Ticks); return new DateTime(ticks * span.Ticks); } public static DateTime Ceil(this DateTime date, TimeSpan span) { long ticks = (date.Ticks + span.Ticks - 1) / span.Ticks; return new DateTime(ticks * span.Ticks); } } And is called like so: DateTime nearestHour = DateTime.Now.Round(new TimeSpan(1,0,0)); DateTime minuteCeiling = DateTime.Now.Ceil(new TimeSpan(0,1,0)); DateTime weekFloor = DateTime.Now.Floor(new TimeSpan(7,0,0,0)); ... Cheers!

    Read the article

  • c++ float subtraction rounding error

    - by Volkan Ozyilmaz
    I have a float value between 0 and 1. I need to convert it with -120 to 80. To do this, first I multiply with 200 after 120 subtract. When subtract is made I had rounding error. Let's look my example. float val = 0.6050f; val *= 200.f; Now val is 121.0 as I expected. val -= 120.0f; Now val is 0.99999992 I thought maybe I can avoid this problem with multiplication and division. float val = 0.6050f; val *= 200.f; val *= 100.f; val -= 12000.0f; val /= 100.f; But it didn't help. I have still 0.99 on my hand. Is there a solution for it? Edit: After with detailed logging, I understand there is no problem with this part of code. Before my log shows me "0.605", after I had detailed log and I saw "0.60499995946884155273437500000000000000000000000000" the problem is in different place.

    Read the article

  • What is the best way to find the digit at n position in a decimal number?

    - by Elijah
    Background I'm working on a symmetric rounding class and I find that I'm stuck with regards to how to best find the number at position x that I will be rounding. I'm sure there is an efficient mathematical way to find the single digit and return it without having to resort to string parsing. Problem Suppose, I have the following (C#) psuedo-code: var position = 3; var value = 102.43587m; // I want this no ? (that is 5) protected static int FindNDigit(decimal value, int position) { // This snippet is what I am searching for } Also, it is worth noting that if my value is a whole number, I will need to return a zero for the result of FindNDigit. Does anyone have any hints on how I should approach this problem? Is this something that is blaringly obvious that I'm missing?

    Read the article

  • Turn off iPhone/Safari input element rounding

    - by Alex
    My website renders well on the iPhone/Safari browser, with one exception: My text input fields have a weird rounded style which doesn't look good at all with the rest of my website. Is there a way to instruct Safari (via CSS or metadata) not to round the input fields and render them rectangular as intended?

    Read the article

  • T-SQL - Date rounding and normalization

    - by arun prakash
    Hi: I have a stored procedure that rounds a column with dates in (yyyy:mm:dd hh:mM:ss) to the nearest 10 minute handle (yyyy:mm:dd hh:mM) 20100303 09:46:3000 ------ 20100303 09:50 but i want to chage it to round it off to the nearest 15 minute handle: 20100303 09:46:3000 ------20100303 09:45 here is my code : IF OBJECT_ID(N'[dbo].[SPNormalizeAddWhen]') IS NOT NULL DROP PROCEDURE [dbo].[SPNormalizeAddWhen] GO CREATE PROCEDURE [dbo].[SPNormalizeAddWhen] As declare @colname nvarchar(20) set @colname='Normalized Add_When' if not exists (select * from syscolumns where id=object_id('Risk') and name=@colname) exec('alter table Risk add [' + @colname + '] datetime') declare @sql nvarchar(500) set @sql='update Risk set [' + @colname + ']=cast(DATEPART(yyyy,[add when]) as nvarchar(4)) + ''-'' + cast(DATEPART(mm,[add when]) as nvarchar(2)) + ''-'' + cast(DATEPART(dd,[add when]) as nvarchar(2)) + '' '' + cast(DATEPART(Hh,[add when]) as nvarchar(2)) + '':'' + cast(round(DATEPART(Mi,[add when]),-1) as nvarchar(2)) ' print @sql exec(@sql) GO

    Read the article

  • T-SQL - Date rounding and normalization

    - by arun prakash
    Hi: I have a stored procedure that rounds a column with dates in (yyyy:mm:dd hh:mM:ss) to the nearest 10 minute handle (yyyy:mm:dd hh:mM) 20100303 09:46:3000 ------ 20100303 09:50 but i want to chage it to round it off to the nearest 15 minute handle: 20100303 09:46:3000 ------20100303 09:45 here is my code : IF OBJECT_ID(N'[dbo].[SPNormalizeAddWhen]') IS NOT NULL DROP PROCEDURE [dbo].[SPNormalizeAddWhen] GO CREATE PROCEDURE [dbo].[SPNormalizeAddWhen] As declare @colname nvarchar(20) set @colname='Normalized Add_When' if not exists (select * from syscolumns where id=object_id('Risk') and name=@colname) exec('alter table Risk add [' + @colname + '] datetime') declare @sql nvarchar(500) set @sql='update Risk set [' + @colname + ']=cast(DATEPART(yyyy,[add when]) as nvarchar(4)) + ''-'' + cast(DATEPART(mm,[add when]) as nvarchar(2)) + ''-'' + cast(DATEPART(dd,[add when]) as nvarchar(2)) + '' '' + cast(DATEPART(Hh,[add when]) as nvarchar(2)) + '':'' + cast(round(DATEPART(Mi,[add when]),-1) as nvarchar(2)) ' print @sql exec(@sql) GO

    Read the article

  • Rounding Decimals to whole numbers

    - by USDblades
    I was looking around and could not find exactly what I was looking for. I want to round all my numbers up to the whole number. Example: 5.9 would be 6 5.5 would be 6 5.1 would be 6 5.000001122 would be 6 5.0 would be 5 I was thinking if I put them into ints that would get rid of the decimal but it did not look right as the decimals were just dropping off. Am I correct here? So I thought about just doing that then adding 1 to the final result which would fix about 99% of the problem but if my result is 5 I do not want to add 1 to it. How would I go about fixing this issue I have?

    Read the article

  • Rounding up milliseconds when printing with Joda Time

    - by RoToRa
    I'm implementing a count-down using Joda Time. I only need a display accuracy of seconds. However when printing the time, the seconds are displayed as full seconds, so when the count down reaches, for example, 900ms, then "0" seconds is printed, but as a count-down it would make more sense to display "1" second, until the time actually reaches 0ms. Example: void printDuration(Duration d) { System.out.println( d.toPeriod(PeriodType.time()).toString( new PeriodFormatterBuilder().printZeroAlways().appendSeconds().toFormatter() ) ); } printDuration(new Duration(5000)); // Prints "5" => OK printDuration(new Duration(4900)); // Prints "4" => need "5" printDuration(new Duration(1000)); // Prints "1" => OK printDuration(new Duration(900)); // Prints "0" => need "1" printDuration(new Duration(0)); // Prints "0" => OK Basically I need to the seconds to be display rounded up from milliseconds and not rounded down. Is there a way to achieve this with Joda without needing to write my own formatter?

    Read the article

  • How to deal with Rounding-off TimeSpan?

    - by infant programmer
    I take the difference between two DateTime fields, and store it in a TimeSpan variable, Now I have to round-off the TimeSpan by the following rules: if the minutes in TimeSpan is less than 30 then Minutes and Seconds must be set to zero, if the minutes in TimeSpan is equal to or greater than 30 then hours must be incremented by 1 and Minutes and Seconds must be set to zero. TimeSpan can also be a negative value, so in that case I need to preserve the sign.. I could be able to achieve the requirement if the TimeSpan wasn't a negative value, though I have written a code I am not happy with its inefficiency as it is more bulky .. Please suggest me a simpler and efficient method. Thanks regards,

    Read the article

  • MPFR Rounding 0.9999 to 1?

    - by Silmaersti
    I'm attempting to store the value 0.9999 into an mpfr_t variable But 0.9999 is rounded to 1 (or some other value != 0.9999) during storage, no matter the round value (GMP_RNDD, GMP_RNDU, GMP_RNDN, GMP_RNDZ) So what's the best method to store 0.9999 in an mpfr_t variable? Is it possible? Here is my test program, it prints "buffer is: 1", instead of the wanted "buffer is: 0.9999": int main() { size_t precision = 4; mpfr_t mpfrValue; mpfr_init2(mpfrValue, precision); mpfr_set_str(mpfrValue, "0.9999", 10, GMP_RNDN); char *buffer = (char*)malloc((sizeof(char) * precision) + 3); mp_exp_t exponent; mpfr_get_str(buffer, &exponent, 10, precision, mpfrValue, GMP_RNDN); printf("buffer is: %s\n", buffer); free(buffer); mpfr_clear(mpfrValue); return 0; } Thanks for any help !

    Read the article

  • C - Rounding number up

    - by Dave
    Hi all, I was curious to know how I can round a number to the nearest tenth. For instance If I had int a = 59 / 4 /* which would be 14.75 and how can i Store the number as 15 in "a"*/ Thanks, Dave

    Read the article

  • SSRS Issue: Rounding to nearest .25

    - by D.R.
    I have an SSRS (2008) report that takes in a raw transactions, then groups and totals them. At the "Total" level, I would like to round the final numbers to the nearest .25, however I cannot find a method to do this. According to what I've read, the Round() function in SSRS only rounds to integers. I have found a couple ways to do it in SQL, but the problem is, I want to do all the calculations with the REAL numbers and just round the result so that I don't introduce a significant amount of error from the real numbers. Here's the best SQL solution I could find: dec(round(number * 4, 0)/4,11,2) as Nearest_Qtr Anyone know how I could do the equivalent in the actual SSRS report? Thanks in advance for the help!

    Read the article

  • Jmagick rounding corners

    - by SPC
    Hello, I've a question concerning Jmagick, how can I round corners of an image, and does this make sense or would it be better to do this using CSS in HTML ? Thanks

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >