Search Results

Search found 218 results on 9 pages for 'rounding'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 4/9 | < Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >

  • MySQL: Matching inexact values using "ON"

    - by Brad
    I'm way out of my league here... I have a mapping table (table1) to assign particular values (value) to a whole number (map_nu). My second table (table2), is a collection of averages (avg) (I couldn't figure out how to properly make a markdown table, please feel free to edit!) table1: table2: (value)(Map_nu) (avg) ---- ----- 1 1 1.111 1.045 2 1.2 1.09 3 1.33333 1.135 4 1 1.18 5 1.389 1.225 6 1.42 1.27 7 1.07 1.315 8 1.36 9 1.405 10 I need to find a way to match the averages from table2 to the closest value in table1. It only need to match to the 2 digit past the decimal, so I've added the Truncated function SELECT map_nu FROM `table1` JOIN table2 ON TRUNCATE(table1.value,2)=TRUNCATE(table2.avg,2) I still miss the values that don't match the averages exactly. Is there a way to pick the nearest truncated value? Thanks!

    Read the article

  • The internal storage of a DATETIME2 value

    - by Peter Larsson
    Today I went for investigating the internal storage of DATETIME2 datatype. What I found out was that for a datetime2 value with precision 0 (seconds only), SQL Server need 6 bytes to represent the value, but stores 7 bytes. This is because SQL Server add one byte that holds the precision for the datetime2 value. Start with this very simple repro declare @now datetime2(7) = '2010-12-15 21:04:03.6934231'   select  cast(cast(@now as datetime2(0)) as binary(7)),         cast(cast(@now as datetime2(1)) as binary(7)),         cast(cast(@now as datetime2(2)) as binary(7)),         cast(cast(@now as datetime2(3)) as binary(8)),         cast(cast(@now as datetime2(4)) as binary(8)),         cast(cast(@now as datetime2(5)) as binary(9)),         cast(cast(@now as datetime2(6)) as binary(9)),         cast(cast(@now as datetime2(7)) as binary(9)) Now we are going to copy and paste these binary values and investigate which value is representing what time part. Prefix  Ticks       Ticks         Days    Days    Original value ------  ----------  ------------  ------  ------  -------------------- 0x  00  442801             75844  A8330B  734120  0x00442801A8330B 0x  01  A5920B            758437  A8330B  734120  0x01A5920BA8330B  0x  02  71BA73           7584369  A8330B  734120  0x0271BA73A8330B 0x  03  6D488504        75843693  A8330B  734120  0x036D488504A8330B 0x  04  46D4342D       758436934  A8330B  734120  0x0446D4342DA8330B 0x  05  BE4A10C401    7584369342  A8330B  734120  0x05BE4A10C401A8330B 0x  06  6FEBA2A811   75843693423  A8330B  734120  0x066FEBA2A811A8330B 0x  07  57325D96B0  758436934231  A8330B  734120  0x0757325D96B0A8330B Let us use the following color schema Red - Prefix Green - Time part Blue - Day part What you can see is that the date part is equal in all cases, which makes sense since the precision doesm't affect the datepart. What would have been fun, is datetime2(negative) just like round accepts a negative value. -1 would mean rounding to 10 second, -2 rounding to minute, -3 rounding to 10 minutes, -4 rounding to hour and finally -5 rounding to 10 hour. -5 is pretty useless, but if you extend this thinking to -6, -7 and so on, you could actually get a datetime2 value which is accurate to the month only. Well, enough ranting about this. Let's get back to the table above. If you add 75844 second to midnight, you get 21:04:04, which is exactly what you got in the select statement above. And if you look at it, it makes perfect sense that each following value is 10 times greater when the precision is increased one step too. //Peter

    Read the article

  • HLSL tex2d sampler seemingly using inconsistent rounding; why?

    - by RJFalconer
    Hello all, I have code that needs to render regions of my object differently depending on their location. I am trying to use a colour map to define these regions. The problem is when I sample from my colour map, I get collisions. Ie, two regions with different colours in the colourmap get the same value returned from the sampler. I've tried various formats of my colour map. I set the colours for each region to be "5" apart in each case; Indexed colour RGB, RGBA: region 1 will have RGB 5%,5%,5%. region 2 will have RGB 10%,10%,10% and so on. HSV Greyscale: region 1 will have HSV 0,0,5%. region 2 will have HSV 0,0,10% and so on. (Values selected in The Gimp) The tex2D sampler returns a value [0..1]. [ I then intend to derive an int array index from region. Code to do with that is unrelated, so has been removed from the question ] float region = tex2D(gColourmapSampler,In.UV).x; Sampling the "5%" colour gave a "region" of 0.05098 in hlsl. From this I assume the 5% represents 5/100*255, or 12.75, which is rounded to 13 when stored in the texture OR when sampled by the sampler; can't tell which. (Reasoning: 0.05098 * 255 ~= 13) By this logic, the 50% should be stored as 127.5. Sampled, I get 0.50196 which implies it was stored as 128. the 70% should be stored as 178.5. Sampled, I get 0.698039, which implies it was stored as 178. What rounding is going on here? (127.5 becomes 128, 178.5 becomes 178 ?!) Edit: OK, http://en.wikipedia.org/wiki/Bankers_rounding#Round_half_to_even Apparently this is "banker's rounding". Is this really what HLSL samplers use? I am using Shader Model 2 and FX Composer. This is my sampler declaration; //Colour map texture gColourmapTexture < string ResourceName = "Globe_Colourmap_Regions_Greyscale.png"; string ResourceType = "2D"; >; sampler2D gColourmapSampler : register(s1) = sampler_state { Texture = <gColourmapTexture>; #if DIRECT3D_VERSION >= 0xa00 Filter = MIN_MAG_MIP_LINEAR; #else /* DIRECT3D_VERSION < 0xa00 */ MinFilter = Linear; MipFilter = Linear; MagFilter = Linear; #endif /* DIRECT3D_VERSION */ AddressU = Clamp; AddressV = Clamp; };

    Read the article

  • C# log base 10 and rounding up to nearest power of 10?

    - by Tom
    Hi, if i have a number between 100 and 1000 i want to get the value 3 because 10^3 = 1000. Likewise, if i had a number between 10 and 100 i would want to get the value 2, because 10^2 is 100. Incase you're wondering, its to do with calculating a probability and i always need to divide through by 10^value, to keep the probability between 0 and 1. For example if i calculate 9256, i need to divide through by 10^4, so that i get a probability of 0.92 I'm not sure how to do the rounding up and how to do the base 10, could someone please help?

    Read the article

  • How to prevent Excel rounding numbers or adding redundant 0's?

    - by Highly Irregular
    I have a column of numbers that appear like this: but the actual value of the shown cell is 20130.153334 Other values have a different number of decimal places. I don't want to add redundant 0's, so I can't just specify a particular number of decimal places to display. I really just want to treat the values as text. I have already changed the format of the cell to Text, as the description for Text is: "Text format cells are treated as text even when a number is in the cell. The cell is displayed exactly as entered.". However, it clearly isn't being displayed exactly as entered! Strangely, if I hit F2 on the cell to go into edit mode, then hit enter, it is then displayed correctly. I can't do this manually for 2000+ records though! How can I prevent the numbers being rounded?

    Read the article

  • Rounding issue when adding floats in python, is this normal?

    - by thepearson
    I just wanted to know if this behavior is expected. If so, can someone explain to me why. This has probably been answered elsewhere I can't seem to find it using Google. Probably not searching with the right terms. I am running Ubuntu 10.04. Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> var = 10.0 >>> var 10.0 >>> var + 5 15.0 >>> var + 5.1 15.1 >>> var + 5.2 15.199999999999999 >>>

    Read the article

  • Silverlight Cream for April 08, 2010 -- #834

    - by Dave Campbell
    In this Issue: Michael Washington, Phil Middlemiss, Yochay Kiriaty, Giorgetti Alessandro, Mike Snow, John Papa, SilverLaw, smartyP, and Pete Brown. Shoutouts: Steve Wortham sent me a link to his RegEx tool that is written in Silverlight... definitely worth a look: Introducing Code Hinting for Regular Expressions Joshua Blake posted his MIX10 materials: MIX10 NUI session sample code From SilverlightCream.com: Silverlight MVVM: An (Overly) Simplified Explanation Michael Washington has a tutorial up for getting your arms (and head) around MVVM and Silverlight, and Blend too. A Chrome and Glass Theme - Part 3 Phil Middlemiss has part 3 up of his tutorial series on building an awesome theme for Silverlight... he's styling the textbox and checkbox this time around, and improving the button too Automatic Rotation Support or Automatic Multi-Orientation Layout Support for Windows Phone Yochay Kiriaty is giving up some WP7 goodness with his post on Multi-Orientation Layout Support ... yeah I had to say it twice myself :) good links and all the code in addition to the good blog post Silverlight Navigation Framework: resolve the pages using an IoC container Giorgetti Alessandro has some pretty cool code up as a proof of concept of using an IoC container with the Navigation Framework of Silverlight 4. Silverlight Tip of the Day No. 109 – Attach to Process Debugging Mike Snow is back doing Tips of the Day... and number 109 is showing how to attach the debugger to a running Silverlight app. Silverlight TV 20: Community Driven Development with WCF RIA Services In his latest Silverlight TV episode, John Papa talks with Jeff Handley about RIA Services, and how feedback from the community helped shape the product. ChildWindowMouseScrollResizeBehavior - Silverlight 3 SilverLaw has a new Behavior up at the Expression Gallery that gives you resizing on a ChildWindow using the Mouse Wheel. Creating a Windows Phone 7 Metro Style Pivot Application [Part 3] smartyP has the 3rd and final episode for his WP7 Pivot up, and this one includes not only the source but a video tutorial. Layout Rounding Pete Brown talks about Layout Rounding and it has nothing to do with rounding corners... it has to do with rounding off where your objects get placed pixel-wise ... I've blogged about this seemingly-anti-aliasing more than once... Pete has the real answer Stay in the 'Light! Twitter SilverlightNews | Twitter WynApse | WynApse.com | Tagged Posts | SilverlightCream Join me @ SilverlightCream | Phoenix Silverlight User Group Technorati Tags: Silverlight    Silverlight 3    Silverlight 4    Windows Phone MIX10

    Read the article

  • How should I work out VAT (UK tax) in my eCommerce site?

    - by Leonard Challis
    We have an ecommerce system in place. The sales actually go through Sage, so we have an export script from our system that uses a third-party Sage Importer program. With a new version of this importer, values are checked more thoroughly. We are getting 1 pence discrepancies because of the way rounding works - our system has always held prices and worked to 4 decimal places. In the checkout the totals would be worked out first, then the rounding to 2 decimal places. The importer does rounding first, though. So, for instance: Our way: Product 1: £13.4561 Qty: 2 Total inc VAT = £32.29 (to 2dp) Importer way: Our way: Product 1: £13.4561 Qty: 2 Total inc VAT = £32.30 (to 2dp) Management are reluctant to lose the 4dp but the developers of the Sage importer have said that this is correct and makes sense -- you woudn't sell a product for £13.4561 in a shop, nor would you charge someone tax at 4 decimal places. I contacted the HMRC and the operator didn't really give me much to go on, telling me a technician would phone back, to which they haven't and I'm still waiting after almost a week and numerous follow-up calls. I did find a PDF on the HMRC's web site, but this did about us much to confuse me as it did to answer my questions. I see that they're happy for people to round up or down, as long it is consistent, but I can't tell whether it should be done on a line by line basis or on the end total of the order. We are now in the position where we need to decide whether it's worth us doing one of the following, or something completely different. Please advise with any experience or information I can read. Change all products on the site to use 2dp Keep 4dp but round each line in the order to 2dp before working out tax Keep it as it is and "fudge" the values at the export script (i.e. make that values correct by adding or subtracting 1p and changing the shipping cost to make the totals still work out) Any thoughts?

    Read the article

  • Bitwise Operations -- Arithmetic Operations..

    - by RBA
    Hi, Can you please explain the below lines, with some good examples. A left arithmetic shift by n is equivalent to multiplying by 2n (provided the value does not overflow), while a right arithmetic shift by n of a two's complement value is equivalent to dividing by 2n(2 to the power n) and rounding toward negative infinity. If the binary number is treated as ones' complement, then the same right-shift operation results in division by 2n and rounding toward zero. Thankx..

    Read the article

  • SQL SERVER – Precision of SMALLDATETIME – A 1 Minute Precision

    - by pinaldave
    I am myself surprised that I am writing this post today. I am going to present one of the very known facts of SQL Server SMALLDATETIME datatype. Even though this is a very well-known datatype, many a time, I have seen developers getting confused with precision of the SMALLDATETIME datatype. The precision of the datatype SMALLDATETIME is 1 minute. It discards the seconds by rounding up or rounding down any seconds greater than zero. Let us see the following example DECLARE @varSDate AS SMALLDATETIME SET @varSDate = '1900-01-01 12:12:01' SELECT @varSDate C_SDT SET @varSDate = '1900-01-01 12:12:29' SELECT @varSDate C_SDT SET @varSDate = '1900-01-01 12:12:30' SELECT @varSDate C_SDT SET @varSDate = '1900-01-01 12:12:59' SELECT @varSDate C_SDT Following is the result of the above script and note that any value between 0 (zero) and 59 is converted up or down. The part that confuses the developers is the value of the seconds in the display. I think if it is not maintained or recorded, it should not be displayed as well. Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: Pinal Dave, SQL, SQL Authority, SQL DateTime, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, T SQL, Technology

    Read the article

  • Creating properly aligned partitions on a replacement disk

    - by Marius Gedminas
    I've a typical small office server with two hard disks configured for RAID-1 (mirroring). Each disk has several partitions: one for swap, the others paired in several /dev/mdX arrays. Every couple of years one of the disks dies and is replaced. The replacement typically goes something like this: # copy partition table from the remaining good disk to the empty replacement disk # (instead of /dev/good_disk and /dev/new_disk I use /dev/sda and /dev/sdb, as appropriate) sfdisk -d /dev/good_disk | sfdisk /dev/new_disk # install boot loader grub-install /dev/new_disk # create swap partition reusing the same UUID, so I don't need to edit /etc/fstab mkswap /dev/new_disk1 -U xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # hot-add the new partitions to my RAID arrays mdadm /dev/md0 -a /dev/new_disk2 mdadm /dev/md1 -a /dev/new_disk5 mdadm /dev/md2 -a /dev/new_disk6 mdadm /dev/md3 -a /dev/new_disk7 mdadm /dev/md4 -a /dev/new_disk8 The disks were originally partitioned with cfdisk back in 2009, and so the partition table is aligned traditionally to cylinder boundaries (255 heads * 63 sectors). This is not the optimum configuration for new 4K-sector drives. My question is: how can I create a set of partitions for the new disk and ensure they're properly aligned, and have correct sizes for my RAID arrays (rounding up is acceptable, I suppose, but rounding down is definitely not)?

    Read the article

  • How to handle monetary values in PHP and MySql?

    - by Songo
    I've inherited a huge pile of legacy code written in PHP on top of a MySQL database. The thing I noticed is that the application uses doubles for storage and manipulation of data. Now I came across of numerous posts mentioning how double are not suited for monetary operations because of the rounding errors. However, I have yet to come across a complete solution to how monetary values should be handled in PHP code and stored in a MySQL database. Is there a best practice when it comes to handling money specifically in PHP? Things I'm looking for are: How should the data be stored in the database? column type? size? How should the data be handling in normal addition, subtraction. multiplication or division? When should I round the values? How much rounding is acceptable if any? Is there a difference between handling large monetary values and low ones? Note: A VERY simplified sample code of how I might encounter money values in everyday life: $a= $_POST['price_in_dollars']; //-->(ex: 25.06) will be read as a string should it be cast to double? $b= $_POST['discount_rate'];//-->(ex: 0.35) value will always be less than 1 $valueToBeStored= $a * $b; //--> any hint here is welcomed $valueFromDatabase= $row['price']; //--> price column in database could be double, decimal,...etc. $priceToPrint=$valueFromDatabase * 0.25; //again cast needed or not? I hope you use this sample code as a means to bring out more use cases and not to take it literally of course. Bonus Question If I'm to use an ORM such as Doctrine or PROPEL, how different will it be to use money in my code.

    Read the article

  • ICU MessageFormat Currency Value Precison Lost

    - by Travis
    This may be a niche question but I'm working with ICU to format currency strings. I've bumped into a situation that I don't quite understand. When using the MesssageFormat class, why for a certain locale (Korea "ko"KR" for example) does it round currency values (e.g. 100.50 becomes ?101). For most locales (such as the US "en_US"), the precision of the argument passed in remains untouched (e.g. 100.50 becomes $100.50). I thought this might be a default rounding issue that some locales have (Swiss Francs "fr_CH" for example have a default 0.05 rounding) but South Korea "ko_KR" has none. Any ideas?

    Read the article

  • Can you get access to the NumberFormatter used by ICU MessageFormat

    - by Travis
    This may be a niche question but I'm working with ICU to format currency strings. I've bumped into a situation that I don't quite understand. When using the MesssageFormat class, is it possible to get access to the NumberFormat object it uses to format currency strings. When you create a NumberFormat instance yourself, you can specify attributes like precision and rounding used when creating currency strings. I have an issue where for the South Korean locale ("ko_KR"), the MessageFormat class seems to create currency strings w/ rounding (100.50 - ?100). In areas where I use NumberFormat directly, I set setMaximumFractionDigits and setMinimumFractionDigits to 2 but I can't seem to set this in the MessageFormat. Any ideas?

    Read the article

  • DisplayObject.rotation not matching trig functions

    - by futuraprime
    I'm trying to label an animated pie chart, and I've been having a great deal of trouble getting rotated objects to line up with trigonometrically-positioned objects. So, for example, if I have a pie piece that's middle is angle theta and has been rotated n degrees in a tween, and then I try to position a label with code like this: label.x = center.x + Math.cos((theta + n)/180 * Math.PI) * radius; label.y = center.y + Math.sin((theta + n)/180 * Math.PI) * radius; the label is often not aligned with the center of the pie slice. Since I am also zooming in to the pie chart a great deal, the error becomes significant enough that it occasionally causes the label to miss the pie slice altogether. The error seems relatively unpredictable, and it looks a great deal like a rounding error, but I don't see any obvious rounding going on (the trig functions evaluate to ten or so decimal places, which should be more than enough here). How can I get these labels to position correctly?

    Read the article

  • .NET Decimal = what in SQL?

    - by dmose
    What's the best data type in SQL to represent Decimal in .NET? We want to store decimal numbers with up to 9 decimal place precision and want to avoid rounding errors etc on the front end. Reading about data types, it appears using Decimal in .NET is the best option because you will not get rounding errors, although it is a bit slower than a Double. We want to carry this through down to the DB and want minimum conversion issues when moving data through the layers. Any suggestions?

    Read the article

  • Strange date relationships with #PowerPivot

    - by Marco Russo (SQLBI)
    A reader of my PowerPivot book highlighted a strange behavior of the relationship between a datetime column and a Calendar table. Long story short: it seems that PowerPivot automatically round the date to the “neareast day”, but instead of simply removing the time (truncating the decimal part of the decimal number internally used to represent a datetime value) a rounding function seems used, moving the date to the next day if the time part contain a PM time. As you can imagine, this becomes particularly...(read more)

    Read the article

  • Effective optimization strategies on modern C++ compilers

    - by user168715
    I'm working on scientific code that is very performance-critical. An initial version of the code has been written and tested, and now, with profiler in hand, it's time to start shaving cycles from the hot spots. It's well-known that some optimizations, e.g. loop unrolling, are handled these days much more effectively by the compiler than by a programmer meddling by hand. Which techniques are still worthwhile? Obviously, I'll run everything I try through a profiler, but if there's conventional wisdom as to what tends to work and what doesn't, it would save me significant time. I know that optimization is very compiler- and architecture- dependent. I'm using Intel's C++ compiler targeting the Core 2 Duo, but I'm also interested in what works well for gcc, or for "any modern compiler." Here are some concrete ideas I'm considering: Is there any benefit to replacing STL containers/algorithms with hand-rolled ones? In particular, my program includes a very large priority queue (currently a std::priority_queue) whose manipulation is taking a lot of total time. Is this something worth looking into, or is the STL implementation already likely the fastest possible? Along similar lines, for std::vectors whose needed sizes are unknown but have a reasonably small upper bound, is it profitable to replace them with statically-allocated arrays? I've found that dynamic memory allocation is often a severe bottleneck, and that eliminating it can lead to significant speedups. As a consequence I'm interesting in the performance tradeoffs of returning large temporary data structures by value vs. returning by pointer vs. passing the result in by reference. Is there a way to reliably determine whether or not the compiler will use RVO for a given method (assuming the caller doesn't need to modify the result, of course)? How cache-aware do compilers tend to be? For example, is it worth looking into reordering nested loops? Given the scientific nature of the program, floating-point numbers are used everywhere. A significant bottleneck in my code used to be conversions from floating point to integers: the compiler would emit code to save the current rounding mode, change it, perform the conversion, then restore the old rounding mode --- even though nothing in the program ever changed the rounding mode! Disabling this behavior significantly sped up my code. Are there any similar floating-point-related gotchas I should be aware of? One consequence of C++ being compiled and linked separately is that the compiler is unable to do what would seem to be very simple optimizations, such as move method calls like strlen() out of the termination conditions of loop. Are there any optimization like this one that I should look out for because they can't be done by the compiler and must be done by hand? On the flip side, are there any techniques I should avoid because they are likely to interfere with the compiler's ability to automatically optimize code? Lastly, to nip certain kinds of answers in the bud: I understand that optimization has a cost in terms of complexity, reliability, and maintainability. For this particular application, increased performance is worth these costs. I understand that the best optimizations are often to improve the high-level algorithms, and this has already been done.

    Read the article

  • Putting BigDecimal data into HSQLDB test database using DbUnit

    - by Denise
    Hi everyone, I'm using Hibernate JPA in my backend. I am writing a unit test using JUnit and DBUnit to insert a set of data into an in-memory HSQL database. My dataset contains: <order_line order_line_id="1" quantity="2" discount_price="0.3"/> Which maps to an OrderLine Java object where the discount_price column is defined as: @Column(name = "discount_price", precision = 12, scale = 2) private BigDecimal discountPrice; However, when I run my test case and assert that the discount price returned equals 0.3, the assertion fails and says that the stored value is 0. If I change the discount_price in the dataset to be 0.9, it rounds up to 1. I've checked to make sure HSQLDB isn't doing the rounding and it definitely isn't because I can insert an order line object using Java code with a value like 5.3 and it works fine. To me, it seems like DBUtils is for some reason rounding the number I've defined. Is there a way I can force this to not happen? Can anyone explain why it might be doing this? Thanks!

    Read the article

  • How do calculators work with precision?

    - by zoul
    Hello! I wonder how calculators work with precision. For example the value of sin(M_PI) is not exactly zero when computed in double precision: #include <math.h> #include <stdio.h> int main() { double x = sin(M_PI); printf("%.20f\n", x); // 0.00000000000000012246 return 0; } Now I would certainly want to print zero when user enters sin(p). I can easily round somewhere on 1e–15 to make this particular case work, but that’s a hack, not a solution. When I start to round like this and the user enters something like 1e–20, they get a zero back (because of the rounding). The same thing happens when the user enters 1/10 and hits the = key repeatedly — when he reaches the rounding treshold, he gets zero. And yet some calculators return plain zero for sin(p) and at the same time they can work with expressions such as (1e–20)/10 comfortably. Where’s the trick?

    Read the article

  • How can a single threaded application like Excel 2003 take more than 50% of a hyper-threaded or dual

    - by Lunatik
    I'm waiting for Excel to finish a recalculation and I notice that the CPU usage as reported by Task Manager occasionally spikes to 51% or 52% on a Pentium 4 with hyper-threading. How is a single-threaded application like Excel 2003 doing this? Is it just a rounding/estimation error on the part of Task Manager? Or is it something to do with HT allocation i.e. I wouldn't see this happening on a genuine dual-core or dual-CPU machine?

    Read the article

  • Why don't computers store decimal numbers as a second whole number?

    - by SomeKittens
    Computers have trouble storing fractional numbers where the denominator is something other than a solution to 2^x. This is because the first digit after the decimal is worth 1/2, the second 1/4 (or 1/(2^1) and 1/(2^2)) etc. Why deal with all sorts of rounding errors when the computer could have just stored the decimal part of the number as another whole number (which is therefore accurate?) The only thing I can think of is dealing with repeating decimals (in base 10), but there could have been an edge solution to that (like we currently have with infinity).

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >