Fixing VBSCRIPT inaccurate mathematical results due to rounding

Posted by jay on Stack Overflow See other posts from Stack Overflow or by jay
Published on 2010-03-29T04:08:21Z Indexed on 2010/03/29 4:53 UTC
Read the original article Hit count: 532

Filed under:

vbscript

|

math

Try running this in a .VBS file

MsgBox(545.14-544.94)

You get a neat little answer of 0.199999999999932!

This rounding issue also occurs unfortunately in Sin(2 * pi) since VB can only ever see the (user defined) variable pi as accurate as 3.14159265358979.

Is rounding it manually (and loosing accuracy) the only way to improve the result? What is the most effective way of dealing with this kind of problem?

© Stack Overflow or respective owner

Related posts about vbscript

Associate mount points with local disks in VBscript / WMI

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a VBscript that outputs various config items about a system. Both hardware and software. I can output disks and their associated partitions. I can output mount points. I do not seem to be able to associate a mount point with a local disk (where it actually is a local disk). I need to be able… >>> More
Any other preprocessor directives in VBScript/Classic ASP?

as seen on Stack Overflow - Search for 'Stack Overflow'
The only pre-process directive that I know about in VBScript / Classic ASP is the #include. I don't know if that is the official name but I'm basically looking for code that can execute code or other instructions before the general VBScript. Are there any other such directives in VBScript? Such as… >>> More
how to pass and access command line arguments in VBscript

as seen on Stack Overflow - Search for 'Stack Overflow'
how to pass and access command line arguments in VBscript >>> More
[OT] : Windows Activation, en masse

as seen on SQL Blog - Search for 'SQL Blog'
This weekend I discovered a minor issue in one of my virtual environments. I had built out 100 VMs based on a Hyper-V template, but I forgot to activate the original source before creating the template, so all of the machines were suddenly out of compliance. While easy enough on a one- or two-machine… >>> More
Retrieve user details from Active Directory using SID

as seen on Server Fault - Search for 'Server Fault'
Hi, How can I find a user in my AD when I have his/her SID. I don't want to rely on other attributes, since I am trying to detect changes to these. Example: I get a message about a change to user record containing: Message: User Account Changed: Target Account Name: test12 Target… >>> More

Related posts about math

how to make a function recursive

as seen on Stack Overflow - Search for 'Stack Overflow'
i have this huge function and i am wondering how to make it recursive. i have the base case which should never come true, so it should always go to else and keep calling itself with the variable t increases. any help would be great thanks def draw(x, y, t, planets): if 'Satellites' in planets["Moon"]: … >>> More
Sunrise / set calculations

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to calculate the sunset / rise times using python based on the link provided below. My results done through excel and python do not match the real values. Any ideas on what I could be doing wrong? My Excel sheet can be found under .. http://transpotools.com/sun_time.xls # Created on… >>> More
Android-Java: Constructing a triangle based on Coordinates on a map and your bearing

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Guys, I'm constructing a geolocation based application and I'm trying to figure out a way to make my application realise when a user is facing the direction of the given location (a particular long / lat co-ord). I've got the math figured, I just have the triangle to construct. //UPDATE So… >>> More
Math.max and Math.min outputting highest and lowest values allowed

as seen on Stack Overflow - Search for 'Stack Overflow'
so I'm trying to make a program that will output the sum, average, and smallest and largest values. I have everything basically figured out except the smallest and largest values are outputting 2147483647 and -2147483647, which I believe are the absolute smallest and largest values that Java will… >>> More
Is Programming == Math?

as seen on Stack Overflow - Search for 'Stack Overflow'
I've heard many times that all programming is really a subset of math. Some suggest that OO, at its roots, is mathematically based. I don't get the connection. Aside from some obvious examples: using induction to prove a recursive algorithm formal correctness proofs functional languages lambda calculus asymptotic… >>> More