Search Results

Search found 26810 results on 1073 pages for 'fixed point'.

Page 6/1073 | < Previous Page | 2 3 4 5 6 7 8 9 10 11 12 13  | Next Page >

  • floating point equality in Python and in general

    - by eric.frederich
    I have a piece of code that behaves differently depending on whether I go through a dictionary to get conversion factors or whether I use them directly. The following piece of code will print 1.0 == 1.0 -> False But if you replace factors[units_from] with 10.0 and factors[units_to ] with 1.0 / 2.54 it will print 1.0 == 1.0 -> True #!/usr/bin/env python base = 'cm' factors = { 'cm' : 1.0, 'mm' : 10.0, 'm' : 0.01, 'km' : 1.0e-5, 'in' : 1.0 / 2.54, 'ft' : 1.0 / 2.54 / 12.0, 'yd' : 1.0 / 2.54 / 12.0 / 3.0, 'mile' : 1.0 / 2.54 / 12.0 / 5280, 'lightyear' : 1.0 / 2.54 / 12.0 / 5280 / 5.87849981e12, } # convert 25.4 mm to inches val = 25.4 units_from = 'mm' units_to = 'in' base_value = val / factors[units_from] ret = base_value * factors[units_to ] print ret, '==', 1.0, '->', ret == 1.0 Let me first say that I am pretty sure what is going on here. I have seen it before in C, just never in Python but since Python in implemented in C we're seeing it. I know that floating point numbers will change values going from a CPU register to cache and back. I know that comparing what should be two equal variables will return false if one of them was paged out while the other stayed resident in a register. Questions What is the best way to avoid problems like this?... In Python or in general. Am I doing something completely wrong? Side Note This is obviously part of a stripped down example but what I'm trying to do is come with with classes of length, volume, etc that can compare against other objects of the same class but with different units. Rhetorical Questions If this is a potentially dangerous problem since it makes programs behave in an undetermanistic matter, should compilers warn or error when they detect that you're checking equality of floats Should compilers support an option to replace all float equality checks with a 'close enough' function? Do compilers already do this and I just can't find the information.

    Read the article

  • How to correctly pass a float from C# to C++ (dll)

    - by RavelT
    I'm getting huge differences when I pass a float from C# to C++. I'm passing a dynamic float wich changes over time. With a debuger I get this: c++ lonVel -0.036019072 float c# lonVel -0.029392920 float I did set my MSVC++2010 floating point model to /fp:fast wich should be the standard in .NET if I'm not mistaken, but this didnt help. Now I cant give out the code but I can show a fraction of it. From C# side it looks like this: namespace Example { public class Wheel { public bool loging = true; #region Members public IntPtr nativeWheelObject; #endregion Members public Wheel() { this.nativeWheelObject = Sim.Dll_Wheel_Add(); return; } #region Wrapper methods public void SetVelocity(float lonRoadVelocity,float latRoadVelocity){Sim.Dll_Wheel_SetVelocity(this.nativeWheelObject,lonRoadVelocity,latRoadVelocity);} #endregion Wrapper methods } internal class Sim { #region PInvokes [DllImport(pluginName, CallingConvention=CallingConvention.Cdecl)] public static extern void Dll_Wheel_SetVelocity(IntPtr wheel,float lonRoadVelocity,float latRoadVelocity); #endregion PInvokes } } And in C++ side @ exportFunctions.cpp: EXPORT_API void Dll_Wheel_SetVelocity(CarWheel* wheel,float lonRoadVelocity,float latRoadVelocity){ wheel->SetVelocity(lonRoadVelocity,latRoadVelocity);} So any sugestions on what I should do in order to get 1:1 results or atleast 99% correct results.

    Read the article

  • Taking fixed direction on hemisphere and project to normal (openGL)

    - by Maik Xhani
    I am trying to perform sampling using hemisphere around a surface normal. I want to experiment with fixed directions (and maybe jitter slightly between frames). So I have those directions: vec3 sampleDirections[6] = {vec3(0.0f, 1.0f, 0.0f), vec3(0.0f, 0.5f, 0.866025f), vec3(0.823639f, 0.5f, 0.267617f), vec3(0.509037f, 0.5f, -0.700629f), vec3(-0.509037f, 0.5f, -0.700629), vec3(-0.823639f, 0.5f, 0.267617f)}; now I want the first direction to be projected on the normal and the others accordingly. I tried these 2 codes, both failing. This is what I used for random sampling (it doesn't seem to work well, the samples seem to be biased towards a certain direction) and I just used one of the fixed directions instead of s (here is the code of the random sample, when i used it with the fixed direction i didn't use theta and phi). vec3 CosWeightedRandomHemisphereDirection( vec3 n, float rand1, float rand2 ) float theta = acos(sqrt(1.0f-rand1)); float phi = 6.283185f * rand2; vec3 s = vec3(sin(theta) * cos(phi), sin(theta) * sin(phi), cos(theta)); vec3 v = normalize(cross(n,vec3(0.0072, 1.0, 0.0034))); vec3 u = cross(v, n); u = s.x*u; v = s.y*v; vec3 w = s.z*n; vec3 direction = u+v+w; return normalize(direction); } ** EDIT ** This is the new code vec3 FixedHemisphereDirection( vec3 n, vec3 sampleDir) { vec3 x; vec3 z; if(abs(n.x) < abs(n.y)){ if(abs(n.x) < abs(n.z)){ x = vec3(1.0f,0.0f,0.0f); }else{ x = vec3(0.0f,0.0f,1.0f); } }else{ if(abs(n.y) < abs(n.z)){ x = vec3(0.0f,1.0f,0.0f); }else{ x = vec3(0.0f,0.0f,1.0f); } } z = normalize(cross(x,n)); x = cross(n,z); mat3 M = mat3( x.x, n.x, z.x, x.y, n.y, z.y, x.z, n.z, z.z); return M*sampleDir; } So if my n = (0,0,1); and my sampleDir = (0,1,0); shouldn't the M*sampleDir be (0,0,1)? Cause that is what I was expecting.

    Read the article

  • Given a start and end point, how can I constrain the end point so the resulting line segment is horizontal, vertical, or 45 degrees?

    - by GloryFish
    I have a grid of letters. The player clicks on a letter and drags out a selection. Using Bresenham's Algorithm I can create a line of highlighted letters representing the player's selection. However, what I really want is to have the line segment be constrained to 45 degree angles (as is common for crossword-style games). So, given a start point and an end point, how can I find the line that passes through the start point and is closest to the end point? Bonus: To make things super sweet I'd like to get a list of points in the grid that the line passes through, and for super MEGA bonus points, I'd like to get them in order of selection (i.e. from start point to end point).

    Read the article

  • About floating point precision and why do we still use it

    - by system_is_b0rken
    Floating point has always been troublesome for precision on large worlds. This article explains behind-the-scenes and offers the obvious alternative - fixed point numbers. Some facts are really impressive, like: "Well 64 bits of precision gets you to the furthest distance of Pluto from the Sun (7.4 billion km) with sub-micrometer precision. " Well sub-micrometer precision is more than any fps needs (for positions and even velocities), and it would enable you to build really big worlds. My question is, why do we still use floating point if fixed point has such advantages? Most rendering APIs and physics libraries use floating point (and suffer it's disadvantages, so developers need to get around them). Are they so much slower? Additionally, how do you think scalable planetary engines like outerra or infinity handle the large scale? Do they use fixed point for positions or do they have some space dividing algorithm?

    Read the article

  • New and Noteworthy Fixed Assets Notes

    - by Oracle_EBS
    A new white paper for Integrating Oracle Inventory Transactions Into Oracle Projects To Generate Asset Lines & Interface Assets To Fixed Assets (Doc ID 1392743.1) A listing of available Oracle E-Business Fixed Assets Diagnostics (Doc ID 1362875.1) Information on the knowledge management enhancements made in My Oracle Support Knowledge Management Version 6.0 Release (Doc ID 1393516.1) The new Period Close Advisor for the Release 12 E-Business Suite (Doc ID 335.1).  What is the Period Close Advisor?  The Period Close Advisor provides guidance on recommended period end procedures for E-Business Release 12.x.  It is intended to be generic and does not relate to a specific organization or industry.  Step by step best practices with tips and troubleshooting references are provided to assist you through each phase.  The EBS R12 Period Close Advisor for Assets data can also be found in a standalone note (Doc ID 1359475.1)

    Read the article

  • Decimal point issue on cocoa app

    - by Manuel Rocha
    I there, I'm trying making my first cocoa app, but I'm having problems with float numbers because of the regional settings. If I write on the TextBox the float number 1.2 I only can get the number 1, but If I write on the same TextBox the same float number but this time with the ',' sign instead (1,2) I can get the right float value. How can I bypass the regional settings? Kind Regards, Manuel Rocha

    Read the article

  • understanding floating point variables

    - by Syom
    There is some problem, i can't understand anyway. look at this code please <script type="text/javascript"> function math(x) { var y; y = x*10; alert(y); } </script> <input type="button" onclick="math(0.011)"> What must be alerted after i click on button? i think 0.11, but no, it alerts 0.10999999999999999 explain please this behavior. thanks in advance

    Read the article

  • C++ floating point precision

    - by Davinel
    double a = 0.3; std::cout.precision(20); std::cout << a << std::endl; result: 0.2999999999999999889 double a, b; a = 0.3; b = 0; for (char i = 1; i <= 50; i++) { b = b + a; }; std::cout.precision(20); std::cout << b << std::endl; result: 15.000000000000014211 So.. 'a' is smaller than it should be. But if we take 'a' 50 times - result will be bigger than it should be. Why is this? And how to get correct result in this case?

    Read the article

  • how IEEE-754 floating point numbers work

    - by hatorade
    Let's say I have this: float i = 1.5 in binary, this float is represented as: 0 01111111 10000000000000000000000 I broke up the binary to represent the 'signed', 'exponent' and 'fraction' chunks. What I don't understand is how this represents 1.5. The exponent is 0 once you subtract the bias (127 - 127), and the fraction part with the implicit leading one is 1.1. How does 1.1 scaled by nothing = 1.5???

    Read the article

  • floating point precision in ruby on rails model validations

    - by Chris Allison
    Hello I am trying to validate a dollar amount using a regex: ^[0-9]+\.[0-9]{2}$ This works fine, but whenever a user submits the form and the dollar amount ends in 0(zero), ruby(or rails?) chops the 0 off. So 500.00 turns into 500.0 thus failing the regex validation. Is there any way to make ruby/rails keep the format entered by the user, regardless of trailing zeros?

    Read the article

  • how floating point numbers work in C

    - by hatorade
    Let's say I have this: float i = 1.5 in binary, this float is represented as: 0 01111111 10000000000000000000000 I broke up the binary to represent the 'signed', 'exponent' and 'fraction' chunks. What I don't understand is how this represents 1.5. The exponent is 0 once you subtract the bias (127 - 127), and the fraction part with the implicit leading one is 1.1. How does 1.1 scaled by nothing = 1.5???

    Read the article

  • Rounding floating-point numbers to 4 decimal points

    - by Himadri
    I have two decimal numbers. I want those number to be same upto 4 decimal points without rounding. If numbers are different I want 2nd number to be replaced by 1st. What if condition should I write? Eg, 1. num1 = 0.94618976 num2 = 0.94620239 If we round these numbers upto 4 decimal then we get 0.9462 same number, but I don't want to round these numbers. 2. num1 = 0.94620239 num2 = 0.94639125 The one way I found is take absolute difference of both numbers say diff and then check the value. My problem is of checking the range of diff. Thank You.

    Read the article

  • Floating Point Arithmetic - Modulo Operator on Double Type

    - by CrimsonX
    So I'm trying to figure out why the modulo operator is returning such a large unusual value. If I have the code: double result = 1.0d % 0.1d; it will give a result of 0.09999999999999995. I would expect a value of 0 Note this problem doesn't exist using the dividing operator - double result = 1.0d / 0.1d; will give a result of 10.0, meaning that the remainder should be 0. Let me be clear: I'm not surprised that an error exists, I'm surprised that the error is so darn large compared to the numbers at play. 0.0999 ~= 0.1 and 0.1 is on the same order of magnitude as 0.1d and only one order of magnitude away from 1.0d. Its not like you can compare it to a double.epsilon, or say "its equal if its < 0.00001 difference". I've read up on this topic on StackOverflow, in the following posts one two three, amongst others. Can anyone suggest explain why this error is so large? Any any suggestions to avoid running into the problems in the future (I know I could use decimal instead but I'm concerned about the performance of that).

    Read the article

  • Floating point precision and physics calculations

    - by Vee
    The gravity Vector2 in my physics world is (0; 0.1). The number 0.1 is known to be problematic, since "it cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4". Having this value for the gravity gives me problems with collisions and creates quite nasty bugs. Changing the value to 0.11 solves these problems. Is there a more elegant solution that doesn't require changing the value at all?

    Read the article

  • Rejigging a floating point equation ...

    - by Jamie
    I'd like to know if there is a way to improve the accuracy of calculating a slope. (This came up a few months back here). It seems by changing: float get_slope(float dXa, float dXb, float dYa, float dYb) { return (dXa - dXb)/(dYa - dYb); } to float get_slope(float dXa, float dXb, float dYa, float dYb) { return dXa/(dYa - dYb) - dXb/(dYa - dYb); } might be an improvement. Suggestions? Edit: It's precision I'm after, not efficiency.

    Read the article

  • Floating point comparison in STL, BOOST

    - by Paul
    Is there in the STL or in Boost a set of generic simple comparison functions? The one I found are always requiring template parameters, and/or instantiation of a struct template. I'm looking for something with a syntax like : if ( is_greater(x,y) ) { ... } Which could be implemented as : template <typename T> bool is_greater(const T& x, const T& y) { return x > y + Precision<T>::eps; }

    Read the article

  • Floating point precision nuances.

    - by user247077
    Hi, I found this code in NVIDIA's cuda SDK samples. void computeGold( float* reference, float* idata, const unsigned int len) { reference[0] = 0; double total_sum = 0; unsigned int i; for( i = 1; i < len; ++i) { total_sum += idata[i-1]; reference[i] = idata[i-1] + reference[i-1]; } // Here it should be okay to use != because we have integer values // in a range where float can be exactly represented if (total_sum != reference[i-1]) printf("Warning: exceeding single-precision accuracy. Scan will be inaccurate.\n"); } (C) Nvidia Corp Can somebody please tell me a case where the warning would be printed, and most importantly, why. Thank you very much.

    Read the article

  • Time difference in seconds (as a floating point)

    - by pocoa
    >>> from datetime import datetime >>> t1 = datetime.now() >>> t2 = datetime.now() >>> delta = t2 - t1 >>> delta.seconds 7 >>> delta.microseconds 631000 Is there any way to get that as 7.631000 ? I can use time module, but I also need that t1 and t2 variables as DateTime objects. So if there is a way to do it with datettime, that would be great.

    Read the article

  • Why is a fixed size buffers (arrays) must be unsafe?

    - by brickner
    Let's say I want to have a value type of 7 bytes (or 3 or 777). I can define it like that: public struct Buffer71 { public byte b0; public byte b1; public byte b2; public byte b3; public byte b4; public byte b5; public byte b6; } A simpler way to define it is using a fixed buffer public struct Buffer72 { public unsafe fixed byte bs[7]; } Of course the second definition is simpler. The problem lies with the unsafe keyword that must be provided for fixed buffers. I understand that this is implemented using pointers and hence unsafe. My question is why does it have to be unsafe? Why can't C# provide arbitrary constant length arrays and keep them as a value type instead of making it a C# reference type array or unsafe buffers?

    Read the article

  • iPhone/Safari: fixed div sticks in the middle of the page when coming back by "history.back(-1)"

    - by praegustator
    Having a problem with fixed div using iPhone's Safari. I know that in iOS 5 the functionality for position:fixed have been added. My div is positioned at the top of the screen and behave pretty good, anyway, there are some bugs during scrolling. But what actually makes me angry - it is the position of this div after coming back to the current page clicking on javascript:history.back(-1) link. The fixed div sticks in the middle of a screen. When I try to scrool, it jumps back on the right position. Is there any cure for the bug?

    Read the article

  • Silverlight layout hack: Centered content with fixed maxwidth

    - by brainbox
     Today we need to create centered content with fixed maxwidth. It is very easy to implement it for fixed width, but is not clear how to achieve the same for maxwidth.The solution to the problem is Grid with 3 columns: <Grid>      <Grid.ColumnDefenitions>            <ColumnDefenition Width="0.01*" />             <ColumnDefenition Width="0.98*" MaxWidth="1280" />            <ColumnDefenition Width="0.01*" />      </Grid.ColumnDefenitions> </Grid>Huh... like html coding xaml coding is still full of dirty tricks =)

    Read the article

  • What's the best way to use requestAnimationFrame and fixed frame rates

    - by m90
    I recently got into using the HTML5-requestAnimationFrame-API a lot on animation-heavy websites, especially after seeing the Jank Busters talk. This seems to work pretty well and really improve performance in many cases. Yet one question still persists for me: When wanting to use an animation that is NOT entirely calculated (think spritesheets for example) you will have to aim for a fixed frame rate. Of course one could go back to use setInterval again, but maybe there are other ways to tackle this. The two ways I could think of using requestAnimationFrame with a fixed frame rate are: var fps = 25; //frames per second function animate(){ //actual drawing goes here setTimeout(function(){ requestAnimationFrame(animate); }, 1000 / fps) } animate(); or var fps = 25; //frames per second var lastExecution = new Date().getTime(); function animate(){ var now = new Date().getTime(); if ((now - lastExecution) > (1000 / fps)){ //do actual drawing lastExecution = new Date().getTime(); } requestAnimationFrame(animate); } animate(); Personally, I'd opt for the second option (the first one feels like cheating), yet it seems to be more buggy in certain situations. Is this approach really worth it (especially at low frame rates like 12.5)? Are there things to be improved? Is there another way to tackle this?

    Read the article

< Previous Page | 2 3 4 5 6 7 8 9 10 11 12 13  | Next Page >