First of all, this is kind of a follow up question on @byte56 excellent anwser on this question concerning picking algorithms.
I'm trying to convert one of his code examples to directX 11 however I have run into some problems ( I can pick but the picking is way off), and I wanted to make sure I had done it right before moving on and checking the rest of my code. I am not that familiar with openGl but I can imagine openGl has different coordinations systems, and functions that alters how you must implement to code a bit.
The getPickRay function on the answer linked is what I'm trying to convert.
This is the part of my code that I think is giving me trouble when converting from openGl to directX Because I'm unsure on how their different coordination systems differs from one another.
PRVecX = ((( 2.0f * mouseX) / ClientWidth ) - 1 ) * tan((viewAngle)/2);
PRVecY = (1-(( 2.0f * mouseY) / ClientHeight)) * tan((viewAngle)/2);
Another thing that I am unsure about is this part:
XMVECTOR worldSpaceNear = XMVector3TransformCoord(cameraSpaceNear, invMat);
XMVECTOR worldSpaceFar = XMVector3TransformCoord(cameraSpaceFar, invMat);
A couple of notes:
The mouse coordinates are already converted so that the top left corner of the client window would be (0,0) and the bottom right (800,600) ( or whatever resolution you would have)
The viewAngle is the same angle that I used when setting the camera view with XMMatrixPerspectiveFovLH.
I removed the variables aspectRatio and zoomFactor because I assumed that they were related to some specific function of his game.
To summarize it up to questions :
Does the openGL coordination system differ in such a way that this equation in the first of my code examples wouldn't be valid when used in DirectX 11 ( with its respective screen coordination system)?
Is the openGL method Matrix4f.transform(a, b, c) equal to the directX method c = XMVector3TransformCoord(b,a)? (where a is a matrix and b,c are vectors). Because I know when it comes to matrices order is important.
