Pixel Shader, YUV-RGB Conversion failing

Posted by TomTom on Game Development See other posts from Game Development or by TomTom
Published on 2012-08-13T14:53:46Z Indexed on 2013/10/28 22:15 UTC
Read the original article Hit count: 1564

Filed under:
|
|

I am tasked with playing back a video hthat comes in in a YUV format as an overlay in a larger game. I am not a specialist in Direct3d, so I am struggling. I managed to get a shader working and am rendering 3 textures (Y, V, U). Sadly I am totally unable to get anything like a decent image. Documentation is also failing me. I am currently loading the different data planes (Y,V,U) in three different textures:

            m_Textures = new Texture[3];
            // Y Plane
            m_Textures[0] = new Texture(m_Device, w, h, 1, Usage.None, Format.L8, Pool.Managed);
            // V Plane
            m_Textures[1] = new Texture(m_Device, w2, h2, 1, Usage.None, Format.L8, Pool.Managed);
            // U Plane
            m_Textures[2] = new Texture(m_Device, w2, h2, 1, Usage.None, Format.L8, Pool.Managed);

When I am rendering them as R, G and B respectively with the following code:

float4 Pixel( float2 texCoord: TEXCOORD0) : COLOR0 { float y = tex2D (ytexture, texCoord);

float v = tex2D (vtexture, texCoord);

float u = tex2D (utexture, texCoord);

//R = Y + 1.140 (V -128)
//G = Y - 0.395 (U-128) - 0.581 (V-128)
//B = Y + 2.028 (U-128)     

float r = y; //y + 1.140 * v;
float g = v; //y - 0.395 * u - 0.581 * v;
float b = u; //y + 2.028 * u;

float4 result;
result.a = 255;
result.r = r; //clamp (r, 0, 255);
result.g = g; //clamp (g, 0, 255);
result.b = b; //clamp (b, 0, 255);

return result;

}

Then the resulting image is - quite funny. I can see the image, but colors are totally distorted, as it should be.

The formula I should apply shows up in the comment of the pixel shader, but when I do it, the resulting image is pretty brutally magenta only. This gets me to the question - when I read out an L8 texture into a float, with

float y = tex2D (ytexture, texCoord);

what is the range of values? The "origin" values are 1 byte, 0 to 255, and the forum I have assumes this. Naturally I am totally off when the values returned are somehow normalized. My Clamp operation at the end also will fail if for example colors in a pixel shader are normalized 0 to 1. Anyone an idea how that works? Please point me also to documentation - I have not found anything in this regard.

© Game Development or respective owner

Related posts about hlsl

Related posts about pixel-shader