I stumbled upon the definition of double.NaN in code:

public const double NaN = (double)0.0 / (double)0.0;

This is done similarly in PositiveInfinity and NegativeInfinity.

double.IsNaN (with removing a few #pragmas and comments) is defined as:

[Pure] 
[ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)]
public static bool IsNaN(double d) 
{
    if (d != d)
    { 
        return true; 
    }
    else 
    {
        return false;
    }
}

This is, by far, the most counterintuitive thing I have ever seen in the .NET framework.

How is 0.0 / 0.0 represented "behind the scenes"? How can division by 0 be possible in double, and why does NaN != NaN?