After switching to VS2010, the managed debug assistant is displaying an error about an unbalanced stack from a call to an unmanaged C++ function from a C# application.
The usuals suspects don't seem to be causing the issue. Is there something else I should check? The VS2008 built C++ dll and C# application never had a problem, no weird or mysterious bugs - yeah, I know that doesn't mean much.
Here are the things that were checked:
The dll name is correct.
The entry point name is correct and has been verified with depends.exe - the code has to use the mangled name and it does.
The calling convention is correct.
The sizes and types all seem to be correct.
The character set is correct.
There doesn't seem to be any issues after ignoring the error and there isn't an issue when running outside the debugger.
C#:
[DllImport("Correct.dll", EntryPoint = "SuperSpecialOpenFileFunc", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Ansi, ExactSpelling = true)]
public static extern short SuperSpecialOpenFileFunc(ref SuperSpecialStruct stuff);
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct SuperSpecialStruct
{
public int field1;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
public string field2;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 20)]
public string field3;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
public string field4;
public ushort field5;
public ushort field6;
public ushort field7;
public short field8;
public short field9;
public uint field10;
public short field11;
};
C++:
short SuperSpecialOpenFileFunc(SuperSpecialStruct * stuff);
struct SuperSpecialStruct
{
int field1;
char field2[256];
char field3[20];
char field4[10];
unsigned short field5;
unsigned short field6;
unsigned short field7;
short field8;
short field9;
unsigned int field10;
short field11;
};
Here is the error:
Managed Debugging Assistant
'PInvokeStackImbalance' has detected a
problem in 'Managed application path'.
Additional Information: A call to
PInvoke function
'SuperSpecialOpenFileFunc' has
unbalanced the stack. This is likely
because the managed PInvoke signature
does not match the unmanaged target
signature. Check that the calling
convention and parameters of the
PInvoke signature match the target
unmanaged signature.
