I have just ported my C++ game to OS X and the first time it ran I get the following exception when trying to call SDL_SetVideoMode.
2012-09-28 15:01:05.437 SCRAsteroids[28595:707] * Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Error (1000) creating CGSWindow on line 259'
* First throw call stack:
(
    0   CoreFoundation                      0x00007fff8b53b716 __exceptionPreprocess + 198
    1   libobjc.A.dylib                     0x00007fff90e30470 objc_exception_throw + 43
    2   CoreFoundation                      0x00007fff8b53b4ec +[NSException raise:format:] + 204
    3   AppKit                              0x00007fff8a26a579 _NSCreateWindowWithOpaqueShape2 + 655
    4   AppKit                              0x00007fff8a268d70 -[NSWindow _commonAwake] + 2002
    5   AppKit                              0x00007fff8a2277e2 -[NSWindow _commonInitFrame:styleMask:backing:defer:] + 1763
    6   AppKit                              0x00007fff8a22692f -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1568
    7   AppKit                              0x00007fff8a2262ff -[NSWindow initWithContentRect:styleMask:backing:defer:] + 45
    8   libSDL-1.2.0.dylib                  0x0000000107c228f6 -[SDL_QuartzWindow initWithContentRect:styleMask:backing:defer:] + 294
    9   libSDL-1.2.0.dylib                  0x0000000107c20505 QZ_SetVideoMode + 2837
    10  libSDL-1.2.0.dylib                  0x0000000107c17af5 SDL_SetVideoMode + 917
    11  SCRAsteroids                        0x0000000107be60fb _ZN11SDLGraphics4initEP6IWorldii + 291
)
libc++abi.dylib: terminate called throwing an exception
Abort trap: 6
My init code looks like this:
if (SDL_Init(SDL_INIT_EVERYTHING) < 0)
  return false;
const SDL_VideoInfo *videoInfo = SDL_GetVideoInfo();
if (!videoInfo) {
  fprintf(stderr, "Video query failed: %s\n",
     SDL_GetError());
  return false;
}
/* the flags to pass to SDL_SetVideoMode */
videoFlags = SDL_OPENGL;       /* Enable OpenGL in SDL */
videoFlags |= SDL_GL_DOUBLEBUFFER; /* Enable double buffering */
videoFlags |= SDL_HWPALETTE;       /* Store the palette in hardware */
/* This checks to see if surfaces can be stored in memory */
if (videoInfo->hw_available)
  videoFlags |= SDL_HWSURFACE;
else
  videoFlags |= SDL_SWSURFACE;
if (w == 0) {
  widthViewport = videoInfo->current_w;
  heightViewport = videoInfo->current_h;
  cout << "Will use full screen resolution of ";   
  videoFlags |= SDL_FULLSCREEN;
} else {
  cout << "Will use full user supplied resolution of ";
  widthViewport = w;
  heightViewport = h;
  videoFlags |= SDL_RESIZABLE;       /* Enable window resizing */
}
cout << widthViewport << "x" << heightViewport << "\n";
  /* This checks if hardware blits can be done */
if (videoInfo->blit_hw)
  videoFlags |= SDL_HWACCEL;
/* Sets up OpenGL double buffering */
SDL_GL_SetAttribute(SDL_GL_DOUBLEBUFFER, 1);
/* get a SDL surface */
surface = SDL_SetVideoMode(widthViewport, heightViewport,
  SCREEN_BPP, videoFlags);
It gets into that last SDL call and throws the exception above. I have tried it in both full screen and resizable window mode, same thing.
I build my app old school, on the command line, as opposed to using Xcode.