I posted a question on gamedev about how to play nsf files (NES console music) in FMOD. It didn't get any results, but since then I made some progress. I decided that the easiest method was just to compile an existing player into a dll and then call it from C# to populate my buffer. The problem now is getting it to sound right, and making sure all my paremeters are correct.

Here are the facts so far:

1. The nsf dll is dealing with shorts, so the data is PCM16.
2. The sample nsf I'm using has a playback rate of 60 Hz.
3. Just for playing around now, I'm using a frequency of 48000.
4. Based on 2 and 3, the dll calculates a necessary buffer size of 48000 / 60hz = 800. This means it will render 800 shorts worth of buffer for every simulated NES frame.

I've so far got my C# code to play the nsf, at the correct pitch and tempo, but it's very grainy / fuzzy, which I'm attributing to the fact that the FMOD read callback is giving a data length of 1600, whereas I should be expecting 800. I've tried playing around with all the numbers and it either crashes, or the music changes pitch, tempo, or both.

Here's some of my C# code:

uint channels = 1, frequency = 48000;

FMOD.MODE mode = (FMOD.MODE.DEFAULT | FMOD.MODE.OPENUSER | FMOD.MODE.LOOP_NORMAL);

FMOD.Sound sound = new FMOD.Sound();
FMOD.CREATESOUNDEXINFO ex = new FMOD.CREATESOUNDEXINFO();
ex.cbsize = Marshal.SizeOf(ex);
ex.fileoffset = 0;
ex.format = FMOD.SOUND_FORMAT.PCM16;

// does this even matter? It doesn't change my results as long as it's long enough for one update
ex.length = frequency; 

ex.numchannels = (int)channels;
ex.defaultfrequency = (int)frequency;
ex.pcmreadcallback = pcmreadcallback;
ex.dlsname = null;

// eventually I will calculate this with frequency / nsf hz, but I'm just testing for now
ex.decodebuffersize = 800;

// from the dll
load_nsf_file("file.nsf", 8, (int)frequency); // 8 is the track number to play

var result = system.createSound(
        (string)null,
        (mode | FMOD.MODE.CREATESTREAM),
        ref ex,
        ref sound);

channel = new FMOD.Channel();
result = system.playSound(FMOD.CHANNELINDEX.FREE, sound, false, ref channel);

private FMOD.RESULT PCMREADCALLBACK(IntPtr soundraw, IntPtr data, uint datalen)
{
    // from the dll
    process_buffer(data, (int)800); // if I use datalen, it usually crashes (I can't get datalen to = 800 safely)

    return FMOD.RESULT.OK;
}

So here are some of my questions:

1. What is the relationship between exinfo.decodebuffersize, frequency, and the datalen parameter of the read callback? With this code sample, it's coming in as 3200. I don't know where that factor of 4 between it and the decodebuffersize comes from.
2. Is datalen in the callback referring to number of bytes, or shorts? The process_buffer function takes a short array and its length. I would expect fmod is talking about shorts as well because I told it PCM16.
3. Maybe my playback quality is bad for some totally different reason. If so I have no idea where to begin solving that. Any ideas there?