Performance: float to int cast and clipping result to range

Posted by durandai on Stack Overflow See other posts from Stack Overflow or by durandai
Published on 2010-05-27T16:05:26Z Indexed on 2010/05/27 19:31 UTC
Read the original article Hit count: 257

Filed under:
|
|

I'm doing some audio processing with float. The result needs to be converted back to PCM samples, and I noticed that the cast from float to int is surprisingly expensive. Whats furthermore frustrating that I need to clip the result to the range of a short (-32768 to 32767). While I would normally instictively assume that this could be assured by simply casting float to short, this fails miserably in Java, since on the bytecode level it results in F2I followed by I2S. So instead of a simple:

int sample = (short) flotVal;

I needed to resort to this ugly sequence:

int sample = (int) floatVal;
if (sample > 32767) {
    sample = 32767;
} else if (sample < -32768) {
    sample = -32768;
}

Is there a faster way to do this?

(about ~6% of the total runtime seems to be spent on casting, while 6% seem to be not that much at first glance, its astounding when I consider that the processing part involves a good chunk of matrix multiplications and IDCT)

  • EDIT The cast/clipping code above is (not surprisingly) in the body of a loop that reads float values from a float[] and puts them into a byte[]. I have a test suite that measures total runtime on several test cases (processing about 200MB of raw audio data). The 6% were concluded from the runtime difference when the cast assignment "int sample = (int) floatVal" was replaced by assigning the loop index to sample.

  • EDIT @leopoldkot: I'm aware of the truncation in Java, as stated in the original question (F2I, I2S bytecode sequence). I only tried the cast to short because I assumed that Java had an F2S bytecode, which it unfortunately does not (comming originally from an 68K assembly background, where a simple "fmove.w FP0, D0" would have done exactly what I wanted).

© Stack Overflow or respective owner

Related posts about java

Related posts about Performance