jit.op * with char ?

Mar 18, 2009 at 6:02pm

jit.op * with char ?


i’ve found a pretty strange behavior in jit.op
when i tried to multiply a 1 plane char matrix by 1, it outputs only zeros, whereas the same works fine on float32 matrix

here is a little patch :

– Pasted Max Patch, click to expand. –

is it normal ?
can you reproduce this ?



Mar 18, 2009 at 6:38pm

remember char data goes from 0 to 255, so to get the results you expect your operand should be 255, not 1.

however this still doesn’t quite give the same behavior as multiplying by 1 (they are all 1 less than the original values). i assume the char data is converted to floating point and back, and the decimal points are truncated.

if you intend to do decimal based math operations on data, probably best to used either type long or float32 and convert them to char after the jit.op (which is as simple as sending the output of jit.op into a char matrix of same dim and planecount).


You must be logged in to reply to this topic.