geddonn

I have been trying to use jit.gen to implement some simple maths on jitter matrices

I have come across two operations that do not function as expected.

I have been trying to use jit.gen to implement some simple maths on jitter matrices
I have come across two operations that do not function as expected.

jit-gen-simple-maths-operations-not-working

The modulo gen patcher does seem to be a bug. If you change the matrix's type to float32 it works, but not in char/fixed mode.

The second patcher with a multiply is not a bug, but an artifact of how fixed point processing works. When multiplying in char mode, you can't think of the char value of 1 as being unity. It's actually 1/255 in fixed point representation. If you look at it this way, you have a matrix with cells (1, 2) and (0, 4). Obviously 0*4 is going to be 0. Counterintuitively so is (1, 2). Here's why. You're multiplying 1/255*2/255 -> 2/(255*255) which in the char format gets rounded down to 0 since it's much much smaller than 1/255. If your cell was something like (50, 128), the result you would get is 25 in char format since 50/255*128/255 -> 25/255 approximately.

The modulo gen patcher does seem to be a bug.  If you change the matrix's type to float32 it works, but not in char/fixed mode.

The second patcher with a multiply is not a bug, but an artifact of how fixed point processing works.  When multiplying in char mode, you can't think of the char value of 1 as being unity.  It's actually 1/255 in fixed point representation.  If you look at it this way, you have a matrix with cells (1, 2) and (0, 4).  Obviously 0*4 is going to be 0.  Counterintuitively so is (1, 2).  Here's why.  You're multiplying 1/255*2/255 -> 2/(255*255) which in the char format gets rounded down to 0 since it's much much smaller than 1/255.  If your cell was something like (50, 128), the result you would get is 25 in char format since 50/255*128/255 -> 25/255 approximately.


Jit.gen simple maths operations not working