jmpelletier

Hi all, this is something I meant to report a long time ago, but I keep on forgetting.

If you send a char matrix in the right inlet, its values get normalized from 0-255 to 0-1. However, this isn't done quite right because 1 * 255 = 0 and 2 * 255 = 1, and so on, which is terribly annoying. 1 * 255 ought to return 1.

buggy-multiplication-with-jit-op

On Nov 6, 2008, at 4:58 PM, Jean-Marc Pelletier wrote:

> Hi all, this is something I meant to report a long time ago, but I 

> If you send a char matrix in the right inlet, its values get 

> normalized from 0-255 to 0-1. However, this isn't done quite right 

> because 1 * 255 = 0 and 2 * 255 = 1, and so on, which is terribly 

Sorry, Jean Marc. This is technically not a bug. You might be 

misunderstanding what is happening. The input is not really 

"normalized" 0 - 1. It is simply considered an eight bit fractional 

fixed point number. There is no special handling for considering 255 

or be unity as opposed to *256*, which would be the *real* unity value 

in this model, but is not representable in 8 bits. This typically to 

be expected from 8 bit fixed point math--i.e.: result = (a*b)>>8L;

Granted this is somewhat inconsistent with various other areas, but 

operates this way for performance and simplicity's sake. If you want, 

you want greater accuracy, I would suggest you use floating point 

matrices or find an alternate solution to your problem.

>
> Hi all, this is something I meant to report a long time ago, but I  
> keep on forgetting.
>
> If you send a char matrix in the right inlet, its values get  
> normalized from 0-255 to 0-1. However, this isn't done quite right  
> because 1 * 255 = 0 and 2 * 255 = 1, and so on, which is terribly  
> annoying. 1 * 255 ought to return 1.

Sorry,  Jean Marc. This is technically not a bug. You might be  
misunderstanding what is happening. The input is not really  
"normalized" 0 - 1. It is simply considered an eight bit fractional  
fixed point number. There is no special handling for considering 255  
or be unity as opposed to *256*, which would be the *real* unity value  
in this model, but is not representable in 8 bits. This typically to  
be expected from 8 bit fixed point math--i.e.: result = (a*b)>>8L;

Granted this is somewhat inconsistent with various other areas, but  
operates this way for performance and simplicity's sake. If you want,  
you want greater accuracy, I would suggest you use floating point  
matrices or find an alternate solution to your problem.

Quote: jkc wrote on Fri, 07 November 2008 10:14

----------------------------------------------------

> Granted this is somewhat inconsistent with various other areas, but 

> operates this way for performance and simplicity's sake. If you want, 

> you want greater accuracy, I would suggest you use floating point 

> matrices or find an alternate solution to your problem.

The main problem this causes is with binary masks. It's not so much a question of precision, but rather I had assumed that multiplication would have been the most efficient way of masking a char matrix. As it turns out, this is not the case as "

However, the jit.op documentation clearly states: "IMPORTANT : If the incoming matrix data has typechar, then integers in the range 0-255 are considered to be floating point values in the range 0.-1." As per your explanation, this is not the case, and I think the documentation should reflect that.

Quote: jkc wrote on Fri, 07 November 2008 10:14
----------------------------------------------------

> Granted this is somewhat inconsistent with various other areas, but  
> operates this way for performance and simplicity's sake. If you want,  
> you want greater accuracy, I would suggest you use floating point  
> matrices or find an alternate solution to your problem.
>

On Nov 6, 2008, at 6:01 PM, Jean-Marc Pelletier wrote:

> The main problem this causes is with binary masks. It's not so much 

> a question of precision, but rather I had assumed that 

> multiplication would have been the most efficient way of masking a 

> char matrix. As it turns out, this is not the case as "

> as I've just now tested, is about as fast.

If speed is an issue for a binary mask, one could also use bitwise and 

(&), since there is no possibility of branching, and a mask of 255 

> However, the jit.op documentation clearly states: "IMPORTANT : If 

> the incoming matrix data has typechar, then integers in the range 

> 0-255 are considered to be floating point values in the range 0.-1." 

> As per your explanation, this is not the case, and I think the 

Noted. In effect they can be considered as such, just not with 

anything greater than 1 bit accuracy within any given computation. 

That's typical with fixed point calculation.

> The main problem this causes is with binary masks. It's not so much  
> a question of precision, but rather I had assumed that  
> multiplication would have been the most efficient way of masking a  
> char matrix. As it turns out, this is not the case as "
> as I've just now tested, is about as fast.

If speed is an issue for a binary mask, one could also use bitwise and  
(&), since there is no possibility of branching, and a mask of 255  
will deliver the exact incoming bits.

> However, the jit.op documentation clearly states: "IMPORTANT : If  
> the incoming matrix data has typechar, then integers in the range  
> 0-255 are considered to be floating point values in the range 0.-1."  
> As per your explanation, this is not the case, and I think the  
> documentation should reflect that.

Noted. In effect they can be considered as such, just not with  
anything greater than 1 bit accuracy within any given computation.  
That's typical with fixed point calculation.

Buggy multiplication with jit.op