Doubles ….10 decimal resolution. Workaround?
I have a situation where I need to have an ==~ object track a phasor~ with up to 10 decimal places. Dissapointingly I can only analyse with a resolution of 6 decimals. Is there a way round this limitation?
PS At least ==~ it displays only 6 decimals. When I input a 10 decimal float number it seems to work, bizarre!
from my understanding this almost sounds like a bug of the comparison object, but
maybe i am missing something.
any signal object in max 6 should normally take 64 bit signals in any inlets.
Your arguments are going to be in floating-point (32 bit), but processing is at 64-bits. What exactly are you trying to do? There may be another way of implementing this that works better.
Peter, Roman, many thanks guys, you are right. I can see It only displays 32bit but I get a 64 bit result when I set folatnum…..phewww!
Sorry didn’t include example. Here it is. It still doesn’t work. I am perplexed.
It seems like the maximum resolution achievable is 1/64
thanks for your help any way, P
----------begin_max5_patcher---------- 710.3ocwVkraaCCD8r7WAAuzKtFhTawEHG5GPu1KEEAzhicXfDo.EUZVP72d IoVriiiricfyEQvQCm4MO8lg54IA3EpGfZL5Gn+fBBddRPf2jyPP29.bI6g7 BVs2M7xBkroDOs8UKURSs3Iv8JBcVXmYqGpFSAX7mgtwJGxEkrhpBVdaZI8G o0eyiUPKZbIhYvSQ3EL4JL5uc9UwL42JjqtQC4lVWoww1LiHySbKTRpeIbV3 vgDbO1UKt66IaCcIqzmN7O0BVAdHEZqcCnuAjrEEdO1pxDx9Bi3r8xjItGSO R9CdhyxWi2KQE990XbXnuFiHtk3j9m6qDiw6CrzS.r4pxRPZ5imAdvCG7uYE M.RB.G3nkJMxnEqVAZjnFwxMMrhhGQHKhCCmmkljRSPKZLneUWgzpFIuFoVt DYT8tjgXRNhqfZ42L8AqOqEBIjaOkYaszQI7FgOooQshkVMybO6FEueBM5vZ lyVWXCvBPu9jZrdSuSsXkzgs9tnghx9BOwLDvkFhOEyFgqhtxSNzVE3bxnsW wGlqDRau0818tRXCTJUbXmwEmm5UB+yBo2Hdu950C5tODcSN.cO1LJRL0Sao dR7pjQ4vOpd6Sjb3vJMiCqsJBz7KF6DE1xGwyxRNH6D9AYmnSfcNia4NiKxZ astxeaVR3XjPb1W70Xui7QyLVsCMI8x0XE4UKjj4tkT5nrV7WWiU0srZk1M4 gb43lzrY19IBwyMYwiwMzyfZ7wyeO8N+JoGSN6ulupUM5795n6mxPaPEGpMB IyHTxs8wNUDQFb5VAmCxs0zkBdkxd2RGFnIQthm1MagjMraShD0tFCOIDt2O 0GaAPOhBvo91xo2V.edvwMd3f3gd4fS7w78M7hgmcR0H5sKDdhNF7bpxm1tS VU08fttKldnXmacmR61lN0uUHa25iHVC2K58OYhKZuL4+.nOLsJ -----------end_max5_patcher-----------
You’re up against the limits of 32 bit floating point here, I think.
the only thing you could do to give this argument to a signal object is to use 2 32 bit numbers and multiply them with each other _after converting them to signal using [sig~]
@ Chris Yes I think that is it. The 32 bit limit. I looked at Claytons Hires objects but there isn’t a 64 bit ==~
@ Roman, Ok I’ll try that.
The weird thing is that I can get a 10 decimal reading from number~ out of its float output, just that pesky ==~ is holding this up :-(
thanks so much everyone for giving me a hand with this.
Why not just use a range? >=~ and < =~ and multiply those to together (for logical and). If this is the usage case you're looking at, it takes care of it perfectly. The neighborhood in this example could be quite wide since you're already reducing the bit depth, so it's not like you'll have a value that's really close anyways.
It’s generally recommended in the programming literature that you avoid testing floating point numbers for equality, and given that there’s mixed types involved, that seems to go double. To read more:
Excellent info Peter, many thanks, great information about the unreliability of testing floats for equality. The result I get are explained by the linked article you include. Enormous thanks to everyone. I will test the range approach.
However I must add that the testing equality works very accurately if kept within the 32bit limit and not if taken outside this. 64 bit resolution would no doubt solve my problem using the existing equality testing. I played with range but unfortunately it doesn’t solve my problem. A Shame as after many years of working with MSP (which I love) I have to migrate to an environment that can handle the needed precision.
Are you trying to do something other than what you posted? I don’t really see what the problem is, but maybe there’s something more?
My question when people go into extreme precision is "Do you really need that precision, or is just the way you want to be able to solve the problem?" In my personal experience, extreme precision and equality are rarely needed together (I can’t think of a particular signal-processing application where they are).
If you have gen~, you can store doubles inside codebox or expr, as they are evaluated as doubles. I prefer the range solution since it gives me further flexibility, but YMMV.
^^^ what peter said.
"A Shame as after many years of working with MSP (which I love) I have to migrate to an environment that can handle the needed precision"
…and use gen~. then you don’t have to migrate.
I will start investigating Gen today.Just upgraded to 6 so its all fairly new. Thanks all!!
Forums > MaxMSP