Precision of codebox variables
I have a codebox variable that increments by a small amount (in my test case 0.000034) every frame to create a growing ramp of arbitrary size. I've discovered that it seems to stall out at 1024.0. Very conspicuous number. Do variables in gen~ not have the precision to represent a number greater than 1023 to greater than 4 decimal places?
Strange. gen~ is supposed to work with 64-bit doubles, which I understand should have around 15-16 digits of precision. 1024.000034 should require only 9 digits of precision. Maybe post a reduced example showing the problem?
That's what I thought too. But every clue I glean as I try to troubleshoot this leads me to believe it's a precision issue. For example, it doesn't happen if the ramp increment is one decimal place larger (say, .000122 rather than .000034).
the image you post is useless. we cannot tell anything from it.
how are you measuring the precision? where do you see it? are you measuring blog? with what? what is ramp? or are you expecting a sampleaccurate measurement to 'do something else' in your code?
at the moment it sounds like you are using a number box externally - which cannot represent this data.
gen~ is crappy and a pain for visualisation of your data. so, more info would be needed.
Yes debugging in gen~ can suck; blog() is a highly useful function Graham of C74 threw together for me when I brought that point up a while back. It logs values into a buffer~ so they can be easily read later. It's here
https://cycling74.com/forums/gen-print-to-consoledebugging
if you're interested, but I wasn't really expecting anyone to diagnose my whole patch, merely illustrating my situation a little better since I was asked. My question is more a general one, about anyone having noticed less-than-expected precision in gen~.
It looks like I've actually solved my problem though, by dividing the ramp variable up into two, one for the integer portion and one for the decimal. They are then added together every sample. It still might truncate a number like 1024.000034 for a sample, but at least it won't stall out the ramp as the decimal variable increments independently.
Unless someone's got a better idea, I'm going to chalk this one up to DSP in Max in general. I guess I can't come up with an example off the top of my head, but I feel like I've had reason to distrust the accuracy of decimal places beyond the 4th before.
Of course there's absolutely no precision issue with incrementing above 1024 with steps of 0.000034:
However from looking at the image you posted above (please, a minimal patcher would be much much better!) you might want to rethink how you detect that edge transition. A really simple way would be to use "(change(ramp % 1024)) < 0", for example.
About the only places where precision is sometimes reduced are in the display of numbers in some user interfaces (for legibility), and in buffer~ (which uses 32 bit floats for some legacy reasons, whereas the rest of the audio signal objects use 64 bit floats).
Graham