seejayjames

I want to store some lists of high-precision floats into [text], to save as comma-separated or tab-separated text files. When I put them directly in, I get 0.001 precision; after a [tosymbol] I get 0.0001. Both read fine (as numbers) into my spreadsheet, so that's good.

I'd like at least 8 decimal places, any thoughts on how to get these working inside [text]? Or if someone knows an easy way to save a jit.matrix as .csv or similar, that would be fine too.

floating-point-number-precision-in-text

Any thoughts on this? I found that [text] is willing to accept ints of many places, so as a workaround I could take all my floats and multiply by a million or whatever power of 10 is needed. Then I could have my spreadsheet re-calculate the right values. But this seems kind of lame...why does [text] round the values like this? [coll] also does some rounding, looks like six decimal places is the most it'll store.

Am I missing something in the Preferences or the float attributes? I set the number of decimal places to 8...I see them all in the box, but it's rounded as it goes into [text] or [coll]. What's up?

Try setting the precision specifically and switching it to a symbol. Replacing the [tosymbol] with [sprintf "%.6f"] works fine here.

Fantastic! Forgot about the sprintf with settable decimal places. Still seems a little strange to have to convert to a symbol to save the number of places for [text], but hey, it works.

Might be nice to have a mention of this float-rounding behavior somewhere in the documentation for [text] and [coll], as well as wherever else it comes up, or maybe a general mention somewhere appropriate. Seeing a high-precision float at one place made me assume it would stay that way elsewhere, so it was a bit surprising that it got automagically rounded. No worries now though.

Fantastic! Forgot about the sprintf with settable decimal places. Still seems a little strange to have to convert to a symbol to save the number of places for [text], but hey, it works. 

Floating-point number precision in [text]