float precision problem

clankill3r's icon

I want to calculate distance with gps. I moved code from processing (java) to max patches.
Problem is that in processing i have those values for example:
dLat: -1.3315805E-7
dLon: 1.7476995E-7

which is like 0.00000017476995 if i'm correct.

In max the value is zero, and so after a few calculations i get nan (not a number).
What can i do?

Max Patch
Copy patch and select New From Clipboard in Max.

:(

In case someone know's processing, here is the proceesing code.

/*
//pc
testP: gps 5.910325 30. 51.988632 57. 28.372467

//keuken
testP: gps 5.91006 10. 51.988777 19. 29.984245
*/

void setup() {
// println(getDistance(51.988632, 5.910325, 51.988777, 5.91006)); // pc > keuken
// println(getDistance(51.988777, 5.910006, 51.988739, 5.910056));
/*
String[] lines = loadStrings("geo.tsv");

String[] tokens = split(lines[0], "t");
float lon1 = float(tokens[2]);
float lat1 = float(tokens[4]);

for (int i = 1; i < 2; i++) { //lines.length-1

tokens = split(lines[i+1], "t");
float lon2 = float(tokens[2]);
float lat2 = float(tokens[4]);

//println(lat2+"t"+lon2);
float d = getDistance(lon1, lat1, lon2, lat2);
println(d*1000);
}
*/
float d = getDistance(5.910161, 51.988705, 5.910171, 51.988697);
println(d*1000);

}

float getDistance(float lon1, float lat1, float lon2, float lat2) {
int R = 6371; // Radius of the earth in km
float dLat = radians(lat2-lat1);
float dLon = radians(lon2-lon1);

println("dLat: "+dLat+"ndLon: "+dLon);

float a = sin(dLat/2) * sin(dLat/2) + cos(radians(lat1)) * cos(radians(lat2)) * sin(dLon/2) * sin(dLon/2);
float c = 2 * atan2(sqrt(a), sqrt(1-a));
float d = R * c; // Distance in km

return d;
}

brendan mccloskey's icon

Hi
I can't find the corresponding expression in your Processing code (maybe the last 5 lines?); but inside your getDistance subpatch, you are trying to find the sqrt of a negative number (on the right, above [atan2]); this generates a complex number, unrepresentable in Max. That's where the "nan" is generated. Sorry if you already knew this.

Brendan

brendan mccloskey's icon

.....to get it to "work", I inverted the result AFTER the sqrt calculation; probably messes up the result you're expecting but here it is anyway:

Max Patch
Copy patch and select New From Clipboard in Max.

Brendan

Peter Castine's icon

Max works with single-precision floating point values, which gives you approximately 6-digit precision in the mantissa and can handle exponents (base 10) of about +/-38.

This is all considerably less powerful than the double-precision floats used in Java, but should not be a limitation with the values you say you are using.

Square-roots of negative numbers will NaN no matter what representation you have. From a quick look, it seems that what you have implemented in your max patch is float c = 2 * atan2(sqrt(a), sqrt(0-a)); rather than float c = 2 * atan2(sqrt(a), sqrt(1-a));.

The problem is further exacerbated by your using send/receive pairs and, apparently, relying on left-to-right order. When you s/r, all bets on execution order are off.

I've also loadbanged the radius of the earth into your patch.

Max Patch
Copy patch and select New From Clipboard in Max.

The following is only the [p GetDistance] part of your patch, you'll have to manually copy/paste it into your top-level patch.

clankill3r's icon

Thanks peter,

i allready had it fixed with javascript which is much easier to read for such things (didn't know before that javascript was possible).

But good to know it is possible.