erratic values in coll-file

Feb 19, 2014 at 2:28am

erratic values in coll-file

I am reading a csv file using the patch Justin G http://cycling74.com/forums/topic/importing-from-excel-csv-questions

Now, the data that I get go back two numbers after the decimal point, you can check it

When loading the file in coll suddenly I get 6 numbers after the decimal point. The textfile does not show me those 6 numbers, just two like on the webpage. So what is wrong with the coll object?

On a sidenote:the 6th column are very big integers. When loading them into coll they are misinterpreted which I believe is a consequence of the limitation of Max. Is there a solution to deal with these values somehow? Can I divide them by 1000. or so before loading them into coll without having to resort to Excel?

Attachments:
  1. table-1.txt
#281536
Feb 19, 2014 at 1:55pm

Here is a js example that should do the trick. For big integer you need to run Max 6 in 64 bits to be able to do that, or divide them in javascript before going to coll (see the javascript code).

Attachments:
  1. csv-to-coll.zip
#281597
Feb 20, 2014 at 3:59am

That looks amazing. But still a quirk: while I see the coll gets loaded (and the division by 100 works fine as well), I cannot do basic operation on the coll like deleting a line or reading line by line using a number box. And the first column (containing dates) is gone when reading using a ‘next’ message box.

Nevertheless: this js example is very helpful!
_____
I think I see what is wrong: there is no index number at the coll so Max tries to interpret the first column as an index number which does not work. How should I add such an index number with the JS code?

#281643
Feb 21, 2014 at 4:04pm

Here you go ;-)

#281753
Feb 24, 2014 at 2:06am

Thank you! I have one last question that I cannot get working: I want to reverse the contents of the csv file. At the moment, the dataset is antichronological: the latest date comes at the first row and I want the earliest date to come first. I tried various commands but I do not seem to get it working. Could you give me a hint on how I should make this reversal?

Here goes the code I worked on. I parse the data as floats so I do not have to do a fromsymbol later on and also removed the header line containing the variable names.

function import(filename)
{
var f = new File(filename);

if (f.open) {
var i = 0;
outlet(0, "clear");
f.readline();

while (f.position < f.eof) {
var str = f.readline();
var a = str.split(","); // conver strings to array (elements are delimited by a coma)
// a[5] /= 1000; // uncomment to devide the 6th column by 1000

var date = Date.parse(a[0]);
var open = parseFloat(a[1]);
var high = parseFloat(a[2]);
var low = parseFloat(a[3]);
var close = parseFloat(a[4]);
var volume = parseFloat(a[5]);
var adjusted_close = parseFloat(a[6]);
outlet(0, i++, date,open,high,low,close,volume,adjusted_close); // store in the coll
}
f.close();
} else {
error("couldn't find the file ("+ filename +")\n");
}
}

#281920
Feb 26, 2014 at 6:38am

I’m afraid I don’t have much time to investigate that for you but basically the date parsing is failing, you didn’t create the date object: var date = new Date(a[0]); should do the trick.

I would just store everything in a javascript array that you could sort after by comparing the dates.

#282138
Feb 27, 2014 at 5:11am

Solved! With some other poeple’s help I made it finally. I had to alter your code a bit to get the reverse reading but now it seems to work. Only when loading large csv files (with over 2000 entries for example) it takes a sceond or two to load. I suppose that is inherent to the slower nature of Javascript right? Anyway, thank you for your help. Here goes my code:
function import(filename)
{
var f = new File(filename);
var csv = [];
var x = 0;
if (f.open) {
var str = f.readline(); //Skips first line.
while (f.position < f.eof) {
var str = f.readline();
csv.push(str);
}
f.close();
} else {
error("couldn't find the file ("+ filename +")\n");
}
//Now you may do whatever you want with each line in the array
for (var i=(csv.length-1); i>=0; i--) {
var str = csv.join("\n");
var a = csv[i].split(","); // convert strings to array (elements are delimited by a coma)
var date = Date.parse(a[0]);
var newdate = parseFloat(date);
var open = parseFloat(a[1]);
var high = parseFloat(a[2]);
var low = parseFloat(a[3]);
var close = parseFloat(a[4]);
var volume = parseFloat(a[5]);
var volume1000 = volume /= 1000;
var adjusted_close = parseFloat(a[6]);

outlet(0, x++, newdate,open,high,low,close,volume1000,adjusted_close); // store in coll

}}

#282307
Mar 1, 2014 at 8:43am

I made a version using date sorting. On my 3 years old machine it takes about 100ms to fill the coll with a unsorted csv file of ~2k lines.

#282551

You must be logged in to reply to this topic.