erratic values in coll-file


    Feb 19 2014 | 10:28 am
    I am reading a csv file using the patch Justin G https://cycling74.com/forums/importing-from-excel-csv-questions
    Now, the data that I get go back two numbers after the decimal point, you can check it
    When loading the file in coll suddenly I get 6 numbers after the decimal point. The textfile does not show me those 6 numbers, just two like on the webpage. So what is wrong with the coll object?
    On a sidenote:the 6th column are very big integers. When loading them into coll they are misinterpreted which I believe is a consequence of the limitation of Max. Is there a solution to deal with these values somehow? Can I divide them by 1000. or so before loading them into coll without having to resort to Excel?

    • Feb 19 2014 | 9:55 pm
      Here is a js example that should do the trick. For big integer you need to run Max 6 in 64 bits to be able to do that, or divide them in javascript before going to coll (see the javascript code).
    • Feb 20 2014 | 11:59 am
      That looks amazing. But still a quirk: while I see the coll gets loaded (and the division by 100 works fine as well), I cannot do basic operation on the coll like deleting a line or reading line by line using a number box. And the first column (containing dates) is gone when reading using a 'next' message box.
      Nevertheless: this js example is very helpful! _____ I think I see what is wrong: there is no index number at the coll so Max tries to interpret the first column as an index number which does not work. How should I add such an index number with the JS code?
    • Feb 22 2014 | 12:04 am
    • Feb 24 2014 | 10:06 am
      Thank you! I have one last question that I cannot get working: I want to reverse the contents of the csv file. At the moment, the dataset is antichronological: the latest date comes at the first row and I want the earliest date to come first. I tried various commands but I do not seem to get it working. Could you give me a hint on how I should make this reversal?
      Here goes the code I worked on. I parse the data as floats so I do not have to do a fromsymbol later on and also removed the header line containing the variable names.
      function import(filename) { var f = new File(filename);
      if (f.open) { var i = 0; outlet(0, "clear"); f.readline();
      while (f.position < f.eof) { var str = f.readline(); var a = str.split(","); // conver strings to array (elements are delimited by a coma) // a[5] /= 1000; // uncomment to devide the 6th column by 1000
      var date = Date.parse(a[0]); var open = parseFloat(a[1]); var high = parseFloat(a[2]); var low = parseFloat(a[3]); var close = parseFloat(a[4]); var volume = parseFloat(a[5]); var adjusted_close = parseFloat(a[6]); outlet(0, i++, date,open,high,low,close,volume,adjusted_close); // store in the coll } f.close(); } else { error("couldn't find the file ("+ filename +")\n"); } }
    • Feb 26 2014 | 2:38 pm
      I'm afraid I don't have much time to investigate that for you but basically the date parsing is failing, you didn't create the date object: var date = new Date(a[0]); should do the trick.
      I would just store everything in a javascript array that you could sort after by comparing the dates.
    • Feb 27 2014 | 1:11 pm
      Solved! With some other poeple's help I made it finally. I had to alter your code a bit to get the reverse reading but now it seems to work. Only when loading large csv files (with over 2000 entries for example) it takes a sceond or two to load. I suppose that is inherent to the slower nature of Javascript right? Anyway, thank you for your help. Here goes my code:function import(filename) { var f = new File(filename); var csv = []; var x = 0; if (f.open) { var str = f.readline(); //Skips first line. while (f.position < f.eof) { var str = f.readline(); csv.push(str); } f.close(); } else { error("couldn't find the file ("+ filename +")\n"); } //Now you may do whatever you want with each line in the array for (var i=(csv.length-1); i>=0; i--) { var str = csv.join("\n"); var a = csv[i].split(","); // convert strings to array (elements are delimited by a coma) var date = Date.parse(a[0]); var newdate = parseFloat(date); var open = parseFloat(a[1]); var high = parseFloat(a[2]); var low = parseFloat(a[3]); var close = parseFloat(a[4]); var volume = parseFloat(a[5]); var volume1000 = volume /= 1000; var adjusted_close = parseFloat(a[6]);
      outlet(0, x++, newdate,open,high,low,close,volume1000,adjusted_close); // store in coll
      }}
    • Mar 01 2014 | 4:43 pm
      I made a version using date sorting. On my 3 years old machine it takes about 100ms to fill the coll with a unsorted csv file of ~2k lines.