erratic values in coll-file


    Feb 19 2014 | 10:28 am
    I am reading a csv file using the patch Justin G http://cycling74.com/forums/topic/importing-from-excel-csv-questions
    Now, the data that I get go back two numbers after the decimal point, you can check it
    When loading the file in coll suddenly I get 6 numbers after the decimal point. The textfile does not show me those 6 numbers, just two like on the webpage. So what is wrong with the coll object?
    On a sidenote:the 6th column are very big integers. When loading them into coll they are misinterpreted which I believe is a consequence of the limitation of Max. Is there a solution to deal with these values somehow? Can I divide them by 1000. or so before loading them into coll without having to resort to Excel?

    • Feb 19 2014 | 9:55 pm
      Here is a js example that should do the trick. For big integer you need to run Max 6 in 64 bits to be able to do that, or divide them in javascript before going to coll (see the javascript code).
    • Feb 20 2014 | 11:59 am
      That looks amazing. But still a quirk: while I see the coll gets loaded (and the division by 100 works fine as well), I cannot do basic operation on the coll like deleting a line or reading line by line using a number box. And the first column (containing dates) is gone when reading using a 'next' message box.
      Nevertheless: this js example is very helpful!
      _____
      I think I see what is wrong: there is no index number at the coll so Max tries to interpret the first column as an index number which does not work. How should I add such an index number with the JS code?
    • Feb 22 2014 | 12:04 am
    • Feb 24 2014 | 10:06 am
      Thank you! I have one last question that I cannot get working: I want to reverse the contents of the csv file. At the moment, the dataset is antichronological: the latest date comes at the first row and I want the earliest date to come first. I tried various commands but I do not seem to get it working. Could you give me a hint on how I should make this reversal?
      Here goes the code I worked on. I parse the data as floats so I do not have to do a fromsymbol later on and also removed the header line containing the variable names.
      function import(filename)
      {
      var f = new File(filename);
      if (f.open) {
      var i = 0;
      outlet(0, "clear");
      f.readline();
      while (f.position < f.eof) {
      var str = f.readline();
      var a = str.split(","); // conver strings to array (elements are delimited by a coma)
      // a[5] /= 1000; // uncomment to devide the 6th column by 1000
      var date = Date.parse(a[0]);
      var open = parseFloat(a[1]);
      var high = parseFloat(a[2]);
      var low = parseFloat(a[3]);
      var close = parseFloat(a[4]);
      var volume = parseFloat(a[5]);
      var adjusted_close = parseFloat(a[6]);
      outlet(0, i++, date,open,high,low,close,volume,adjusted_close); // store in the coll
      }
      f.close();
      } else {
      error("couldn't find the file ("+ filename +")\n");
      }
      }
    • Feb 26 2014 | 2:38 pm
      I'm afraid I don't have much time to investigate that for you but basically the date parsing is failing, you didn't create the date object: var date = new Date(a[0]); should do the trick.
      I would just store everything in a javascript array that you could sort after by comparing the dates.
    • Feb 27 2014 | 1:11 pm
      Solved! With some other poeple's help I made it finally. I had to alter your code a bit to get the reverse reading but now it seems to work. Only when loading large csv files (with over 2000 entries for example) it takes a sceond or two to load. I suppose that is inherent to the slower nature of Javascript right? Anyway, thank you for your help. Here goes my code:
      function import(filename)
      {
      var f = new File(filename);
      var csv = [];
      var x = 0;
      if (f.open) {
      var str = f.readline(); //Skips first line.
      while (f.position < f.eof) {
      var str = f.readline();
      csv.push(str);
      }
      f.close();
      } else {
      error("couldn't find the file ("+ filename +")\n");
      }
      //Now you may do whatever you want with each line in the array
      for (var i=(csv.length-1); i>=0; i--) {
      var str = csv.join("\n");
      var a = csv[i].split(","); // convert strings to array (elements are delimited by a coma)
      var date = Date.parse(a[0]);
      var newdate = parseFloat(date);
      var open = parseFloat(a[1]);
      var high = parseFloat(a[2]);
      var low = parseFloat(a[3]);
      var close = parseFloat(a[4]);
      var volume = parseFloat(a[5]);
      var volume1000 = volume /= 1000;
      var adjusted_close = parseFloat(a[6]);
      outlet(0, x++, newdate,open,high,low,close,volume1000,adjusted_close); // store in coll
      }}
    • Mar 01 2014 | 4:43 pm
      I made a version using date sorting. On my 3 years old machine it takes about 100ms to fill the coll with a unsorted csv file of ~2k lines.