Announce: Max Package Downloader

Apr 19, 2013 at 2:47pm

Announce: Max Package Downloader

Hello following a discussion here about a package manager for Max Packages, I decided to code one and to my surprise the basics were easier to do than I thought. For the moment it is still very basic, it will download packages. In the future it could get much fancier with upgrade/delete/disable etc. The idea is to have a tool in the extras menu where you can download new packages. The list of packages is fetched from github meaning it will always be updated with new packages (providing people format their tools as Max 6.1 packages)

Download and more info here: https://github.com/natcl/max_package_downloader

NOTE: I haven’t tested this on Windows yet and it will probably fail due to paths being different.

#67822
Apr 19, 2013 at 3:34pm

More great work nat. I’ll save my cheeky feature requests for a later date !

#243840
Apr 19, 2013 at 4:23pm

This is brilliant ! But I have this message in the Max Window : Could not load class ‘unzip’. Then the download is OK but no unzip occurs.

#243841
Apr 19, 2013 at 4:37pm

Can you tell me what system you are on ?
Can you give me the full path of where you installed the Package ?
Did you restart max ?

Thanks

#243842
Apr 19, 2013 at 4:45pm

Solved !
I had to put the max_package_downloader-master folder into Max6.1/Packages instead of Documents/Max/Packages. Now it works properly.
I run Max 6.1.1 on OSX 10.8.3

Thanks !

#243843
Apr 19, 2013 at 4:54pm

Thanks for the report.

In theory it should also work in the Documents folder, I will try to investigate.

#243844
Apr 19, 2013 at 5:32pm

Lê Quan Ninh:
Can you tell me what other packages you have in your ~/Documents/Max/Packages folder ?
I was able to reproduce on my laptop and it seems that it is conflicting with another package.

Thanks.

#243845
Apr 19, 2013 at 6:46pm

Nothing else than ejies-master downloaded with your Max Package Downloader.
But I probably must say that Max6.1 is on a SSD and my Documents are on a hard drive. The Documents folder is linked with the OS via a symbolic link (http://en.wikipedia.org/wiki/Symbolic_link)

#243846
Apr 19, 2013 at 6:50pm

Ahh yeah Max has a lot of troubles with symbolic links, that might explain the problem…

#243847
Apr 19, 2013 at 8:49pm

Very nice nat ! But, as noted by Lê Quan Ninh, it seems that java-classes in the Document folder aren’t recognised ! It’s the same with eijes’ java-classes, can we call it a BUG ?

#243848
Apr 19, 2013 at 9:02pm

I’m getting the same “Could not load class ‘unzip’” error, and I don’t have any symbolic links. System info:

{
“version” : “Version 6.1.3 (2519669)”,
“platform” : “mac”,
“arch” : “x64″,
“osversion” : “Mac OS X Version 10.7.5 x86_64″,
“samplerate” : 44100,
“iovs” : 64,
“sigvs” : 64,
“scheduler_in_audio_interrupt” : “off”,
“audio_drivername” : “Core Audio”,
“audio_driver_subname” : “”,
“eventinterval” : 2,
“overdrive” : “off”,
“mixerparallel” : “on”,
“mixercrossfade” : 0,
“mixerlatency” : 30.0,
“mixerramptime” : 10.0
}

#243849
Apr 19, 2013 at 9:25pm

mzed: any other packages in your Packages folder ?

#243850
Apr 19, 2013 at 9:27pm

Note:

I just updated the package to support repositories/zips where the actual package is not at the root. I added TapTools and jit.gl.hap.
For those who downloaded it already, please delete it and re-download. It will also now rename the folder according to the Package name.

#243851
Apr 19, 2013 at 9:35pm

nnimar: it seems to work for some and not for others, will try to see what could be happening….
Any particular info about your setup ?

#243852
Apr 19, 2013 at 9:56pm

Yes.
- CNMAT (made by me for me)
- ejies
- another resonators test package by me
- zsa.descriptors

#243853
Apr 19, 2013 at 10:07pm

Mzed:
Could you move (temporarily) those packages out of the folder to test if there is a conflict ?
I suspect there are some problems when multiple packages using java-classes are used.
thanks

#243854
Apr 19, 2013 at 10:45pm

There’s a big bug with the latest source, some packages won’t download, will fix soon.

#243855
Apr 19, 2013 at 10:52pm

Removing the packages fixed the problem.

#243856
Apr 20, 2013 at 12:52am

mzed: thanks for testing, it means there’s some sort of conflict with java when multiple packages are present. Will report this to Cycling if I can reproduce here.

#243857
Apr 20, 2013 at 2:00am

Just pushed an update, should be functional again, please completely remove previous version and re-install.

Now includes:
Max ToolBox
TapTools
jit.gl.hap
ejies

#243858
Apr 20, 2013 at 8:16am

Works perfectly now (Max_Package_Downloader is symbolic link ready !). Thanks !

#243859
Apr 20, 2013 at 5:57pm

Thanks Lê ! Good to know !

#243860
Apr 20, 2013 at 10:16pm

For all the curious and early adopters, it should now work on both Windows and Mac !
Note that at this time it will show packages for all platforms regardless of your os, better filtering will be implemented soon !

#243861
Apr 21, 2013 at 5:48am

I just downloaded and got essentially the same error:
Could not load class ‘package_downloader’
…with all the other packages in place in my downloads folder.

Removing the home-made CNMAT package fixed it. It is the only other package with a java-classes folder, so I suspect that Max has a problem with multiple java folders in the packages folders.

#243862
Apr 21, 2013 at 1:04pm

mzed: is that a big package ? Any chance you could send it to me so I can narrow down the issue ?

thanks !

#243863
Apr 23, 2013 at 2:22am

It seems like it has to do with the very large number of files I have. I’ll email you off the forum.

#243864
Apr 23, 2013 at 4:28am

as far as feature requests for this awesome project. really I just like the idea of it functioning to help you find a remotely available external that you’ve never even heard of before that will help you complete the patch you are working on. Right I mean, that’s what’s so cool about the idea imo. Gratitude Nathanael for an awesome tool!

#243865
Apr 23, 2013 at 4:14pm

First pull request submitted…!

#243866
Apr 23, 2013 at 5:48pm

So, Nat: how much do you want to keep the download code in Max rather than rolling it into the Java (which you only seem to be using for unzipping at the moment). I ask because if you’re thinking of doing more sophisticated dependency/version checking, authentication and so on, I can see that getting untidy in the Max world. (I’m still thinking of Maven, APT/RPM and so on.)

Another thought: have you thought of separating the package management system from some kind of “bootloader” which fetches the package manager itself? I’m not sure what that would achieve, except that it might allow the package manager to slowly get more sophisticated without sacrificing ease of (initial) installation.

Third question: all other things being equal, would you have chosen Python over Java?

#243867
Apr 23, 2013 at 6:26pm

Hi Nick,

My goal is to have as much as possible in Java/Javascript, although for the moment jit.uldl seems to handle the dowloading side of things quite well. For example, I just moved the menu/filtering logic in a JS file instead of using dict objects and such, makes it much cleaner. Emmanuel Jourdan suggested that each package could incorporate a package_info.json file which will help with comparing versions and such. The master json file would then link to the package_info.json file instead of the actual zip file.

I would definitely do it in Python if I could as my Java skills are quite bad. (although it can be a nice occasion to learn) What’s the state of jython as a package ? That’s the main reason I went straight to mxj as bundling jython in a package seemed difficult the last time I checked.

#243868
Apr 23, 2013 at 8:34pm

Right… Maven uses POM (Project Object Module) files which provide version and dependency information; a JSON info file would do something similar.

One thing I was wondering was whether you could actually use the Maven API directly. I had a quick look at Leiningen, which calls into Maven from Clojure; it doesn't look that hard, but it certainly isn't trivial, and would need quite a bit of support code.

Re: Jython. The real problem is that Max packages don't support JAR files, which is a truly major drawback. Joshua suggested that unpacked libraries of class files might work. I may quickly knock up an unpacker in Python or Clojure and see if that works; if so, then Jython could work inside your system. (Alternatively, your system could be extended to deal with JAR files independently, although in that case you really should be making use of Maven.)

Then again, someone might just implement native Python…

[attachment=221928,5477]

Attachments:
  1. Max036.jpg
#243869
Apr 23, 2013 at 8:54pm

Where does that screenshot come from ? Is that using the shell object ?

#243870
Apr 23, 2013 at 10:03pm

Hi Nat,
Thanks very much!
Philippe

#243871
Apr 23, 2013 at 10:55pm

@Nat Nope, it comes from an embryonic native Python object…

#243872
Apr 23, 2013 at 10:57pm

Ohhhhhh…. Let me know if you need testing ! I’ve been dreaming of this for a while now…

#243873
Apr 24, 2013 at 7:59am

@nick …..This thread just got double cool…..woahhh. I feel like we just got such a rad sneak peek.

#243874
Apr 24, 2013 at 8:25am

It’s very early days… Mac-only at present (I need to get my Windows environment updated) and in fact 64-bit only until I sort out the build options. I’m mainly doing it as an exercise in talking to bits of the Max API at the native code level from a nice language with good interop. I’m certainly not planning to abandon the JVM any time soon.

#243875
Apr 24, 2013 at 2:31pm

That’s really interesting Nick…
It uses it’s own python ?

#243876
Apr 24, 2013 at 8:37pm

It’s a Max object containing Python 3.3. (The built-in version on the Mac is something like 2.7.)

#243877
Apr 24, 2013 at 8:44pm

That’s even more interesting if it’s self-contained, much more predictable ! Well as I said, let me know if you need testing, Python is my language of choice and I would probably end up using it extensively.

On a side note, I just added Jeremy Bernstein’s chess object (uci) to the package downloader, you will need to update the Package Downloader first in order to get it as there was a bug in the renaming code. Best is to trash the folder and redownload from github:

https://github.com/natcl/max_package_downloader/archive/master.zip

Thanks !

#243878
Apr 25, 2013 at 8:16am

I’ll let you know how it goes… The multithreading is going to be the tricky part. What I want is a lightweight language wrapper which works well in the scheduler in the way that js doesn’t.

#243879
Apr 25, 2013 at 8:17am

Hey Nat,
A bug report !
-At first i had tried to install the package in Applications/Max 6.1/Packages and it doesn’t work, but maybe that’s intentional
-On open, my max window says “Could not load class ‘package_downloader’”. Seems to work though, it does also say :

MXJ System CLASSPATH:
   /Applications/Max 6.1/Cycling '74/java/lib/jitter.jar
   /Applications/Max 6.1/Cycling '74/java/lib/jode-1.1.2-pre-embedded.jar
   /Applications/Max 6.1/Cycling '74/java/lib/max.jar
   /Applications/Max 6.1/Cycling '74/java/lib/NN.jar
MXJClassloader CLASSPATH:
   /Applications/Max 6.1/Cycling '74/java/classes/
Jitter initialized
Jitter Java support installed

Max is 6.1.2, 32 bits, OSX 10.6.8

-More importantly : i tried to download Toolbox and immediately after jit.gl.hap, and their archive name is master for both projects, so they override each other !… and the only one that remains is the last downloaded one. Maybe you should ask that archives have the name of the package ?.. or not allow overriding by modifiying the name of something that already exists ? or rename an archive on download with the name of the package ? or unzip an archive in the folder once it’s downloaded (maybe that’s what the package_downloader class is supposed to do ?) ? don’t know what is possible…

anyway this looks great so far and potentially as useful as the Max Toolbox ;) i might contribute my student project soon, even if it’s far from as polished as the other projects, just for science you know…

#243880
Apr 25, 2013 at 1:00pm

vichug, did you do a clean install of the package downloader. recently ? these bugs were corrected.
If the class can’t load it won’t work. Do you have other packages installed ? There are some conflicts with java when multiple packages are present.

#243881
Apr 25, 2013 at 1:15pm

i had all my max6.0.8 and max5 externals/custom patches in filepaths preferences, i removed them now, am reinstalling the package downloader and will let you know how it goes. Though i had installed it from the link you provided in the previous post, this morning.

#243882
Apr 25, 2013 at 1:20pm

And everything is fine now. Just, one little thing : the hint when mouse pointer is on “Package Downloader” is : “control a master clock and report time values” :)

#243883
Apr 25, 2013 at 2:02pm

haha nice catch, seems like I copy pasted that comment box from somewhere else :P

#243884
Apr 25, 2013 at 4:43pm

^^
another problem (in fact completely unrelated) appears : it is that i can’t have different max preference paths for Max 6.1 and Max 6.0.8. This could be a problem. Do you know if there is any workaround ?

#243885
Apr 25, 2013 at 5:01pm

Hmm are you saying that when you set the path in Max 6.1 it also affects the path in 6.0.8 ?
I thought the preferences files were independent,

#243886
Apr 25, 2013 at 7:17pm

i thought the same :/

#243887
Apr 25, 2013 at 7:51pm

And if you put the Package downloader in your max 6.1/Packages folder it doesn’t work ?

#243888
Apr 27, 2013 at 10:49am

Just trying out the Downloader. It wasn’t working, so I cleared out my packages folder (except for vizzie & _downloader_master) and restarted max. I get these messages…

MXJ System CLASSPATH:
/Applications/Max 6.1/Cycling ’74/java/lib/jitter.jar
/Applications/Max 6.1/Cycling ’74/java/lib/jode-1.1.2-pre-embedded.jar
/Applications/Max 6.1/Cycling ’74/java/lib/max.jar
MXJClassloader CLASSPATH:
/Applications/Max 6.1/Cycling ’74/java/classes/
Jitter initialized
Jitter Java support installed
jit.uldl: could not set directory to /Users/davidstevens/Documents/Max/Packages
jit.uldl: could not set directory to /Users/davidstevens/Documents/Max/Packages
dictwrap: file not found: /Users/davidstevens/Documents/Max/Packages/max_packages.json
js: menu_filterer.js: Javascript TypeError: keys is null, line 15
js: error calling function read
dictwrap: no dictionary named TapTools, cannot set value
dictwrap: no dictionary named TapTools, cannot set value
Package Downloader: Error: package not available
dictwrap: no dictionary named Max ToolBox, cannot set value
dictwrap: no dictionary named Max ToolBox, cannot set value
dictwrap: no dictionary named ejies, cannot set value
dictwrap: no dictionary named ejies, cannot set value

#243889
Apr 27, 2013 at 3:40pm

Hi David,
seems like your documents folder in not in a typical location, can you give the full path of your packages folder ?
also what version of Max / os are you on ?

#243890
Apr 28, 2013 at 9:27pm

hi Nat,

OSX 10.8.3 (12D78)
MacBookPro Retina, mid 2012; 2.3Ghz Intel Core i7; 8Gb 1600MHz DDR3

Macintosh HD/Applications/Max 6.1/packages

don’t know if it’s relevant, but projects are in

~/Documents/Max 6 Projects

I just deleted, redownloaded and reinstalled Max 6.1.2 and I’m still getting the same issue, here’s the message I got opening the Downloader in the fresh copy of Max…
(I just looked more carefully at the messages, and it seems that the expected path is
Macintosh HD/Applications/Max/Packages
yet the installer creates a folder called Max 6.1, and “packages” is created with lower case P.)
Sorry – just looked again (a case of seeing what I expected to see!), and it seems to be that the packages are meant to be in a folder called Packages, in a folder called Max in my Documents folder. Do i take this to mean that I needed to manually create this folder? Have I totally failed to read the instructions? My assumption was that everything goes inside the new “packages” folder inside the main Max folder.

MXJ System CLASSPATH:
/Applications/Max 6.1/Cycling ’74/java/lib/jitter.jar
/Applications/Max 6.1/Cycling ’74/java/lib/jode-1.1.2-pre-embedded.jar
/Applications/Max 6.1/Cycling ’74/java/lib/max.jar
MXJClassloader CLASSPATH:
/Applications/Max 6.1/Cycling ’74/java/classes/
Jitter initialized
Jitter Java support installed
jit.uldl: could not set directory to /Users/davidstevens/Documents/Max/Packages
jit.uldl: could not set directory to /Users/davidstevens/Documents/Max/Packages
dictwrap: file not found: /Users/davidstevens/Documents/Max/Packages/max_packages.json
js: menu_filterer.js: Javascript TypeError: keys is null, line 15
js: error calling function read
dictwrap: no dictionary named TapTools, cannot set value
dictwrap: no dictionary named TapTools, cannot set value
dictwrap: no dictionary named ejies, cannot set value
dictwrap: no dictionary named ejies, cannot set value

#243891
Apr 28, 2013 at 10:02pm

Hi David,
yes I should specify this in the readme.
you need to have a Max/Packages folder in your documents folder. The downloader uses this folder because it is independent of the main app and won’t get deleted/overwritten by a new version of Max. It also means you can have multiple versions of Max and they will all share the same packages.

#243892
Apr 29, 2013 at 11:30pm

hey Nat,

that did it.

Does that mean that I should put _all of my 3rd party stuff in that path? (including packages that I format for 6.1 from 6.0 versions). And that the packages folder inside the Max6.1 folder isn’t really for anything except packages that come with the main max download? (which is basically only vizzie)

#243893
Apr 30, 2013 at 1:01am

Hi David,
that’s really a personal choice, I personally like having my stuff in the Documents folder, it’s also in sync with how other apps work like the Arduino IDE or Processing. I usually run the keyserver version of Max as well as the normal version (with an iLok) so I find it very useful to have this in a separate directory.

#243894

You must be logged in to reply to this topic.