ARToolkit for Max/MSP
New Project on github.
https://github.com/cap10subtext/jit.artkmulti
I'm just totally swallowing my pride on this one and posting this obscene mess of code. Forgive the cliche but I'm NOT a classically trained programmer, it's been my one of my pet projects for learning C which of course means lots of trial and error and beginners mistakes.
In particular I'm having issue with deleting and creating a new object and changing the camera dims during run time.
One thing that's missing from this code post (though it should be obvious what's required), you need a folder called artk.support in the Max5/ folder (can't be in the cycling 74 folder at the moment, fopen rejects any spaces in the path to camera.dat or multi/objectdata2
objectdata2 is being used with 4 obejcts, currently hard coded for 4 outlets.
The help patch is particularly scary but it's just an early version I picked out at this late hour so there was something to test with.
I know it's rough but I've heard a lot of interest in this recently. Hopefully someone will find it useful.
Currently the compile is Intel Mac only.
Hi !
Am I the only one to get this:
Jitter 1.7.0 installed
jit.artkmulti: unable to load object bundle executable
2011-03-06 10:46:56.953 MaxMSP[4444:20b] Error loading /Applications/Max5/artk.support/jit.artkmulti.mxo/Contents/MacOS/jit.artkmulti: dlopen(/Applications/Max5/artk.support/jit.artkmulti.mxo/Contents/MacOS/jit.artkmulti, 262): no suitable image found.
Did find:
/Applications/Max5/artk.support/jit.artkmulti.mxo/Contents/MacOS/jit.artkmulti: mach-o, but wrong architecture
I'm on 10.5
Anyway, this going to be very nice to artoolkit for MAx ! Just have to make it work :)
Cheers.
Ad
Will check compiler settings. What type of machine are you on?
great work!
but I have a strange behavior in my patch. the cube (or whatever it's rendered) is not erased when loosing the track of the pattern. It's supposed to be like that? A work around would be to send a not draw (or something similar) message to jit.gl.sketch or jit.gl.model when the pattern is lost, but for doing so some index number (0 or 1) should be sent together with the list with coordinates. Or we could do something else to get the object erased?
Mac book pro 10.5.8
Xcode 3.1
max 5.1.7
Thank you in advance.
Ad-
Ad.: I've tweaked something in the compiler that may have been causing the problem. Please feel free to try this one.
sandroid: the first element of the list is now 1 or 0 (visible or not) so you can connect your gl object to an enable $1 message to add or remove your model from the scene.
New Downloads are here:
https://github.com/downloads/cap10subtext/jit.artkmulti/ARTookitMax.zip
Damn, I just typed out this big, detailed post and my browser ate it....
Take 2... (greatly summarized due to frustration of having to retype it all):
Here are the main issues I'm contending with:
1. Including external files: fopen won't accept (even properly escaped) spaces. Suggestions? Where to keep file, workarounds to fopen?
2. Can you realloc after jit_object_alloc? I've checked that the sizeof everything I want to realloc into x:
x = (t_jit_artoolkit *)realloc(x, sizeof(x) + (sizeof(ObjectData_T) * *objnum));
seems to be within the 16384 bytes limit so I'm a little confused (not very savvy with memory management in C).
3. Reinit based on change of incoming dims: obviously Jitter handles this a little differently than ARToolkit. My attempt to do this:
if ((in_minfo.dim[0]!= xsize)||(in_minfo.dim[1]!= ysize)) {
arVideoCapStop();
arVideoClose();
argCleanup();
xsize = in_minfo.dim[0];
ysize = in_minfo.dim[1];
post("input dimensions changed");
jit_artkmulti_param_load();
}
Sends ARToolkit into a death spiral. So clearly I'm not re-initializing ARToolkit properly. Perhaps someone familiar with ARToolkit will stumble across this post and point out my silly mistake.
thanks for the update!
despite the problems you just related, I's really cool to have artoolkit ported to jitter.
I will use a lot with my students this year.
thank you
New Project page up (easier download link for those scared of github) ;)
Hello
Apparently you made the exrernal with xcode 4, 10.6 leopard…
that might be the reason it doesn't work on 10.5.
Cheers.
Ad.
Ad.: Xcode 3.2.5 actually, with the base system being set to 10.5. I tested an earlier version on my other computer which is 10.5. Will do again and see if I can find the problem. Just been a bit busy lately.
oups— you're right… I reinstal max, but nothing change…
Can anyone reproduce the bug ?
Just dont get it…
mac book pro 10.5.8.
Max 5.1.7
Java 1.5
Ad-
Can you replicate what I've just done and post a screen shot with object, Max printout, and machine info? Maybe it'll give me a clue...
Ad. I've posted a new download but I'm not sure if it'll help. Be absolutely sure you've trashed all the old versions and then put artk.support at /Max5.
(It's at the same link)
http://www.aranarproductions.com/artk/
It's compiled as 32-bit Universal, Base system 10.5, valid architectures x86_64 i386. That should do the trick, and I've ensured that the project settings are the same for the executable as they are for the project info.
If that's not it, we may need help from the c74 crew on this...
I format my entire disc, and everything is just fine now !
Thank you.
Ad.
Glad it's working! Sorry you had to go through all that trouble, but all the same, I'm glad it's not the external causing you grief. ;)
Hello cap10subtext,
thanks a lot for this external,
My Idea was to connect artoolkit with the jit.ogre external, but it seems there is a bug when i try to connect jit.ogre models and draw_pixels.
My jit.window is showing the both but flash...
do you have any suggestions?
Hi Pulsart, check out the following post, I've had some issues with the jit.ogre external too (the potential is irresistible, no?) which are outlined there (including the draw_pixels issue and an animation "bug?" I'm sure you'll come across). Rob posted a solution with a videoplane that I found worked in some situations but not in others. So I'm not sure what the ideal solution is I had a few issues with it regarding mixing it with other GL shapes (his solution is posted in that thread). Maybe it'll work for your purposes?
The other thing I was having trouble with was changing the z offset and rotation transform on jit.ogre objects. But perhaps mention to Rob (or add to the jit.ogre thread) that you too would be interested in seeing if an equivalent to the gltranslate or glrotate feature could be added to the jit.ogre object?
I am very interested too in the possibility of integrating Artoolkit and Jit.ogre.
+1.
A new download has been posted:
This is the first compile I've done for ARToolkit for Max/MSP/Jitter for the PC! So far tested on Windows XP. I need PC Beta testers since I have limited access to a PC. Anyone who can help me with the code is welcome to as well, there are few differences between this and the Apple compile notably the path names and a few different headers. Could also use some pointers about the resolution. 640x480 is fine on a Mac for tracking markers, it seems really pixellated on the Windows machines I've tried.
There were a lot of issues with this code, but instructions are all there in the README to get it working. There was a huge issue with the dlls but the ones you need are included, simple install them in the local support folder. Again I have to remind all that the code is provided AS-IS. I bear no responsibility for anything that might happen to your system as a result of using my terrible code. :)
Note the other thing different about this version is that you need to install the multi folder in your local project folder before you start your jit.artkmulti object. Otherwise it won't find the markers and you'll get a crash when you start the video.
Hi cap10subtext - a quick bug report on windows 7 (not sure if its just me or everyone though), the object looks for camera_para and object_data files in C:Program FilesCycling '74Max 5.0artk.support , although for most people it will be in C:Program Files (x86)Cycling '74Max 5.0artk.support
I checked I have the paths set up correctly (with the (x86)) in preferences. I overcame the problem by copying artk.support to a new path where it was looking for the files.
Thanks for the report. If anyone else has Windows 7 (64 bit) I'd appreciate knowing if that path is uniform between machines.
New download posted on Github to correct path name to C:Program Files (x86)Cycling '74Max 5.0artk.support for anyone else with this issue. Would love feedback as to whether this solves the problem.
Note: it's not a 64 bit compile, it's just the different path name.
Hi cap10,
Is it possible to organise artk to follow the standard max paths?
I generally keep all my foreign objects in a particular folder which is added to MaxPath.
Artk did not work from here and seemed to want to be in Max5 root. Would prefer not to symlink . .
Apart from that it seems to work. Mac i7 QC, 10.6.
Cheers
MM
Macciza: May not be possible, there's an issue with the function that parses the path name that won't allow for spaces in the paths on a Mac and I think that it's in one of the libs. Have had a hard time tracking it down and don't know if I can recompile it.
Would also need an example of how to use the Max paths in an external.
I will keep trying.
Okay, after tinkering a bit I think I've solved the issue with spaces in the path name. Turns out it was a parsing issue in one of the classes.
If i can figure out how using a max path works for other externals I may be able to implement it after all.
New object posted for OS X, should work with the Max File paths now. Just place the folder anywhere in the Max search paths and add this:
max fileformat .dat DATA 0 "data file" file;
into the file:
/Applications/Max5/Cycling '74/init/max-fileformats.txt
Request to Cycling '74 to add the Data file to max-fileformats.txt permanently.
To build a standalone, just add the camera_para.dat and the multi folder to .app/Contents/support/
This release works with a new format for object_data2 in /multi, so you should use the new one with the new object.
Hi cap10subtext - I've tested your win7 x64 fix and it works fine for me (I have deleted the old non-(x86) folder).
I also have a question - in the windows readme, does the line "The calibration of the camera is default .dat camera file only" mean its not possible to replace the default camera_para.dat file with a user-made one?
Thanks
It's possible, but may not be desirable at this time. I tried a different calibration that was MacBook iSight specific and instead of improving tracking it mucked it up. Not sure what the fix for that would be at this time. If someone who knew ARToolkit front to back could point out the issue in the code I'd fix it in a heartbeat...
Interesting. Maybe try a test dat after doing the one-step calibration yourself: I'm using a video camera (HV20) + capture stick and found it reduced flickering quite a lot (although I also changed some lighting around that time too, so I couldn't say for sure it was just calibration).
Also another note - on your website it says to edit object_data2, but so far mine has only been reading object_data (I've actually renamed 2 to avoid confusion). I am only using single marker tracking (of 4 markers) though, so if thats related to multi-marker tracking, ignore this.
Thanks for producing this external - its awesome and just what I need for a project I'm doing in a nail salon soon. The only challenge at the moment is how temperamental artoolkit is - every morning I load up the working patch from last night and have to find out why its now crashing.
But this seems to be the way with the standalone artoolkit binaries too (crashing if the camera is the wrong kind, or the marker file isn't found). I guess its the price of working with such specialist tools.
I just checked and the alignment is in fact better with the default file. (Assuming that the dat file doesn't influence the tracking, and only affects the alignment of objects rendered.)
This could be because I failed to play with the lens_angle and position offsets to counter the new alignment, but to my eye the default is good enough for me.
Thanks for the kind words. It's been a dream of mine for the longest time to get something working natively in Max but of course it's come together finally now that I barely have time to work on it. So I apologize for the crashing, I just haven't had the time to to do all the error checking that the open source ARToolkit seems to lack to make it play nice with Max.
And I apologize in advance that the windows versions won't get quite the same level of attention because frankly I hate using windows. Would be more than happy to post the Visual Studio 2008 or 2010 project files if someone wants to help maintain it.
And for windows it is object_data not object_data2. As I said, not a lot of time to throw this together so my naming conventions are out of whack! :)
Oh and regarding the calibration, it's because there's a bit of a hack involved to get a coordinate system that works with out too much fiddling in Max. In ARToolkit, I believe there is an extra step they do to apply the resulting projection Matrix from the camera file to the transformation matrix but it's done as a rendering step in OpenGL (I could be mistaken about certain details) which make calls to the glu viewport which we aren't using in Max so I just bypassed it and corrected the tracking to work with the default file. Ulitmately would love to have it work properly but I'd need to pick the brain of an ARToolkit expert and a better C programmer than I.
New object posted for Windows, this object loads properly on my machine (XP SP 3) but I'm having all sorts of unrelated webcam issues and can't test it properly. Please let me know if it's not working for you.
This release should work with the Max File paths now, therefore the x86 path fix should be no longer necessary, please feel free to report any bugs. Just place the support folder anywhere in the Max search paths and REMEMBER to add this:
max fileformat .dat DATA 0 "data file" file;
into the file:
Program FilesCycling '74Max 5Cycling '74initmax-fileformats.txt (or the Runtime equivalent).
I'd like to request to Cycling '74 to add the Data file to max-fileformats.txt permanently, if anyone has any complaints or a conflicting standard please let me know.
Have never built a PC standalone but should work when dlls, camera params and multi folder copied to support folder.
This release works with a new format for object_data2 (note it no longer supports object_data from the original download) in multi. Please erase old support folders in your path to avoid a conflict.
Copy of GPL license added to PC release.
Hi cap10subtext - thanks for the update. I've had the previous version running without issues since a while after my last post, once I narrowed down some 'rules' to avoid crashes. Basically the bullet-proof way to launch it for me is by double-clicking the patch I want to edit while Max is closed. The fact opening it from the file menu in Max doesn't work makes me suspect it might be related to filepaths, so maybe this new version will fix that.
I've created a rough proof-of-concept of an automatic-thresholder for single black and white markers. It should reduce drop-outs due to changes in light like glare or shadows. I'm about to move to colour markers, so I won't have time to work further on it. I think a lot of the heavy lifting involved could be moved to the GPU, but at the moment it isn't too intensive.
It's very rough so I think the patching that determines how often it runs doesn't actually work properly (did it late at night) and the patching in general is just how it came out while working through the problem (this is the first patch I've posted).
Additional improvements could include running the 3m on each plane for colour markers (and only using full/zero RGB values on the markers like 255 0 0 / 255 255 0 /0 0 0), or running for a few frames after losing the marker and widening the sample radius (in case it can pick it up again in the same area with more contrast data). Note the patch is looking for marker 2.
"Basically the bullet-proof way to launch it for me is by double-clicking the patch I want to edit while Max is closed. The fact opening it from the file menu in Max doesn't work makes me suspect it might be related to filepaths, so maybe this new version will fix that."
Hmmm... interesting about the file menu issue, hadn't noticed that. Hopefully this does get resolved with the latest update.
Glad it's working out for you. The way I see it, there are likely 3 more very important issues, however, it's possible that they will never fully be resolved.
1) I need to figure out how to properly handle the memory of this object. I have an example which I'm waiting on which should give me an idea of how to accomplish this by properly allocating memory to the loading of the marker "objects".
2) There doesn't seem to be a way to safely handle the ARToolkit libraries, the normal cleanup routines don't have any effect.
3) there must be a way to correct for the proper camera calibration files but this totally escapes me at the moment. You can see when the marker gets further from the camera the skew becomes more obvious.
The final issue which is less pressing is that the "detection" output is still not precise... It seems to have trouble with multiple markers entering and leaving the scene simultaneously.
Having said all of that, I like where you are going with this patch (you are right in that it seemed to attempt the fix once, then it would stop working). Happy augmenting!
I'm trying to move from single markers to multiple markers now (tracking multiple independent object, not single-position multi-pattern markers), but I'm not 100% sure if the object is supposed to support this.
At the moment I have 4 patterns loaded, but when multiple patterns are on screen, the object will only output position data for one. If I cover that marker up, it will switch to outputting another. Is the multi-marker tracking only for the kind of multi-pattern, single point tracking shown here: http://www.hitl.washington.edu/artoolkit/documentation/tutorialmulti.htm or is something wrong in my objectdata formatting etc.?
Additionally, I realised I was needlessly recreating cv.jit.threshold above, so I've switched to that for much more stable marker detection in changing lighting conditions. I'd like to find a shader that does something similar, as all the cv objects can slow things down a lot, but I've yet to come across one other than a pixelbender script.
It's supposed to run on 4 independent markers. As long as any of them track it's probably not an issue with object_data.
Which object are you using again? Can you remind me? Is it the most current one with the fixed path on PC?
Someone pointed out a bug to me recently which I've been working on but it sounded more innocuous than the one you described. Basically they all tracked but the first element of the lists (visible 1/0) was not updating properly if all objects were removed from the scene at once.
If you are using the PC version it's possible I mucked something up in the translation between Mac and PC, I was getting some errors I'd never dealt with before in VS.
If you send me more details I will do my best to investigate. Perhaps even a patch that illustrates the problem? I find recording a little video and using that for the live feed makes it easier to replicate the issue.
Sorry for the inconvenience...
Hey cap10subtext, thanks for the response. No worries about the inconvenience - thanks for releasing and supporting the object in the first place!
I've uploaded a quick demo video using the help patch (with added thresholder) here: http://www.youtube.com/watch?v=m4pBPfnWhPg
You can see it clearest by watching which message boxes get updated (only one per frame). I haven't been able to recognise a pattern to which marker it 'chooses' yet. Please forgive the noisy signal too - its night here.
I'm using the most recent object- but I've checked the previous one and it shows the same behaviour.
Here is the content of my multi directory: http://www.bferns.com/files/multi.zip
And the patch and Max window output:
Jitter 1.7.0 installed
cosm version: 'Mad Professor'
*** Initializing ARToolkit for Max/MSP***
ARToolkit Property of ARToolworks
Max External by Andrew Roth -- aroth21@yorku.ca
Alpha Release Only - r004
Image size (x,y) = (640,480)
Opening Data File /Program Files (x86)/Cycling '74/Max 5.0/artkmulti support/multi/object_data2
About to load 4 Markers
Marker finger1 from oulet 1
Marker finger2 from oulet 2
Marker finger3 from oulet 3
Marker finger4 from oulet 4
--->>> Objectfile num = 4
jit.gl.render: building GL on window "artoolkit"...
My guess is that you haven't added the following line:
max fileformat .dat DATA 0 "data file" file;
into the text file:
Applications/Max5/Cycling '74/init/max-fileformats.txt
Then restart. Max doesn't natively recognize .dat files in the search path.
I add this line, I restart, max and my computer but I have the same errors:
Camera Params at: /Applications/Max5/artk.support/camera_para.dat
Camera parameter load error!!
Opening Data File /Applications/Max5/artk.support/multi/object_data2
Can't find the file - quitting
--->>> Objectfile num = 0
From your screenshot it looks like you are using an older version of the external. I could be mistaken, but try downloading the latest one from aranarproductions.com/artk and if that doesn't help, I'll look into it as soon as possible.
I reinstall the last version from site, still same errors.
I use Mac OS X 10.5.8
Max Msp 5.1.7
Xcode 3.0
I just update Java
These are the messages from max window
*** Initializing ARToolkit for Max/MSP***
ARToolkit Property of ARToolworks
Max External by Andrew Roth -- aroth21@yorku.ca
Alpha Release Only - r002
Image size (x,y) = (640,480)
Camera Params at: /Applications/Max5/artk.support/camera_para.dat
Camera parameter load error!!
Opening Data File /Applications/Max5/artk.support/multi/object_data2
Can't find the file - quitting
--->>> Objectfile num = 0
Okay, totally my fault. It was a bad link on that page. Fixed now.
This should be the direct download to the new one:
https://github.com/downloads/cap10subtext/jit.artkmulti/ARToolkitMax.zip
It should be r003.
Delete the old, and dump in the new.
I have a problem with the barcodes not being tracked at all. I am on win xp, I have downloaded the win pc of artoolkit, i have followed the readme instructions as well. I am not sure what I am doing wrong...
One possibility Adity: if the webcam is being mirrored in your webcam manager it likely won't work.
Also if you opened the patch, closed it and reopened it, or made an object and then opened the help patch, it causes the output to cease completely. Only way to get it going again is restarting Max. Totally inconvenient, I know.
Thank you very much for all. Now it works like charsm
pecta: Glad to hear it! Sorry for the inconvenience.
adityapretzelz: feel free to post screen shots, sometimes that helps determine the problem.
I have another question: What format must be the markers. On http://www.roarmot.co.nz/ar/ they save in pdf format.
I saw in multi folder a strange format .
Thank for your patience.
Perhaps the newer versions of ARToolkit accept that format; however, the open source one still requires a patt format.
Confusingly the files in the multi folder are called patt.a, etc...
You can use the site you posted to create and print a marker, but then you should go the the following site to create the .patt file you need to add:
http://flash.tarotaro.org/blog/2009/07/12/mgo2/
(set image segments to 16x16)
Put your new patterns in the multi folder, and edit object_data in a text editor (word pad or text edit) to add the name of the marker and a descriptor (one word no spaces) so the Max window will tell you which outlet the position reports from. I suggest making a backup copy of object_data.
For more information on making markers:
http://www.artoolworks.com/support/library/Creating_and_training_new_ARToolKit_markers
I made using http://flash.tarotaro.org/blog/2009/07/12/mgo2/ a j.patt file . I can use this or must change something to have a file equivalent with patt.c?
pecta - MGO2 should work fine. You can also use http://saqoo.sh/a/en/676 as a downloadable AIR app. People have found they give different results, so experimentation is the best way to go.
If you have a relatively simple pattern some people have found greater success by hand-editing the pattern files in a text editor (it will hopefully be clear when you open it - RGB values in multiple fixed-width columns and rotations).
Also although ARTK+ and some modifications accept colour markers, I think with ARTK you're stuck with black and white, despite the RGB marker data; but thats just an assumption I made from some old documentation.
Mgo2 work fine but my confusion is related with the output file from MGO2. It is a j.patt. In multi every file is patt.a, patt.c , etc.
On http://www.artoolworks.com/support/library/Creating_and_training_new_ARToolKit_markers it is not very clear how i change from j.patt to patt.j.
thanks
you can just rename the file from j.patt to patt.j
Note: just installed 5.1.9, had to re-add the DATA file format back to the text file to make it recognize camera_param.dat
Jit.artkmulti works on Max 6, but the help file doesn't.
Improvements en route... Stay tuned.
First steps into a new help file. Haven't had time to put a proper one together yet but this should solve the basic issues with draw_pixels being deprecated.
By the way, there seems to be an issue with the model not loading at the correct scale. I could just be doing something wrong... but scale it up manually and it works.
Thanks for this great work. It's really sweat to have all-axis fiducials in Max :)
Is there a clear list of what the 14 floats are for the outlets in [jit.artkmulti]? Also, what is the scale of those outputs?
I'm having a hard time trying to figuring out how to recreate it in gl, tying rotation to rotation attributes, etc.
here is my patch, if it helps.
thanks,
mattt.
Hi matt, the list of outputs should be in the earlier help files. I've attached a version to the bottom of this post.
jit.artkmulti outputs a list with the following elements:
1. marker visible? (enable $1)
2., 3., 4., 5., rotation in Axis-Angle {w, x, y, z}
6., 7., 8., position {x, y, z}
Are you on Max 5? The only way to "properly" do this in Max 5 is to hand over the rendering of the objects to jit.gl.sketch because you need to do the transforms hierarchically using rotation, position then gltranslate (inherited position) and glrotate (inherited rotation), but I'm not sure if it's possible to do with text objects (I can't recall ever trying). To do that you'd have to look at the jit.gl.sketch help file and the draw_matrix command.
In Max 6 this is easy because of the anim.gl.node objects which allow for hierarchical transformations without needing gltranslate and glrotate.
If you are going to be doing more AR/VR stuff and you want to use Max, I really suggest using Max 6 wherever possible. It's so much easier.
Hope this helps?
hey,
i have upgraded the support patch a bit. you can load 4 obj files and they also disappear when the marker is not visible. hope this is helpful for someone :)
lci
Thanks for contributing! I haven't had a chance to look at this in a while, it was originally designed so that $1 would tell you if the marker was visible or not but some users were reporting that it didn't work perfectly with multiple markers. How does your approach work when you add and remove single markers from the scene?
Hey
I'm trying to use arToolkit on Max 6. I open the help file but when I start the qmetro max only shows my the rendered image, the virtual one. But I can't see me on the screen, do I have to put anything else in the patch to to that?
I'm running it in a 10.6 OSx
Thanks!
Soudns like an interesting topic, what adityapretzelz said made me think about the recognition barcode webcam java lib. Hope I can know more...
Great post! And I want to recognise barcodes in my win7, but I have any idea about java code to recognise barcodes. hope your post will help me.
Thanks!
Hi,
I’m using Windows8 but I have same issue as JOPAMO :
"I’m trying to use arToolkit on Max 6. I open the help file but when I start the qmetro max only shows my the rendered image, the virtual one. But I can’t see me on the screen, do I have to put anything else in the patch to to that?"
Anyone know what should we do in this situation?