Infrared Drawing Installation: Drawing 2D images based on Tracked LED Pen?

    Gmix
    Nov 03 2011 | 3:35 pm
    Hello all, Jitter newbie here, wondering how difficult of an installation this would be to put together for an Art Class I am taking. I have done some tracking using the wiiremote's infrared camera before using laser pointers and infrared LED pens, so I know how to get the drawing part set up, but the rest I'm not to sure about... Here's what I'm going for:
    The user will use an LED pen to draw a line on a projected surface. The patch will generate a line segment based on the start and endpoint of the user's "stroke." Then Jitter would select (randomly for now) from a specified library of .bmp files and "draw" in that figure according to the orientation of the line segment that was drawn. The images will be "lassoed" (cropped to their form) digital photos of random objects or people. Each one will have a "skeleton" that will be used as a reference for orientation according to what angle the line was drawn. So, this will have to be saved as some kind of metadata with each file or group of files. After x number of lines are drawn, the image is saved to a new .bmp file. This would give the user the sensation of painting an entire scene or portrait using just minimal "brush strokes."
    I'm pretty sure Jitter is capable of doing all that but these are the parts that I could use some advice or at least a hint for what objects to look at:
    1) storing metadata for the "skeletons" -Maybe this isn't even necessary if I just save each file so that the default orientation would just be straight up and down? Then I would just have to measure the angle of the drawn line and offset the .bmp by that much
    2) Getting a straight line from the user input -It is likely that since these users will not have practice drawing straight lines, and they might forget to release the on/off for the LED in time, creating an unwanted curve or tail at the end of the segment. I would then need a way to average the line out or tell it to take just the part that follows one consistent direction across the screen
    3) Using the angle of the line input to tell Jitter how to draw the next .bmp - For now I just want the images to appear roughly in the same region of the projected surface as it was drawn and at the same angle - Incorporating Scaling would be nice, but not essential for this first version
    If I can learn how to accomplish those three tasks, I think I should be able to patch together the rest in Max. Let me know if you have experience with anything like this, or can help point me in the right direction. So far, I'm just working with an LCD for drawing the line input, and not really sure how to convert it to a jitter matrix so it can be properly analyzed by whatever jitter objects I will need. Thanks a billion,
    G
    • seejayjames
      Nov 04 2011 | 1:37 am
      Interesting project with some definite challenges. You can start by replacing lcd with jit.lcd though? so it's in a matrix form for you. You might look to jit.sketch as well for drawing the lines...working in GL land might make some of the orientation etc. easier later, anything you draw can be scaled/rotated/repositioned after drawing, unlike LCD or jit.LCD.
      For the line drawing, I would rely on the start/stop points only, and just connect a line between these. It won't feel as "organic" but you're not really making a "paint" program, it sounds almost more like a CAD thing...? so that lines are auto-straightened, you're building a kind of architecture in the scene? I think it would be a bit tricky to take a curved line and figure out a good straight-line approximation, though using the start and end points and the delta it would be feasible. You could easily incorporate scaling with the length of the line.
      Curious to hear your progress, this seems like a visual-art version of many "paint-music" ideas which have been tossed around. I like the idea of making minimal brush strokes able to paint a whole scene. I also wonder how far you could go with the technique, it might be interesting to have additional refinements possible with specific brush strokes.
      And, if you use GL land you can make it 3D if you want... :)
    • Gmix
      Nov 08 2011 | 8:01 pm
      Thanks for the help again, seejayjames... Really appreciate your support and encouragement for all my wacky Max ideas, seems you always have some very helpful hints.
      So, I've started messing around with jit.lcd, and got it to "paint" an image (as long as I started in the top left and sloped down to the right), and everything scales fine. Assuming I was satisfied with having all of my images oriented vertically, I could almost be satisfied with what I have and call it a day. However, I wish there was some way to take the slope of the line (which I have already calculated inside of the mousedrawing sub patch) and use this to set each new object's orientation. There doesn't seem to be an offset or orientation attribute for the "drawpict" command in jit.lcd, so I guess that means I will have to use a separate jitter object to do this. What do you suggest as the simplest 2d method for achieving this offset idea, jit.rota maybe? I don't need to worry about 3d drawing for now, just something that would allow me to rotate the picture based on the slope between the start and end points of the line. I'd also like it to keep the new image file centered over the drawn line, and not just use say the left or right edge of the image to line it up.
      I really wish for simplicity's sake that in the end I could draw in the same window over which the images appear, and I hope this will still be possible even if I need several different Jitter objects to get the proper orientation. This may seem like a silly question, but I would be feeding this other jitter object (the one for handling the rotation) into the jit.lcd, right? Or would I combine them both into a new pwindow?
      Here's what I have so far, so you can see where I'm at so far. Thanks again for the help, G