lyallmarcus

I'm wanting to turn a live video feed of the silhouette of a body into a vector outline.

I'm grabbing a live video image and converting into a single plane high-contrast image. So the body is black and the surrounding area is white.

I want to do some kind of edge detection that gives me the co-ordinates of the black-white edge. But I want to list them in order so that if I joined them up using the lineto command in gl.sketch I would get a nice continuous shape.

A bit like 'join-the-dots'. So that an outline of the figure can be drawn live.

I've been pointed towards Cyclops, but I'm not sure this is right.

I want to do some kind of edge detection that gives me the co-ordinates of the black-white edge. But I want to list them in order  so that if I joined them up using the lineto command in gl.sketch I would get a nice continuous shape.

I've been pointed towards Cyclops, but I'm not sure this is right. 

making-a-vector-outline-from-a-silhouette

In a perfect world, the following would work:

1) Take your silhohuette, threshold it to get a binary image

2) take the gradient of the binary image (you can probably do just

fine with jit.sobel, if not, you can use the finite difference

method..if you do use this method, I can send you an object that does

3) Threshold the gradient to get just the edge pixels

4) spit out the cell coordinates of the edge pixels ( you can use

5) convert from pixel to opengl coordinates

1)Image has noise, need to smooth it first. Try a median filter. I

can send you one of these too if you need it.

2)too many coordinates (oversampling)-->try out jit.demultiplex, but

this might cause you to lose smaller shapes like fingers and feet.

3) probably more issues, but no more come to mind.

1) Take your silhohuette, threshold it to get a binary image
2) take the gradient of the binary image (you can probably do just
fine with jit.sobel, if not, you can use the finite difference
method..if you do use this method, I can send you an object that does
it)
3) Threshold the gradient to get just the edge pixels
4) spit out the cell coordinates of the edge pixels ( you can use
xray.jit.cellcoord)
5) convert from pixel to opengl coordinates
6) draw

Problems you might encounter:
1)Image has noise, need to smooth it first.  Try a median filter.  I
can send you one of these too if you need it.
2)too many coordinates (oversampling)-->try out jit.demultiplex, but
this might cause you to lose smaller shapes like fingers and feet.
3) probably more issues, but no more come to mind.

> 2)too many coordinates (oversampling)-->try out jit.demultiplex, but

> this might cause you to lose smaller shapes like fingers and feet.

it would be worthwhile to try downsampling the image (pre-edge detect 

would be more cpu efficient, but could give better quality?) to a low 

resolution - play with the dim message that works best...no need to be 

it is nice to know about the xray.jit.cellcoord. i started to reply to 

this email with a gedanken patch, but my brain got lazy/stumped at how 

to pull just the white coords w/ jitter's obejct set. Looks like wes 

> 2)too many coordinates (oversampling)-->try out jit.demultiplex, but
> this might cause you to lose smaller shapes like fingers and feet.

it would be worthwhile to try downsampling the image (pre-edge detect 
would be more cpu efficient, but could give better quality?) to a low 
resolution - play with the dim message that works best...no need to be 
a slave to 4x3!

it is nice to know about the xray.jit.cellcoord.  i started to reply to 
this email with a gedanken patch, but my brain got lazy/stumped at how 
to pull just the white coords w/ jitter's obejct set.  Looks like wes 
solved that one, too.

You guys are working way too hard... Happy Valentine's.

You guys are working way too hard...  Happy Valentine's.

That's pretty cool. You love that isosurf object don't you!

That's pretty cool.  You love that isosurf object don't you!

And here's a little toy begat from Andrew's example:

www.killingfrenzy.com/Jitter/Vectorizer.zip

Beware the "detail" setting on slower machines.

And here's a little toy begat from Andrew's example:
www.killingfrenzy.com/Jitter/Vectorizer.zip
Sloppy, but fun to play with.
Beware the "detail" setting on slower machines.


Couldn't get Andrew's thing to open. (could see foo window but couldn't see the patch window). But the vectorizer patch is definitely a step in the right direction.

But I want to be able to access the individual points that make up the shape. So isosurf is doing exactly the right thing in that it's making a shape from the silhouette. But I haven't got access to the data that makes up the shape.

I started off making a patch like wesley's suggestion (listed underneath).

I can get all of the coordinates and get them to draw a shape, but they aren't nicely sorted as they are in the isosurf examples. They just come out in matrix order. So when you use lineto in jit.gl.sketch, it draws the lines in matrix order, rather than the elegant outline manner of the isosurf example.

How do I get the isosurf object to spit out the nicely sorted list of opengl coordinates it is using to draw the shape?

The reason I need to do this is that I want to use the convert the co-ordinates into control voltages to send to an oscilloscope-type device.

Currently I'm doing this through a soundcard that allows DC offset.

Here's my crap patch that doesn't work at all well, just to show you where I've been....

But I want to be able to access the individual points that make up the shape. So isosurf is doing exactly the right thing in that it's making a shape from the silhouette. But I haven't got access to the data that makes up the shape. 

The reason I need to do this is that I want to use the convert the co-ordinates into control voltages to send to an oscilloscope-type device. 

Can you elaborate this (if you don't mind of course) since I am kind 

of curious what you mean by oscilloscope-type device.

Can you elaborate this (if you don't mind of course) since I am kind  
of curious what you mean by oscilloscope-type device.
Thank you so much!

Well. it's an oscilloscope at the moment. But it might end up being a dangerously modified CRT projector. 

Well. it's an oscilloscope at the moment. But it might end up being  a dangerously modified CRT projector. 


and I forgot to join jit.robcross to jit.iter in my patch.

Here's some things I noticed about your patch. If possible, do math

operations in matrix form. They're much faster than doing them in max

messages. Second, jit.iter is always a bottleneck for large matrices

(meaning > say 20 cells). Instead, hook your matrix up to jit.gl.mesh

I also noticed you had a jit.bsort object in your patch. If you

decide you need to sort your matrix, I would recommend against using

jit.bsort since it's quite a slow sorting algorithm. I would instead

use xray.jit.quicksort. It's several times faster. The the patch

below, I used xray.jit.cellcoords to grab white cells. You can

Hi Marcus,
Here's some things I noticed about your patch.  If possible, do math
operations in matrix form.  They're much faster than doing them in max
messages.  Second, jit.iter is always a bottleneck for large matrices
(meaning > say 20 cells).  Instead, hook your matrix up to jit.gl.mesh
and render lines that way.

I also noticed you had a jit.bsort object in your patch.  If you
decide you need to sort your matrix, I would recommend against using
jit.bsort since it's quite a slow sorting algorithm.  I would instead
use xray.jit.quicksort.  It's several times faster.  The the patch
below, I used xray.jit.cellcoords to grab white cells.  You can
download it here: http://www.mat.ucsb.edu/~whsmith/ .

BTW, you can always do @matrixoutput 1 on jit.gl.isosurf.

It's funny that only text is shown when I open the saved patch.

It's funny that only text is shown when I open the saved patch.
Am I missing any module not installed ?

The X-ray objects are well worth having.
http://www.mat.ucsb.edu/~whsmith/

I have been playing with X-ray objects happily now, but the problem on

I guess the patch missed a few lines like,

By the way, I have looked into another project Wesley has done,

"A-Contrario". The description of what it does is very interesting, but the

project file downloadable doesn't work in my PC. Wesley, any chance we may

get a windows version of the project ? We are missed "xray.jit.probsegment"

I have been playing with X-ray objects happily now, but the problem on
loading the saved patch presist.

I guess the patch missed a few lines like,
-- max v2;
-- #N vpatcher 211 99 758 452;
right ?

By the way, I have looked into another project Wesley has done,
"A-Contrario". The description of what it does is very interesting, but the
project file downloadable doesn't work in my PC. Wesley, any chance we may
get a windows version of the project ? We are missed "xray.jit.probsegment"
and "xray.jit.median" to have it run.

> I guess the patch missed a few lines like,

What version of Max are you running?? The patch code pasted was from

the clipboard and only works with the newer versions on Max (I think

4.5 and >, but I could be wrong). Basically, it's not a full patcher

> By the way, I have looked into another project Wesley has done,

> "A-Contrario". The description of what it does is very interesting, but the

> project file downloadable doesn't work in my PC. Wesley, any chance we may

> get a windows version of the project ? We are missed " xray.jit.probsegment"

True, these 2 objects are not compiled on a PC yet. I'll see if I

> I guess the patch missed a few lines like,
> -- max v2;
> -- #N vpatcher 211 99 758 452;
> right ?

What version of Max are you running??  The patch code pasted was from
the clipboard and only works with the newer versions on Max (I think
4.5 and >, but I could be wrong).  Basically, it's not a full patcher
but a patcher chunk.

> By the way, I have looked into another project Wesley has done,
> "A-Contrario". The description of what it does is very interesting, but the
> project file downloadable doesn't work in my PC. Wesley, any chance we may
> get a windows version of the project ? We are missed " xray.jit.probsegment"
> and "xray.jit.median" to have it run.
>

True, these 2 objects are not compiled on a PC yet.  I'll see if I
can't find time this weekend to do it.

> What version of Max are you running?? The patch code pasted was from

> the clipboard and only works with the newer versions on Max (I think

> 4.5 and >, but I could be wrong). Basically, it's not a full patcher

I am using the latest version of Max/Jitter. Maybe I am too green on using

the program, I don't know the different between patcher and patcher chunk.

I always copy and paste the texts into a text file and save with .pat

extension. Usually it will work when I open it, but sometimes it fails with

> True, these 2 objects are not compiled on a PC yet. I'll see if I

> What version of Max are you running??  The patch code pasted was from
> the clipboard and only works with the newer versions on Max (I think
> 4.5 and >, but I could be wrong).  Basically, it's not a full patcher
> but a patcher chunk.

I am using the latest version of Max/Jitter. Maybe I am too green on using
the program, I don't know the different between patcher and patcher chunk.

I always copy and paste the texts into a text file and save with .pat
extension. Usually it will work when I open it, but sometimes it fails with
only text displayed. _|~~|O

> True, these 2 objects are not compiled on a PC yet.  I'll see if I
> can't find time this weekend to do it.

> > What version of Max are you running?? The patch code pasted was from

> > the clipboard and only works with the newer versions on Max (I think

> > 4.5 and >, but I could be wrong). Basically, it's not a full patcher

Sorry, those weren't technical terms. If you're using the latest max,

you should be able to open a new patcher window and paste the text in

there. The text will magically become a patcher.

On 2/16/06, William Wong  wrote:
> > What version of Max are you running??  The patch code pasted was from
> > the clipboard and only works with the newer versions on Max (I think
> > 4.5 and >, but I could be wrong).  Basically, it's not a full patcher
> > but a patcher chunk.
>

Sorry, those weren't technical terms.  If you're using the latest max,
you should be able to open a new patcher window and paste the text in
there.  The text will magically become a patcher.

Thanks for all the excellent advice so far.

For any newbies (like myself) following this thread, I've run into a couple of issues.

I installed the xray objects but couldn't see them in max.

So I built an object list and put it into the cycling74/init folder. Bingo!

Couldn't find one with the installer. This one isn't very elegant (sorry Wes) but seems to do the trick.

But my main issue is that if I open the patches that have been posted, I only see a jit.window and not the patch itself.

Same problem on Andrew and Wes's patches. Thought it had something to do with the lines at the top of the text. I'm assuming these refer to where the patcher window appears.

I thought that because I'm on a little powerbook, maybe the patcher window was out of sight. So edited the text so that it's the same as the patch I posted. Still no joy.....

I'm probably missing something very obvious here.....

So I built an object list and put it into the cycling74/init folder. Bingo! 

But my main issue is that if I open the patches that have been posted, I only see a jit.window and not the patch itself. 

Same problem on Andrew and Wes's patches. Thought it had something to do with the lines at the top of the text. I'm assuming these refer to where the patcher window appears. 

I'm probably missing something very obvious here.....


> I installed the xray objects but couldn't see them in max.

Do you mean they didn't show up in the New Object List? Or when you

right click on the patcher background? If you put the objects in

/jitter-externals/xray then they are definitely in your search path

and you should be able to use them without having to make a list. 

Personally, I don't use the object lists in max so I've never bothered

> But my main issue is that if I open the patches that have been posted, I only see a jit.window and not the patch itself.

> Same problem on Andrew and Wes's patches. Thought it had something to do with the lines at the top of the text. I'm assuming these refer to where the patcher window appears.

> I thought that because I'm on a little powerbook, maybe the patcher window was out of sight. So edited the text so that it's the same as the patch I posted. Still no joy.....

> I'm probably missing something very obvious here.....

What are the precise steps you do to take text and turn it into a

patcher? I'm working at 1280x854. This shouldn't make a difference. 

Do you take the text and paste it into a new max patcher? This is the

way you should go. I've never had a problem with this technique but

maybe your system doesn't take to it somehow. If you paste the text

into a new patcher, the patcher won't be offscreen as it's right there

> I installed the xray objects but couldn't see them in max.
>
Do you mean they didn't show up in the New Object List?  Or when you
right click on the patcher background?  If you put the objects in
/jitter-externals/xray then they are definitely in your search path
and you should be able to use them without having to make a list. 
Personally, I don't use the object lists in max so I've never bothered
to make one.
>
> But my main issue is that if I open the patches that have been posted, I only see a jit.window and not the patch itself.
>
> Same problem on Andrew and Wes's patches. Thought it had something to do with the lines at the top of the text. I'm assuming these refer to where the patcher window appears.
>
> I thought that because I'm on a little powerbook, maybe the patcher window was out of sight. So edited the text so that it's the same as the patch I posted. Still no joy.....
>
> I'm probably missing something very obvious here.....
>

What are the precise steps you do to take text and turn it into a
patcher?  I'm working at 1280x854.  This shouldn't make a difference. 
Do you take the text and paste it into a new max patcher?  This is the
way you should go.  I've never had a problem with this technique but
maybe your system doesn't take to it somehow.  If you paste the text
into a new patcher, the patcher won't be offscreen as it's right there
where you created it.

still don't see them in the right-click context.

but with the list they show up in the new object window.

I've been opening a text file, copying the posted patch text into that and then saving it. Then opening as a patch. Dumbkopf!

Please excuse my ignorance. Will examine patch and let you know how I get on.

xray.jit.cellcoords looks like the daddy....

still don't see them in the right-click context.
but with the list they show up in the new object window.

I've been opening a text file, copying the posted patch text into that and then saving it. Then opening as a patch. Dumbkopf! 

Please excuse my ignorance. Will examine patch and let you know how I get on. 

So using matrixoutput on jit.gl.isosurf seems to be spitting out the gl co-ordinates in order. Which is fantastic.

The only thing is that there are rather a lot of them.

The dimension of the matrix seem much bigger than they should be. ie, if the dimensions given to the isosurf object are 80 x 60 x 2, then the resulting 'matrixoutput' matrix is

My ideal scenario would be to filter this matrixoutput data so that I get a 2d matrix at around the same resolution as what I'm feeding the isosurf object. So if isosurf res is 80 x 60, I get a matrixoutput 1 x 480.

I don't need the third dimension as I'm only drawing in 2d.

Trying jit.submatrix, etc but no consistent joy yet.

The reason I need to filter the information is that it will eventually be fed as a 2d 1 x 1920 matrix to jit.release @ 25fps to create a 2 channel 48k audio signal.

Sorry for the length of this saga. Here's an amalgam patch.

So using matrixoutput on jit.gl.isosurf seems to be spitting out the gl co-ordinates in order. Which is fantastic. 

The dimension of the matrix seem much bigger than they should be. ie, if the dimensions given to the isosurf object are 80 x 60 x 2, then the resulting 'matrixoutput' matrix is 

I don't need the third dimension as I'm only drawing in 2d. 

The patch below outputs a single line matrix of about 5-6000 points.

But if I try to reduce it down to my target of a fixed matrix 1920 points, the drawing order gets messed up.

I've bastardised the cellvalue help file to show what I mean.

Any tips on maintaining the drawing order or am I trying to square a circle?

But if I try to reduce it down to my target of a fixed matrix 1920 points, the drawing order gets messed up. 

I've bastardised the cellvalue help file to show what I mean.
Try pressing the adapt switch.

You can get less point and better efficiency by using a smaller

matrix. I set the dims to 80 x 60 and the contours basically look the

same. About the contouring algorithm. It doesn't follow curves at

all. It works by scanning through a matrix of values one row after

the other. If you imagine the video image in grayscale format as a

heightmap with bright values as peaks and black values as valleys, the

contouring algorithm is looking for the location of particular

heights. Obviously, this is going to be between pixels for the most

part. So, when the algorithm encounters a change in value from one

pixel to another that passes through the level it is looking for, it

creates a line at that location. The result of this is that the

output is sets of pairs of points without regard for order. When you

force the output into a fixed size matrix, you are mucking up the

carfully chosen pairs of points and getting bad connections between

lines. If you absolutely must have a fixed number of points, I'm

afraid this approach will only cause many headaches. If you can

handle a certain number of points below a certain maximum, this

I would suggest playing with the dimensions of the matrix you apply

the contouring algorithm to as that will determine for the most part

about how many points you get back. then,I would add a safety that

clips the size of the matrix to a certain value.

If you absolutely must have the points in order, I'm afraid you're out

of luck as both the contourmap and the isolevel algorithms say nothing

about connectivity. This requires quite a bit of extra logic that is

a pain in th arse to implement, which is why no one has done it for a

jitter object so far (as far as I know). I did a rudimentary

connectivity-based object a few months ago, and it was really

ok,
You can get less point and better efficiency by using a smaller
matrix.  I set the dims to 80 x 60 and the contours basically look the
same.  About the contouring algorithm.  It doesn't follow curves at
all.  It works by scanning through a matrix of values one row after
the other.  If you imagine the video image in grayscale format as a
heightmap with bright values as peaks and black values as valleys, the
contouring algorithm is looking for the location of particular
heights.   Obviously, this is going to be between pixels for the most
part.  So, when the algorithm encounters a change in value from one
pixel to another that passes through the level it is looking for, it
creates a line at that location.  The result of this is that the
output is sets of pairs of points without regard for order.  When you
force the output into a fixed size matrix, you are mucking up the
carfully chosen pairs of points and getting bad connections between
lines.  If you absolutely must have a fixed number of points, I'm
afraid this approach will only cause many headaches.  If you can
handle a certain number of points below a certain maximum, this
approach can work.

making a 'vector outline' from a silhouette.