Data Mining

May 15, 2010 at 8:25am

Data Mining

Hello maxers,

i update/improve externals,
(mac only but sources with).

[zoulou] for Factor Oracle,
[charlie] for SOM,
[foxtrot] for HMM,
[yankee] for Growing Neural Gas.

Thanks for feedback.

Attachments:
  1. FCZY.zip
#50365
May 18, 2010 at 6:25am

Hello maxers,

in case of : i update my_generative-1.8 in “share pages” ;

i add an external [november] for cellular automata sequencing ; you can use it only with my abstraction ; so that’s really not useful for community ; but for fun …

Work still in progress (mac only) ; thanks for reporting bugs, comments, and feedback.

#180682
May 18, 2010 at 12:17pm

Your externals look very interesting!
Could you please write some words about [zoulou]?
What do the “lrs” and “straight” parameters do ?

I’ve found a paper on Factor Oracle for pattern matching.
But [zoulou] doesn’t seem to do pattern matching, does it?

#180683
May 18, 2010 at 1:02pm

These externals are indeed very interesting, but I don’t quite understand the list format they seem to be using, are the four entries somehow related? Is the order of which results are put out somehow the result or is the result within the numbers of the list being sent out?

Are you supposed to use the list positions for different data like [note vel lenght duration]?

I’ve started compiling some of them for windows, so I will post them here when done.

#180684
May 19, 2010 at 5:56am

Hello maxers,

@mudang :

[zoulou] doesn’t really do pattern matching ;

the algorithm is used to make a kind of data memory where same patterns are connected so when you “bang” [zoulou] you randomly navigate inside this memory (straight / forward in time / backward in time) but ALWAYS respecting a kind of coherence ;
“straight” is percentage of chance that you go straight in recorded data ;
“lrs” is threshold of SameRepeatingNumber to go back in time.

That’s really hard to explain well for me ; i now that i should do a paper to explain, but to be honest i’m lazy ; on your downloaded pdf it may have a drawing of the graf generate by Factor Oracle, and to resume [zoulou] just randomly run on arc between nodes …

You can use [zoulou] to find repeating patterns in a data flow too …

!
!
!

@veqtor : thanks for windows compilation.

[zoulou] accepts all kind of int.

All the others accept all kind of list with a max size of 256 (i’m not sure) but limited to midi values (0 – 127) ; f.e. “learn 40 50 67 12 127″ ; when you “bang” externals you ask to give you randomly a vector ; you can choose which on using “int” index ; or transform a list provided ; each atom of the list is test with a simple mutation ratio chance ; if yes, it is change with external’s one ; if no, no change.

For both : try “dump_max” message to understand what data structure contains. Tell me if you want more info.

#180685
May 19, 2010 at 11:32am

Here’s the windows compile, I’m going to make November also when I have time!

Attachments:
  1. FCZY_win.zip
#180686
May 21, 2010 at 12:06pm

So, I don’t quite get it, when I use an int for index, is it giving me something which could be an continuation of a series which starts with that int?

#180687
May 21, 2010 at 12:39pm

Could you make some changes so the other objects can handle i dunno, at least numbers up to 2048? I’m trying to encode chord and root properties into a single int by using a shift register…

another feature that would be great is if one could force zoulou to start from a specific place. so maybe you send a list like [go 24] and the next bang will make it continue from there

#180688
May 21, 2010 at 4:14pm

Hello Veqtor,

Concerning “int” ; if you are speaking about [zoulou] : no, that’s just the way to add number to the structure ; if you are speaking about [charlie] or [yankee] : no, giving an “int” just tell the structure which node (so which vector) to send out the outlet.

(Even if prefer to keep externals simplest i can), I’ll do features you asked ; i mean “go” message for [zoulou] and a way to let [charlie] and [yankee] dealing without clipping values provided ; (I’ll not do it for [foxtrot] as large alphabet consume more CPU).

If somebody have others suggestions that’s time !

#180689
May 21, 2010 at 6:48pm

awesome! that would be great!

how big values can you make them handle? the bigger is obviously the better!

#180690
May 22, 2010 at 12:55pm

Hello Veqtor,

do the job ; [zoulou]‘s "go" message and no more limit for [charlie] & [yankee] ; don’t test them so much ; so tell me if you find bugs ; you should not but … ;-)

Attachments:
  1. FCZY.zip
#180691
May 22, 2010 at 3:50pm

Hello Maxers,

Post scriptum : to compile [november] you need to add "commonsyms.c" to your project.

#180692
May 26, 2010 at 11:47am

Thanks a lot, I will get to compiling them once again for windows! =D

#180693
May 26, 2010 at 4:08pm

Hello Veqtor,

you’re welcome ; nice to have feedback.

#180694
May 27, 2010 at 10:08am

So, how many entries does zoulou handle?
This is the new versions…

Attachments:
  1. 578.FCZY.WIN.zip
#180695
May 27, 2010 at 10:09am

Also, I don’t quite get how to use charlie, foxtrot and yankee. Since they just output random vectors, how do I use them in a composition? Do you use zoulou for melodies and sequences of different kinds and the three others for other stuff? Like what?

#180696
May 27, 2010 at 4:08pm

Hello Veqtor,

How many entries does zoulou handle ?

arbitrary limit : 262144 !

How do I use them in a composition ?

I don’t know !

?
?
?

I don’t use them alone ; that’s part of “my_generative” project ->(SharePages///MacintoshOnly) + (MayBeEasyToAdaptForWindows) ; there is a tutorial where i investigate some ways, but as it’s work in progress, i’m not sure at all where it leads me and : i spend more time to code/test it than using it for music ...

That’s true that [zoulou] & [foxtrot] are design for melodies and [charlie] & [yankee] for chords … but it should be possible to use those externals for any data mining task.

I aim (as thousands of guys) to create a VirtualPlasticFriend to play with me (I stink, I am shy and ugly) : for the moment i just have legs and arms ; If somebody is interesting by the project, i’m a totally AnarchopenSourceDude ;-)

PS : “dump_max” & “dump_lcd” messages help to see data in the structure to imagine how it can be used … chaos vs order !

#180697
May 27, 2010 at 4:20pm

charlie and yankee are for chords? hmm, how would you input chords into them?

as lists of pitches?

#180698
May 27, 2010 at 4:24pm

Hello Veqtor,

yep ! most of time “int” are “pitch” ; but in my project i use [zoulou] for rhythm (TaKI Gamala gamala Taki Taki) or duration ; it’s up to you.

#180699
Jun 10, 2010 at 7:34pm

Hey, some more suggestions for zoulou:
1. “suggest” message, this puts zoulou in a mode in which it outputs all the possible next steps, so if send it a bang or go it will output what possible states it can go to, this way one can select from them yourself using some other method by sending the one you want back to zoulou using “go

2. “path ” message, this makes zoulou find the shortest path from where it is now to whatever entry you put in the argument.

are these hard, possible? Would be awesome for what I’m using it for!

#180700
Jun 11, 2010 at 5:50am

Hello Veqtor,

1. easy to do ;
2. much more hard ; need to implement another algorithm for path finding.

I promise nothing, but I am going to think about it …

#180701
Jun 11, 2010 at 4:19pm

Hello raja,

thanks for feedback ; these externals are part of “my_generative” project you can find at vanille béchamel’s share pages ; my humble contribution to the AutomaticMusicGraal ; still lots of work to do as it’s not easy to find a way to compose without a linear approach.

In the :

pitch_track -> midi live improvisation -> generate sound,

process, i focus on the second job ;

and as i found so many ideas in community’s patch/pdf, i think that’s fair to share ;-)

#180703
Jun 13, 2010 at 4:03pm

Hello Veqtor,

add "get" message -> position data in new rightmost outlet :

- lrs / length repeated suffix values + refer destination ;
- arc / value + arc destination ;
.
.
.
- straight / straight value + straight destination (position + 1) ;

"path" ???

i’ll try ;-)

Attachments:
  1. ZoulouBETA.zip
#180704
Jun 16, 2010 at 4:32pm

works great! thanks a lot!

#180705
Jun 20, 2010 at 3:48pm

Hello Veqtor,

do the job for "path" ; implement a kind of BFS algo ; should work … but in case of : thanks for reporting bugs (crash ?) ; nice to have a beta tester team ;-)

"path_go" : if on (default) : find the shortest path and go next position founded.
"path_backward" : if on (default) : use “backward length repeat suffix link” while searching.

Alfred Duschnock.

#180706
Aug 3, 2010 at 1:57pm

Hello maxers,

just in case ; minor update.

Attachments:
  1. FCZY.zip
#180707
Aug 6, 2010 at 6:32pm

path is great, thanks! about time you release this with some proper examples and some well laid out help files maybe?

#180708
Aug 26, 2010 at 9:07am

Hello maxers,

updated my share pages ; nothing very new excepted

- for [november] : i fix it. Now it seems to work ;-)
- and [papa] : verbose mode for “NN total error” ; easier to experiment ;-)

Salutations maxicales.

#180709
Nov 29, 2010 at 4:56pm

Hello maxers,

here [romeo] prototype for building “Treillis de Galois” (Concept Galois Lattice) ; that’s for mac (but sources with).

That’s time for suggestions and bugs, as it’s still in dev, and i have the C code well in mind ;-)

Thanks for feedback.

Attachments:
  1. Romeo.zip
#180710
Nov 30, 2010 at 10:07am

Great! I will try it out and compile for win.
What is it useful for?

#180711
Nov 30, 2010 at 5:09pm

Hello Veqtor,

[romeo] implements, more or less, an “Incremetal Formal Concept Lattice” with the Hass diagram associated (without object/extent) with a short memory ; when maximum number concepts is reach, concepts are killed randomly to let the lattice add new ones …

http://www.info2.uqam.ca/~godin/compint95.pdf

http://en.wikipedia.org/wiki/Formal_concept_analysis

What is it useful for ?

I’ve just done it so i don’t know ;-)

as [charlie] or [yankee], idea is to extract/build a model from a live midi stream … but while [charlie] and [yankee] build models “approximating” the midi stream, [romeo] contains the midinote exactly ; so it is more like [zoulou].

When you bang [romeo] you navigate randomly on the Hass diagram ; with “random” message you get a concept randomly ; “change” message navigate iteratively to fit list length …

More questions do not hesitate …
sorry, hard to explain in english ;-)

#180712
Mar 12, 2012 at 1:34am

Hi guys,

pizza,

how can i control the iteration’s on [yankee]?

Cheers,
JM

#180713
Mar 12, 2012 at 6:36am

Hello,

do you use veqtor’s Windows compile ? or last on (http://cycling74.com/toolbox/fcyzropq/) for mac ?

I changed behavior of my externals to keep them more homogenous ; now an integer message is used to get a response : a list with this integer as length …

#180714
Mar 12, 2012 at 4:56pm

Hi,

I’m using the last version for mac. The idea is to draw on an lCD a classic
representation of the GNG algorithm.

So far i’m trying to understand the data but the it’s not being an easy task, once all the values coming from the left outlet are mapped to midi – 0-127 – and the ones coming from right outlet are mapped to the lcd with that paintrect message…

Do you have any cue on this?

Thanks,
JM

#180715
Mar 12, 2012 at 5:12pm

Hello,

1. you should download the last version from my toolbox page as it seems you are using an old one (i do not use paintrect anymore in the right outlet) ; you can access node’s data now with “dump $1″ message.

2. i implemented (and most of times committed few changes) all the algorithms to deal with MIDI data ; so for [yankee] there is currently no way to get anything else than [0, 127] integers.

3. it should not be very hard to hack it to deal with doubles (sources are on github) ; but to be honest i do not use it so much as i do not have good results with it for generative music.

EDIT :
download is there : http://cycling74.com/toolbox/fcyzropq/
code is there : https://github.com/jeanSapristi/Grenadine/blob/master/pizAlgorithms/pizNeuralGas.c

#180716
Mar 13, 2012 at 10:49am

Many thanks Vanille.

Just one more question, the learn message contains 4 elements, you’r call them argv in the source. Are they the inpuct vector? i mean, 4 elements matching 2 input vectors with 2 dimensions?

cheers,
JM

#180717
Mar 13, 2012 at 12:25pm

Hello,

you can provide two arguments at creation for a [yankee] object :

- the maximum number of nodes (default is 20).
- the size of the vector for each node (aka dimensions) (default is 4).

for example [yankee 40 6] means that your neural lattice can grow until 40 nodes ; and that each node is defined by a vector of 6 values/dimensions … so in pizNeuralGasAdd (PIZNeuralGas *x, long argc, long *argv) the array provided is cut in “6 long” slices to be learned one by one.

“4 elements matching 2 input vectors with 2 dimensions ?”

Yes if it is a 2 dimensions GNG ; but can be ONE vector for a 4 dimensions GNG ;-)

HTH

NOTA BENE : My GNG is implemented from http://www.booru.net/download/MasterThesisProj.pdf ; but i changed few stuff in the process (arcs have NOT age attribute, life/death is slightly different in mine …).

#180718

You must be logged in to reply to this topic.