Storing Long Lists

Jun 9, 2007 at 9:22pm

Storing Long Lists

I was wondering if anyone has found a great way to
read in and cycle through long lists of numbers. I’m
talking around 10,000 values. I’ve tried stringing
together several coll objects, but as these can only
hold a couple hundred lines, it is quite inefficient.

Anyone have any ideas?

Thanks,

Joe

Got a little couch potato?
Check out fun summer activities for kids.

http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz

#32388
Jun 9, 2007 at 9:31pm

Quoting Joe S :

> I was wondering if anyone has found a great way to
> read in and cycle through long lists of numbers. I’m
> talking around 10,000 values. I’ve tried stringing
> together several coll objects, but as these can only
> hold a couple hundred lines, it is quite inefficient.
>
> Anyone have any ideas?

This is exactly the kind of task that I think is much
better handled in one of the “auxiliary” languages available
in max/msp, like java or javascript, or… Trying to
string together a bunch of [colls] is strangely complicated.

Because I know it well, I would use [rtcmix~] (UB with
new features almost done! — waiting until NIME finished)
with this script:

tenthousandvals { …… }

outtime = 0
for (i = 0; i < 10000; i = i+1) {
MAXMESSAGE(outtime, tenthousandvals[i])
outtime = outtime + 0.5
}

This would sequentially send out every value in the loaded array
tenthousandvals[], one every 0.5 seconds. You could also set it up
to bang the [rtcmix~] object to send out a val, and load the array
from max/msp if necessary.

I’m sure there are a number of other good ways to accomplish this.

brad

http://music.columbia.edu/~brad

#106435
Jun 10, 2007 at 12:30am

10,000 values sounds like a lot, but 10,000 samples or 10,000 pixels
is not much at all and there abjects around to deal with those; why
don’t you try jitter matrices, or if you don’t own jitter, why not
use a msp soundbuffer ?

ciao,

Joost.

On Jun 9, 2007, at 11:22 , Joe S wrote:

> I was wondering if anyone has found a great way to
> read in and cycle through long lists of numbers. I’m
> talking around 10,000 values. I’ve tried stringing
> together several coll objects, but as these can only
> hold a couple hundred lines, it is quite inefficient.
>
> Anyone have any ideas?
>
> Thanks,
>
> Joe
>
>
>
> ______________
> Got a little couch potato?
> Check out fun summer activities for kids.
> http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities
> +for+kids&cs=bz

——————————————-

Joost Rekveld
———– http://www.lumen.nu/rekveld

——————————————-

“The question of whether a computer can think
is no more interesting than the question
of whether a submarine can swim.”
(E. W. Dijkstra)

——————————————-

#106436
Jun 10, 2007 at 12:35am

and, euhm, what’s wrong with good old ‘table’ ?
or do you want floats ?

ciao,

Joost.

On Jun 10, 2007, at 2:30 , Joost Rekveld wrote:

> 10,000 values sounds like a lot, but 10,000 samples or 10,000
> pixels is not much at all and there abjects around to deal with
> those; why don’t you try jitter matrices, or if you don’t own
> jitter, why not use a msp soundbuffer ?
>
> ciao,
>
> Joost.
>
> On Jun 9, 2007, at 11:22 , Joe S wrote:
>
>> I was wondering if anyone has found a great way to
>> read in and cycle through long lists of numbers. I’m
>> talking around 10,000 values. I’ve tried stringing
>> together several coll objects, but as these can only
>> hold a couple hundred lines, it is quite inefficient.
>>
>> Anyone have any ideas?
>>
>> Thanks,
>>
>> Joe
>>
>>
>>
>> _______________
>> Got a little couch potato?
>> Check out fun summer activities for kids.
>> http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities
>> +for+kids&cs=bz
>
>
>
>
>
> ——————————————-
>
> Joost Rekveld
> ———– http://www.lumen.nu/rekveld
>
> ——————————————-
>
> “The question of whether a computer can think
> is no more interesting than the question
> of whether a submarine can swim.”
> (E. W. Dijkstra)
>
> ——————————————-
>
>

#106437
Jun 10, 2007 at 12:50am

for floats you could use Larray from the Lobjects..

sorry for the noise spread over 3 posts..

ciao,

Joost.

On Jun 10, 2007, at 2:35 , Joost Rekveld wrote:

> and, euhm, what’s wrong with good old ‘table’ ?
> or do you want floats ?
>
> ciao,
>
> Joost.
>
> On Jun 10, 2007, at 2:30 , Joost Rekveld wrote:
>
>> 10,000 values sounds like a lot, but 10,000 samples or 10,000
>> pixels is not much at all and there abjects around to deal with
>> those; why don’t you try jitter matrices, or if you don’t own
>> jitter, why not use a msp soundbuffer ?
>>
>> ciao,
>>
>> Joost.
>>
>> On Jun 9, 2007, at 11:22 , Joe S wrote:
>>
>>> I was wondering if anyone has found a great way to
>>> read in and cycle through long lists of numbers. I’m
>>> talking around 10,000 values. I’ve tried stringing
>>> together several coll objects, but as these can only
>>> hold a couple hundred lines, it is quite inefficient.
>>>
>>> Anyone have any ideas?
>>>
>>> Thanks,
>>>
>>> Joe
>>>
>>>
>>>
>>> ________________
>>> Got a little couch potato?
>>> Check out fun summer activities for kids.
>>> http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities
>>> +for+kids&cs=bz
>>
>>
>>
>>
>>
>> ——————————————-
>>
>> Joost Rekveld
>> ———– http://www.lumen.nu/rekveld
>>
>> ——————————————-
>>
>> “The question of whether a computer can think
>> is no more interesting than the question
>> of whether a submarine can swim.”
>> (E. W. Dijkstra)
>>
>> ——————————————-
>>
>>
>

#106438
Jun 12, 2007 at 6:09am

Joe S schrieb:
> I was wondering if anyone has found a great way to
> read in and cycle through long lists of numbers. I’m
> talking around 10,000 values. I’ve tried stringing
> together several coll objects, but as these can only
> hold a couple hundred lines, it is quite inefficient.

I have used colls with more than 10000 lines, what’s the problem?

Another approach is a buffer~ and poke~…

Stefan


Stefan Tiedje————x——-
–_____———–|————–
–(_|_ —-|—–|—–()——-
– _|_)—-|—–()————–
———-()——–www.ccmix.com

#106439
Jun 12, 2007 at 7:46am

I would be carefully when using Larray, cos it will only output lists up to 256 members, like most of the max-objects!
Thats why I decided to learn Java, which really works great!!

Gerald

#106440
Jun 12, 2007 at 2:59pm

#106441
Jun 12, 2007 at 3:13pm

On 12 juin 07, at 16:59, Joost Rekveld wrote:

> but any java object would have the same limitation, no ?

Nope, the limitation is in the C objects, not in what’s going thru
the connection (there’s also some memory limitations). As an example,
you can have a look to my ejies which include loads of java externals
to deal with list. The standart [mxj list.*] package is worth the try
too.

ej

#106442
Jun 12, 2007 at 3:50pm

aha,
I learned something there…

thanks,

Joost.

On Jun 12, 2007, at 5:13 , Emmanuel Jourdan wrote:

> On 12 juin 07, at 16:59, Joost Rekveld wrote:
>
>> but any java object would have the same limitation, no ?
>
> Nope, the limitation is in the C objects, not in what’s going thru
> the connection (there’s also some memory limitations). As an
> example, you can

——————————————-

Joost Rekveld
———– http://www.lumen.nu/rekveld

——————————————-

“The mystery of the world is the visible, not the invisible”
(Oscar Wilde)

——————————————-

#106443
Jun 13, 2007 at 8:24am

Yepp, thats exactly the thing! I hope in some following Max-Versions they will extend or blow away this limit. I think it comes from the very beginnings of Max, when computing ressources were limited.

For myself I decided to do all the list processing in Java, because these things are much easier here.
Just be careful when dropping a huge list out of a java object, cos Max will crash very easy – or its crashing on my machine at least!

Gerald

#106444
Jun 13, 2007 at 9:14am

On 13 juin 07, at 10:24, Gerald wrote:

> Yepp, thats exactly the thing! I hope in some following Max-
> Versions they will extend or blow away this limit. I think it comes
> from the very beginnings of Max, when computing ressources were
> limited.
>
> For myself I decided to do all the list processing in Java, because
> these things are much easier here.
> Just be careful when dropping a huge list out of a java object, cos
> Max will crash very easy – or its crashing on my machine at least!

I haven’t seen anything like that with 4.5.6 and higher. If you have
some example of crash, we can look at it though.

ej

#106445
Jun 13, 2007 at 6:25pm

A 256-item list requires 2kB storage (on Mac, Windows packs that data into 1.5 kB). Many objects processing that data need to make a temporary copy, so in a typical processing chain you might get up to around 30kB or so.

A 10,000-item list would require 80kB, and in processing that could easily blow up to into the MB ballpark. Not the end of the world on modern hardware, but it’s worth asking the question “is that necessary?”

Lists can contain any mixture of floats, ints, and symbols. It’s that flexibility in the data structure that inflates memory requirements. If you know you’re working with an array of floats, or of ints (or even an array of symbols), you can work much more efficiently with MSP buffers or Jitter matrices or Java/JScript/rtCMix/Perl/fillintheblank.

The 256-item limit is not a characteristic of the C language, nor of Max. It’s a *convention* that is adhered to by hundreds of list-processing objects. Lifting the convention will mean that all of those objects will need to be updated, both the bundled objects and all those wonderful 3rd party offerings. That won’t happen in one night, so be careful what you wish for in Max 5.

– P.

#106446
Jun 13, 2007 at 7:54pm

>>t’s a *convention* that is adhered to by hundreds of list-processing objects. < <

Thats exactly what I wanted to say, but I couldnt find this clear words! Of course I’m not expecting, that all the objects will be rewritten, but probably there will be a possibility to determine the type of list in the future, not to have only Atoms, but to be able to use a “real” array of floats, double(!) etc. within max.

The crash I was talkin ’bout happened always, when I sent a (too) long list to “print”!

Cherrs

Gerald

#106447
Jun 14, 2007 at 6:37am

Jitter permits this – think of a 1D, 1 plane matrix as a simple
array. You have char, long, float and double to choose from.

jb

Am 13.06.2007 um 21:54 schrieb Gerald:

> to be able to use a “real” array of floats, double(!) etc. within max.

#106448
Jun 17, 2007 at 9:10pm

Gerald schrieb:
> The crash I was talkin ’bout happened always, when I sent a (too)
> long list to “print”!

Usually too long lists are just truncated, they must not crash Max. You
should send a patch showing this behaviour to the support if it is
reproducible…

Stefan


Stefan Tiedje————x——-
–_____———–|————–
–(_|_ —-|—–|—–()——-
– _|_)—-|—–()————–
———-()——–www.ccmix.com

#106449

You must be logged in to reply to this topic.