How to load huge mp3 files into buffer~

Naqoyqatsi's icon

Hi everyone !

I am searching for a way to load huge mp3

Naqoyqatsi's icon

As i can not edit my posting --> small add:

Maybe i do not have enuff RAM but i dont think so. Those files are about 700 - 1 GB when encoded. As i have 2 GB ram (1,5+ GB free while running XP and Max) it should fit in.

Gregory Taylor's icon

As you might have noticed from the refpage, the
import message will allow you to import mp3 files,
but they're uncompressed when imported - it's just
as if you loaded an AIFF file of the same duration,
which is considerably larger. That's the source of
your problem with loading a buffer~ object, I'll wager.

Chris Muir's icon

On Jul 10, 2008, at 9:07 AM, BP wrote:

> Maybe i do not have enuff RAM but i dont think so. Those files are
> about 700 - 1 GB when encoded. As i have 2 GB ram (1,5+ GB free
> while running XP and Max) it should fit in.

I'm not sure that this is your problem, but I believe that compressed
files get decoded into linear files when imported, so they will take
considerably more memory than the file size might indicate (the same
as they would uncompressed).

-C

Chris Muir
cbm@well.com    
http://www.xfade.com

Naqoyqatsi's icon

Thank you Chris Muir and Gregory Taylor for your reply !

Maybe there is away to load only a part of an .wav file.
The part would be 20000 milliseconds plus and minus to the current position in a groove~ object. That way it could prebuffer enuff data to do some scratching and speed changing.

I guess one would need 2 buffers because if i refresh only 1 buffer the playing would stop until the buffer has finsihed loading. Hmm something that can perform partial file reading on the fly.

Thank you all

Peter Castine's icon

Quote: Naqoyqatsi wrote on Thu, 10 July 2008 18:07
----------------------------------------------------
> As i can not edit my posting --> small add:
>
> Maybe i do not have enuff RAM but i dont think so. Those files are about 700 - 1 GB when encoded. As i have 2 GB ram (1,5+ GB free while running XP and Max) it should fit in.
----------------------------------------------------

One hour of audio would be about 600MB in a stereo PCM (linear) 16bit format. But don't forget that MSP internally handles audio as *32* bit floats, so one hour needs 1.2GB storage.

Add to that whatever overhead Max/MSP needs... and that 1.5 GB that Activity Monitor says you have free is not an awful lot. I don't know all the ins and outs of how Max is dealing with Virtual Memory, so the problem *might* be elsewhere. But memory is a prime suspect.

Sorry not to have more concrete help.

Best -- Peter

Salvator's icon

Hi,

In my case, I have a computer with 3.5Gb of RAM, and I can't load a 600Mo (1 hour) of stereo audio...

I suspect there is some limitation inherent to the buffer~ object but also limited RAM allocation with Max.

Anyone ever suceed to load 1 hour of stereo audio ?

Salvator

Naqoyqatsi's icon

Thank you all for your answers. now the case is way clearer. i ll have to find a solution where i only load a part of the big file into the buffer.

i am currently working on such a solution. i ll post a patch if i could solve it.

Thank you all again for your help "

Naq

Naqoyqatsi's icon

Maybe there is a possibility to split a file into parts and load each part after the other.

here is something i made. still got problems.
Copy and then import from clipboard as you surly know.

{
    "patcher" :     {
        "fileversion" : 1,
        "rect" : [ 21.0, 55.0, 828.0, 506.0 ],
        "bglocked" : 0,
        "defrect" : [ 21.0, 55.0, 828.0, 506.0 ],
        "openrect" : [ 0.0, 0.0, 0.0, 0.0 ],
        "openinpresentation" : 0,
        "default_fontsize" : 12.0,
        "default_fontface" : 0,
        "default_fontname" : "Arial",
        "gridonopen" : 0,
        "gridsize" : [ 15.0, 15.0 ],
        "gridsnaponopen" : 0,
        "toolbarvisible" : 1,
        "boxanimatetime" : 200,
        "imprint" : 0,
        "metadata" : [ ],
        "boxes" : [             {
                "box" :                 {
                    "maxclass" : "comment",
                    "text" : "buffer~ test2:nnThe second replace is not working because it would load the entire file. How can i give two values to $1 $2 ? ",
                    "linecount" : 7,
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 0,
                    "id" : "obj-66",
                    "fontsize" : 12.0,
                    "patching_rect" : [ 383.0, 228.0, 156.0, 103.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "comment",
                    "text" : "buffer~ test1:nnThe first replace is working because $2 gives me 0 and $1 gives me the duration divided by 10.nnThat would be my 1/10 part of the whole wav filennI think $2 gives out 0 because it does not belong to this place (wouldnt it be $1 first and then $2?)",
                    "linecount" : 15,
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 0,
                    "id" : "obj-65",
                    "fontsize" : 12.0,
                    "patching_rect" : [ 217.0, 222.0, 150.0, 213.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "print",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 0,
                    "id" : "obj-59",
                    "fontsize" : 12.0,
                    "patching_rect" : [ 490.0, 202.0, 34.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "flonum",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-60",
                    "outlettype" : [ "float", "bang" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 379.0, 150.0, 67.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "/ 10.",
                    "numinlets" : 2,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-61",
                    "outlettype" : [ "float" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 379.0, 125.0, 34.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "flonum",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-62",
                    "presentation_rect" : [ 30.0, 30.0, 50.0, 20.0 ],
                    "outlettype" : [ "float", "bang" ],
                    "fontsize" : 11.595187,
                    "patching_rect" : [ 379.0, 101.0, 84.0, 20.0 ],
                    "triscale" : 0.9
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "message",
                    "text" : "replace Track1.wav $1 $2",
                    "numinlets" : 2,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-63",
                    "outlettype" : [ "" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 379.0, 173.0, 147.0, 18.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "buffer~ test2",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-64",
                    "outlettype" : [ "float", "bang" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 379.0, 202.0, 77.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "button",
                    "numinlets" : 1,
                    "numoutlets" : 1,
                    "id" : "obj-23",
                    "outlettype" : [ "bang" ],
                    "patching_rect" : [ 89.0, 96.0, 20.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "print",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 0,
                    "id" : "obj-17",
                    "fontsize" : 12.0,
                    "patching_rect" : [ 328.0, 196.0, 34.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "number",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-16",
                    "outlettype" : [ "int", "bang" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 147.0, 175.0, 50.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "flonum",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-11",
                    "outlettype" : [ "float", "bang" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 217.0, 144.0, 67.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "/ 10.",
                    "numinlets" : 2,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-10",
                    "outlettype" : [ "float" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 217.0, 119.0, 34.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "message",
                    "text" : "open Track1.wav",
                    "numinlets" : 2,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-21",
                    "outlettype" : [ "" ],
                    "fontsize" : 11.595187,
                    "patching_rect" : [ 89.0, 44.0, 98.0, 18.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "flonum",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-29",
                    "presentation_rect" : [ 15.0, 15.0, 50.0, 20.0 ],
                    "outlettype" : [ "float", "bang" ],
                    "fontsize" : 11.595187,
                    "patching_rect" : [ 217.0, 95.0, 84.0, 20.0 ],
                    "triscale" : 0.9
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "sfinfo~",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 6,
                    "id" : "obj-30",
                    "outlettype" : [ "int", "int", "float", "float", "", "" ],
                    "fontsize" : 11.595187,
                    "patching_rect" : [ 89.0, 69.0, 234.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "message",
                    "text" : "0",
                    "numinlets" : 2,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-9",
                    "outlettype" : [ "" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 95.0, 156.0, 32.5, 18.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "toggle",
                    "numinlets" : 1,
                    "numoutlets" : 1,
                    "id" : "obj-7",
                    "outlettype" : [ "int" ],
                    "patching_rect" : [ 50.0, 131.0, 20.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "sig~ 1",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-6",
                    "outlettype" : [ "signal" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 50.0, 154.0, 43.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "ezdac~",
                    "numinlets" : 2,
                    "numoutlets" : 0,
                    "id" : "obj-5",
                    "patching_rect" : [ 42.0, 234.0, 45.0, 45.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "groove~ test",
                    "numinlets" : 3,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-4",
                    "outlettype" : [ "signal", "signal" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 50.0, 205.0, 77.0, 20.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "message",
                    "text" : "replace Track1.wav $2 $1",
                    "numinlets" : 2,
                    "fontname" : "Arial",
                    "numoutlets" : 1,
                    "id" : "obj-3",
                    "outlettype" : [ "" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 217.0, 167.0, 147.0, 18.0 ]
                }

            }
,             {
                "box" :                 {
                    "maxclass" : "newobj",
                    "text" : "buffer~ test",
                    "numinlets" : 1,
                    "fontname" : "Arial",
                    "numoutlets" : 2,
                    "id" : "obj-1",
                    "outlettype" : [ "float", "bang" ],
                    "fontsize" : 12.0,
                    "patching_rect" : [ 217.0, 196.0, 71.0, 20.0 ]
                }

            }
],
        "lines" : [             {
                "patchline" :                 {
                    "source" : [ "obj-3", 0 ],
                    "destination" : [ "obj-1", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-9", 0 ],
                    "destination" : [ "obj-4", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-6", 0 ],
                    "destination" : [ "obj-4", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-7", 0 ],
                    "destination" : [ "obj-6", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-4", 0 ],
                    "destination" : [ "obj-5", 1 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-4", 0 ],
                    "destination" : [ "obj-5", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-21", 0 ],
                    "destination" : [ "obj-30", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ 98.5, 67.0, 98.5, 67.0 ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-30", 3 ],
                    "destination" : [ "obj-29", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-11", 0 ],
                    "destination" : [ "obj-3", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-16", 0 ],
                    "destination" : [ "obj-4", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-3", 0 ],
                    "destination" : [ "obj-17", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-30", 0 ],
                    "destination" : [ "obj-23", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-23", 0 ],
                    "destination" : [ "obj-9", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-63", 0 ],
                    "destination" : [ "obj-59", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-60", 0 ],
                    "destination" : [ "obj-63", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-30", 3 ],
                    "destination" : [ "obj-62", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-63", 0 ],
                    "destination" : [ "obj-64", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-29", 0 ],
                    "destination" : [ "obj-10", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-10", 0 ],
                    "destination" : [ "obj-11", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-62", 0 ],
                    "destination" : [ "obj-61", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
,             {
                "patchline" :                 {
                    "source" : [ "obj-61", 0 ],
                    "destination" : [ "obj-60", 0 ],
                    "hidden" : 0,
                    "midpoints" : [ ]
                }

            }
]
    }

}

Olivier Pasquet's icon

Hello,

I have worked on a mp3 player with tags in Java. But I have stopped
working on it because I was always getting error messages because of
memory problems with large files.

Is MXJ limited in memory ? Even with :
max.jvm.option -Xmx256m
in the max.java.config.txt file.

O..////

Naqoyqatsi's icon

Is there any chance to just load a region from a sound file into buffer.
I know that one can do it for e.g. like that: replace Track01.wav 0 20000. unfortunatly i did not find a way to make that somehow dynamic.

Any ideas about that ?

Thank you in advance !

Naq