std::vector conflicting memory alloc?

Thijs Koerselman's icon

I've written an external using C++ which makes use of std::vector. When I
run it without the debugger attached everything seems to be fine and stable.
When I run the debugger I get an exception on vector functions like resize()
and clear().

It points to a failing assertion in the msvcr71d.dll

For example if I call "x->segList.clear();" on my vector, it triggers the
following function calls in the STD library:

void clear()
{ // erase all
_Tidy();
}

*************************

void _Tidy()
{ // free all storage
if (_Myfirst != 0)
{ // something to free, destroy and deallocate it
_Destroy(_Myfirst, _Mylast);
this->_Alval.deallocate(_Myfirst, _Myend - _Myfirst);
}
_Myfirst = 0, _Mylast = 0, _Myend = 0;
}

*************************

void deallocate(pointer _Ptr, size_type)
{ // deallocate object at _Ptr, ignore size
operator delete(_Ptr);
}

************************
void operator delete( void *pUserData)
{
_CrtMemBlockHeader * pHead;

RTCCALLBACK(_RTC_Free_hook, (pUserData, 0));

if (pUserData == NULL)
return;

_mlock(_HEAP_LOCK); /* block other threads */
__TRY

/* get a pointer to memory block header */
pHead = pHdr(pUserData);

/* verify block type */
_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));

_free_dbg( pUserData, pHead->nBlockUse );

__FINALLY
_munlock(_HEAP_LOCK); /* release other threads */
__END_TRY_FINALLY

return;
}

And that's where it fails, the
"_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));"

The thing is, like I said, all runs fine if I don't run the debugger. Is
anyone using the std:vector<> with success or has a clue what might be going
wrong?

Cheers,
Thijs

Graham Wakefield's icon

Been using various stl containers including vector in externals for a
while on Mac, not yet on Windows. Are you sure you are always
accessing it from the same thread? Stl is not thread safe. Just an
idea.

On Aug 20, 2007, at 4:26 AM, Thijs Koerselman wrote:

> I've written an external using C++ which makes use of std::vector.
> When I run it without the debugger attached everything seems to be
> fine and stable. When I run the debugger I get an exception on
> vector functions like resize() and clear().
>
> It points to a failing assertion in the msvcr71d.dll
>
> For example if I call "x->segList.clear();" on my vector, it
> triggers the following function calls in the STD library:
>
> void clear()
> { // erase all
> _Tidy();
> }
>
> *************************
>
> void _Tidy()
> { // free all storage
> if (_Myfirst != 0)
> { // something to free, destroy and deallocate it
> _Destroy(_Myfirst, _Mylast);
> this->_Alval.deallocate(_Myfirst, _Myend - _Myfirst);
> }
> _Myfirst = 0, _Mylast = 0, _Myend = 0;
> }
>
> *************************
>
> void deallocate(pointer _Ptr, size_type)
> { // deallocate object at _Ptr, ignore size
> operator delete(_Ptr);
> }
>
> ************************
> void operator delete( void *pUserData)
> {
> _CrtMemBlockHeader * pHead;
>
> RTCCALLBACK(_RTC_Free_hook, (pUserData, 0));
>
> if (pUserData == NULL)
> return;
>
> _mlock(_HEAP_LOCK); /* block other threads */
> __TRY
>
> /* get a pointer to memory block header */
> pHead = pHdr(pUserData);
>
> /* verify block type */
> _ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));
>
> _free_dbg( pUserData, pHead->nBlockUse );
>
> __FINALLY
> _munlock(_HEAP_LOCK); /* release other threads */
> __END_TRY_FINALLY
>
> return;
> }
>
>
> And that's where it fails, the "_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead-
> >nBlockUse));"
>
> The thing is, like I said, all runs fine if I don't run the
> debugger. Is anyone using the std:vector<> with success or has a
> clue what might be going wrong?
>
>
> Cheers,
> Thijs
>
>
> --
> http://www.nano-soundworks.com

grrr waaa
www.grahamwakefield.net

Thijs Koerselman's icon

On 8/20/07, Graham Wakefield wrote:
>
> Been using various stl containers including vector in externals for a
> while on Mac, not yet on Windows. Are you sure you are always accessing it
> from the same thread? Stl is not thread safe. Just an idea.
>
>
Hi Graham,

That's good news. I don't do anything with threads at the moment. I did some
more investigating and realize I must be doing something wrong regarding the
allocation of the vector.

If I allocate a local vector I can do whatever, it works fine. If I try to
do the same on the vector that is a member of my class, it doesn't work.
What am I doing wrong? (code below)

Cheers,
Thijs

class Foo
{
public:
t_jit_object ob;
static void *_class;

std::vector mSegList;

static void *alloc(void);
static void free(Foo *x);
static void callProcess(Foo *x);
};

void* Foo::alloc(void)
{
int i;
Foo *x;
t_atom atom;

if (x = (Foo *)jit_object_alloc(Foo::_class))
{
// works fine
std::vector segList;
segList.resize(512);

// throws exception
x->mSegList.resize(512);
}
return x;
}

// function triggered by a max message
void Foo::callProcess(Foo *x)
{
// works fine
std::vector segList;
segList.clear();

// throws exception
x->mSegList.clear();
}

Wesley Smith's icon

You're using a C allocator not a C++ constructor. It doesn't build
any of the class objects, just allocates memory. You can call the
constructor on allocated memory by using the placement new operator:

void *memory = alloc(Foo::_class);
Foo *f = new(memory) Foo;

but this will probably mung the t_object stuff that alloc takes care
of. There may be a way around this, but definitely you can make
mSegList a pointer and allocate it after alloc(Foo::_class);

wes

On 8/20/07, Thijs Koerselman wrote:
>
> On 8/20/07, Graham Wakefield wrote:
> >
> >
> > Been using various stl containers including vector in externals for a
> while on Mac, not yet on Windows. Are you sure you are always accessing it
> from the same thread? Stl is not thread safe. Just an idea.
> >
>
> Hi Graham,
>
> That's good news. I don't do anything with threads at the moment. I did some
> more investigating and realize I must be doing something wrong regarding the
> allocation of the vector.
>
> If I allocate a local vector I can do whatever, it works fine. If I try to
> do the same on the vector that is a member of my class, it doesn't work.
> What am I doing wrong? (code below)
>
> Cheers,
> Thijs
>
>
> class Foo
> {
> public:
> t_jit_object ob;
> static void *_class;
>
> std::vector mSegList;
>
> static void *alloc(void);
> static void free(Foo *x);
> static void callProcess(Foo *x);
> };
>
> void* Foo::alloc(void)
> {
> int i;
> Foo *x;
> t_atom atom;
>
> if (x = (Foo *)jit_object_alloc(Foo::_class))
> {
> // works fine
> std::vector segList;
> segList.resize(512);
>
> // throws exception
> x->mSegList.resize(512);
> }
> return x;
> }
>
> // function triggered by a max message
> void Foo::callProcess(Foo *x)
> {
> // works fine
> std::vector segList;
> segList.clear();
>
> // throws exception
> x->mSegList.clear();
>
> }
>
> --
> http://www.nano-soundworks.com
>
>
>

Thijs Koerselman's icon

Thanks Wes! I think I prefer the pointer approach. Although I understand how
to get around this issue now I'm still a bit unsure if I understand the
nature of the problem.

If I understand this correctly the line "std::vector mSegList;"
inside a c++ function both allocates memory AND calls the vector
constructor.
The same line of code placed in my class declaration only specifies mSegList
as a member of type std::vector<> and allocates sufficient memory for it.
Because my Foo class is allocated by the Max/C allocation (instead of
calling the Foo() constructor like you normally would on a C++ class) the
constructor of mSegList never gets called and the std::vector object for
mSegList isn't created. Am I right about this?

I'm still curious to what Grahams approach is to constructing vectors in his
C++ code...?

All the best,
Thijs

On 8/20/07, Wesley Smith wrote:
>
> You're using a C allocator not a C++ constructor. It doesn't build
> any of the class objects, just allocates memory. You can call the
> constructor on allocated memory by using the placement new operator:
>
> void *memory = alloc(Foo::_class);
> Foo *f = new(memory) Foo;
>
> http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10
>
>
> but this will probably mung the t_object stuff that alloc takes care
> of. There may be a way around this, but definitely you can make
> mSegList a pointer and allocate it after alloc(Foo::_class);
>
> wes
>
>
> On 8/20/07, Thijs Koerselman wrote:
> >
> > On 8/20/07, Graham Wakefield wrote:
> > >
> > >
> > > Been using various stl containers including vector in externals for a
> > while on Mac, not yet on Windows. Are you sure you are always accessing
> it
> > from the same thread? Stl is not thread safe. Just an idea.
> > >
> >
> > Hi Graham,
> >
> > That's good news. I don't do anything with threads at the moment. I did
> some
> > more investigating and realize I must be doing something wrong regarding
> the
> > allocation of the vector.
> >
> > If I allocate a local vector I can do whatever, it works fine. If I try
> to
> > do the same on the vector that is a member of my class, it doesn't work.
> > What am I doing wrong? (code below)
> >
> > Cheers,
> > Thijs
> >
> >
> > class Foo
> > {
> > public:
> > t_jit_object ob;
> > static void *_class;
> >
> > std::vector mSegList;
> >
> > static void *alloc(void);
> > static void free(Foo *x);
> > static void callProcess(Foo *x);
> > };
> >
> > void* Foo::alloc(void)
> > {
> > int i;
> > Foo *x;
> > t_atom atom;
> >
> > if (x = (Foo *)jit_object_alloc(Foo::_class))
> > {
> > // works fine
> > std::vector segList;
> > segList.resize(512);
> >
> > // throws exception
> > x->mSegList.resize(512);
> > }
> > return x;
> > }
> >
> > // function triggered by a max message
> > void Foo::callProcess(Foo *x)
> > {
> > // works fine
> > std::vector segList;
> > segList.clear();
> >
> > // throws exception
> > x->mSegList.clear();
> >
> > }
> >
> > --
> > http://www.nano-soundworks.com
> >
> >
> >
>

Thijs Koerselman's icon

On 8/20/07, Thijs Koerselman wrote:
>
>
> The same line of code placed in my class declaration only specifies
> mSegList as a member of type std::vector<> and allocates sufficient memory
> for it.

I meant to say:
The same line of code placed in my class declaration only specifies mSegList
as a member of type std::vector<> and jit_object_alloc() allocates
sufficient memory for it.

Wesley Smith's icon

> If I understand this correctly the line "std::vector mSegList;"
> inside a c++ function both allocates memory AND calls the vector
> constructor.
> The same line of code placed in my class declaration only specifies mSegList
> as a member of type std::vector<> and allocates sufficient memory for it.
> Because my Foo class is allocated by the Max/C allocation (instead of
> calling the Foo() constructor like you normally would on a C++ class) the
> constructor of mSegList never gets called and the std::vector object for
> mSegList isn't created. Am I right about this?

yes
wes

Graham Wakefield's icon

I'm doing some template hackery to make max externals feel like C++
classes, with constructors and all so that member variables
initialize properly.

It's a bit rough around the edges, but here's the jist. When (if) I
clean this up nicely and document it, I may post it online as a very
very lightweight alternative to flext for simple C++ max externals.
Basically I'm using overloaded new and delete operators to allow Max
to manage memory while still calling my own constructors. I'm also
doing a couple of other things that make the user code look much
simpler (and I'm a fan of that).

HTH

My test class external:

#include
#include "max.cpp.h"

class TestExternal : public MaxCpp
{
public:

    TestExternal(t_symbol * s, long ac, t_atom * av);
    ~TestExternal();

protected:

    std::vector    mIntVector;

};

TestExternal::TestExternal(t_symbol *s, long ac, t_atom *av)
: MaxCpp()
{        
    // do stuff
}

TestExternal :: ~TestExternal()
{
    // undo stuff
}

// called when library is loaded
extern "C" int main(void)
{    
    t_class    * c = MaxVessel::maxMakeClass("testexternal");

    // register this as a class for Max UI & store static reference
    class_register(CLASS_BOX, c);
}

And here's the magic, in max.cpp.h:

#ifndef MAXMSP_CPP_H
#define MAXMSP_CPP_H

#include

#include "ext.h"
#include "ext_common.h"
#include "commonsyms.h"
#include "ext_obex.h"
#include "ext_path.h"
#include "z_dsp.h"
#include "buffer.h"    

// abstraction to provide inlet & outlet signals in a safer way
typedef std::vector signalArray;

template
class MaxCpp
{

protected:
    // for Max external-loading & Obex:
    t_pxobject            maxOb;        // Max 'instance' pointer
    void *                maxObex;    // Max Obex container
    static void *        maxClass;    // Max 'class' pointer
    static char *        maxClassName; // our own reference

private:
    static void *        maxAlloc(t_symbol *s, long argc, t_atom *argv);
    static void            maxFree(T *x);
    static void *        operator new(size_t);
    static void            operator delete(void *);

public:    

    // called from 'main' to create class ptr
    static t_class *    maxMakeClass(char * name);    // use this for Max objects

    // use normal C++ constructors & destructors:
    MaxCpp();
    ~MaxCpp();    

    // 'dump' obex outlet:
    t_object *            maxObexOutlet;
};

// singleton class object
template
void * MaxCpp::maxClass = 0;
template
char * MaxCpp::maxClassName;

template
inline t_class * MaxCpp::maxMakeClass(char * name)
{
    t_class * c = class_new(
        name,
        (method)T::maxAlloc,
        (method)T::maxFree,
        (short)sizeof(T),
        0L,
        A_GIMME,
    0);

    maxClassName = name;

    // initialize the common symbols, since we want to use them
    common_symbols_init();

    // register the byte offset of obex with the class
    class_obexoffset_set(c, calcoffset(T, maxObex));

    // add methods for dumpout and quickref    
    class_addmethod(c, (method)object_obex_dumpout,    "dumpout",        A_CANT, 0);
    class_addmethod(c, (method)object_obex_quickref,"quickref",        A_CANT,
0);

    // store static ref
    MaxCpp::maxClass = c;

    return c;
}

template
inline void * MaxCpp :: maxAlloc(t_symbol * sym, long ac, t_atom *
av)
{
    return new T(sym, ac, av);
}

template
inline void MaxCpp :: maxFree(T * x)
{
    delete x;
}

template
inline void * MaxCpp :: operator new(size_t t)
{
    return object_alloc((t_class *)MaxCpp::maxClass);
}

template
inline void MaxCpp :: operator delete(void *)
{
    // nothing to do - Max will manage the memory
}

template
inline MaxCpp::MaxCpp()
{        
    // add a generic outlet
    maxObexOutlet = (t_object *)outlet_new(this, 0L);
    object_obex_store(this, _sym_dumpout, maxObexOutlet); // also set it
as dumpout        
}

template
inline MaxCpp :: ~MaxCpp()
{
}

#endif

On Aug 20, 2007, at 11:15 AM, Thijs Koerselman wrote:

> Thanks Wes! I think I prefer the pointer approach. Although I
> understand how to get around this issue now I'm still a bit unsure
> if I understand the nature of the problem.
>
> If I understand this correctly the line "std::vector
> mSegList;" inside a c++ function both allocates memory AND calls
> the vector constructor.
> The same line of code placed in my class declaration only specifies
> mSegList as a member of type std::vector<> and allocates sufficient
> memory for it. Because my Foo class is allocated by the Max/C
> allocation (instead of calling the Foo() constructor like you
> normally would on a C++ class) the constructor of mSegList never
> gets called and the std::vector object for mSegList isn't created.
> Am I right about this?
>
> I'm still curious to what Grahams approach is to constructing
> vectors in his C++ code...?
>
> All the best,
> Thijs
>
>
> --
> http://www.nano-soundworks.com
>
> On 8/20/07, Wesley Smith wrote: You're
> using a C allocator not a C++ constructor. It doesn't build
> any of the class objects, just allocates memory. You can call the
> constructor on allocated memory by using the placement new operator:
>
> void *memory = alloc(Foo::_class);
> Foo *f = new(memory) Foo;
>
> http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10
>
>
> but this will probably mung the t_object stuff that alloc takes care
> of. There may be a way around this, but definitely you can make
> mSegList a pointer and allocate it after alloc(Foo::_class);
>
> wes
>
>
> On 8/20/07, Thijs Koerselman < thijskoerselman@gmail.com> wrote:
> >
> > On 8/20/07, Graham Wakefield wrote:
> > >
> > >
> > > Been using various stl containers including vector in externals
> for a
> > while on Mac, not yet on Windows. Are you sure you are always
> accessing it
> > from the same thread? Stl is not thread safe. Just an idea.
> > >
> >
> > Hi Graham,
> >
> > That's good news. I don't do anything with threads at the moment.
> I did some
> > more investigating and realize I must be doing something wrong
> regarding the
> > allocation of the vector.
> >
> > If I allocate a local vector I can do whatever, it works fine.
> If I try to
> > do the same on the vector that is a member of my class, it
> doesn't work.
> > What am I doing wrong? (code below)
> >
> > Cheers,
> > Thijs
> >
> >
> > class Foo
> > {
> > public:
> > t_jit_object ob;
> > static void *_class;
> >
> > std::vector mSegList;
> >
> > static void *alloc(void);
> > static void free(Foo *x);
> > static void callProcess(Foo *x);
> > };
> >
> > void* Foo::alloc(void)
> > {
> > int i;
> > Foo *x;
> > t_atom atom;
> >
> > if (x = (Foo *)jit_object_alloc(Foo::_class))
> > {
> > // works fine
> > std::vector segList;
> > segList.resize(512);
> >
> > // throws exception
> > x->mSegList.resize(512);
> > }
> > return x;
> > }
> >
> > // function triggered by a max message
> > void Foo::callProcess(Foo *x)
> > {
> > // works fine
> > std::vector segList;
> > segList.clear();
> >
> > // throws exception
> > x->mSegList.clear ();
> >
> > }
> >
> > --
> > http://www.nano-soundworks.com
> >
> >
> >
>

grrr waaa
www.grahamwakefield.net

Thijs Koerselman's icon

On 8/22/07, Graham Wakefield wrote:
>
> I'm doing some template hackery to make max externals feel like C++
> classes, with constructors and all so that member variables
> initialize properly.
>
> It's a bit rough around the edges, but here's the jist. When (if) I
> clean this up nicely and document it, I may post it online as a very
> very lightweight alternative to flext for simple C++ max externals.
> Basically I'm using overloaded new and delete operators to allow Max
> to manage memory while still calling my own constructors. I'm also
> doing a couple of other things that make the user code look much
> simpler (and I'm a fan of that).

Thanks a lot Graham! That's beautiful. I'll have to adjust it to make it
compatible with the Jitter externals I'm writing, but I get the concept. I
started using pointers to my member classes, but its polluting my code. Your
solution is very clean and lightweight.

Cheers,
Thijs

Graham Wakefield's icon

Hi all,

Thought I'd share this, since it's been useful for me already. I
improved the C++ template I've been using to build MaxMSP objects
somewhat - now includes wrapping of max messages to instance methods,
MSP callbacks and text editors for example. The basic idea was to
make a very simple bridge - just a single .h file - to make external
development a bit simpler for C++ oriented developers. I threw
together a quick page of documentation that should give the general
idea:

www.mat.ucsb.edu/~wakefield/maxcpp.htm

If anyone makes use of this template, please let me know, and
especially if you spot something that could be improved or added to
it! AFAIK it should compile fine for Windows - but I haven't tested
it yet so do let me know if you have trouble.

Best,

Graham

PS. A much more comprehensive and feature packed C++ alternative is
to use the Flext library (http://grrrr.org/ext/flext/), which also
builds for PD etc. In contrast to Flext, I tried to minimize the use
of macros by employing templates instead (partly as a learning
exercise).

On Aug 22, 2007, at 3:22 AM, Thijs Koerselman wrote:

>
> On 8/22/07, Graham Wakefield wrote:
> I'm doing some template hackery to make max externals feel like C++
> classes, with constructors and all so that member variables
> initialize properly.
>
> It's a bit rough around the edges, but here's the jist. When (if) I
> clean this up nicely and document it, I may post it online as a very
> very lightweight alternative to flext for simple C++ max externals.
> Basically I'm using overloaded new and delete operators to allow Max
> to manage memory while still calling my own constructors. I'm also
> doing a couple of other things that make the user code look much
> simpler (and I'm a fan of that).
>
> Thanks a lot Graham! That's beautiful. I'll have to adjust it to
> make it compatible with the Jitter externals I'm writing, but I get
> the concept. I started using pointers to my member classes, but its
> polluting my code. Your solution is very clean and lightweight.
>
> Cheers,
> Thijs
>
>
>

grrr waaa
www.grahamwakefield.net

Rob Ramirez's icon

sounds great, and i will try it out as soon as i get time.
i wonder if you notice any differences in compile time when you are using all those templates.

thanks for sharing.

Wesley Smith's icon

It's not that significant a difference. Anyway, it's best to let the
compiler do more work for you if you can.

wes

On 10/22/07, Robert Ramirez wrote:
>
> sounds great, and i will try it out as soon as i get time.
> i wonder if you notice any differences in compile time when you are using all those templates.
>
> thanks for sharing.
>

Graham Wakefield's icon

Didn't notice it... However the nice thing is that the runtime
performance should be entirely unaffected.

On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:

>
> sounds great, and i will try it out as soon as i get time.
> i wonder if you notice any differences in compile time when you are
> using all those templates.
>
> thanks for sharing.

grrr waaa
www.grahamwakefield.net

bbnickell's icon

Very cool templated-code; well, there is no run-time hit since there is no
polymorphism in the code (from what I could see anyway) and the compiler is
doing all of the work here :)

Will play around with this, thanks!

Brandon

On 10/22/07, Graham Wakefield wrote:
>
> Didn't notice it... However the nice thing is that the runtime
> performance should be entirely unaffected.
>
> On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:
>
>
> sounds great, and i will try it out as soon as i get time.
> i wonder if you notice any differences in compile time when you are using
> all those templates.
>
> thanks for sharing.
>
>
> grrr waaa
> www.grahamwakefield.net
>
>
>
>
>
>
>

Graham Wakefield's icon

Let me know if you have any helpful additions/corrections!

On Oct 25, 2007, at 10:10 PM, Brandon Nickell wrote:

> Very cool templated-code; well, there is no run-time hit since
> there is no polymorphism in the code (from what I could see anyway)
> and the compiler is doing all of the work here :)
>
> Will play around with this, thanks!
>
> Brandon
>
> On 10/22/07, Graham Wakefield wrote:
> Didn't notice it... However the nice thing is that the runtime
> performance should be entirely unaffected.
>
> On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:
>
>>
>> sounds great, and i will try it out as soon as i get time.
>> i wonder if you notice any differences in compile time when you
>> are using all those templates.
>>
>> thanks for sharing.
>
> grrr waaa
> www.grahamwakefield.net
>
>
>
>
>
>
>

grrr waaa
www.grahamwakefield.net