std::vector conflicting memory alloc?

Aug 20, 2007 at 11:26am

std::vector conflicting memory alloc?

I’ve written an external using C++ which makes use of std::vector. When I
run it without the debugger attached everything seems to be fine and stable.
When I run the debugger I get an exception on vector functions like resize()
and clear().

It points to a failing assertion in the msvcr71d.dll

For example if I call “x->segList.clear();” on my vector, it triggers the
following function calls in the STD library:

void clear()
{ // erase all
_Tidy();
}

*************************

void _Tidy()
{ // free all storage
if (_Myfirst != 0)
{ // something to free, destroy and deallocate it
_Destroy(_Myfirst, _Mylast);
this->_Alval.deallocate(_Myfirst, _Myend – _Myfirst);
}
_Myfirst = 0, _Mylast = 0, _Myend = 0;
}

*************************

void deallocate(pointer _Ptr, size_type)
{ // deallocate object at _Ptr, ignore size
operator delete(_Ptr);
}

************************
void operator delete( void *pUserData)
{
_CrtMemBlockHeader * pHead;

RTCCALLBACK(_RTC_Free_hook, (pUserData, 0));

if (pUserData == NULL)
return;

_mlock(_HEAP_LOCK); /* block other threads */
__TRY

/* get a pointer to memory block header */
pHead = pHdr(pUserData);

/* verify block type */
_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));

_free_dbg( pUserData, pHead->nBlockUse );

__FINALLY
_munlock(_HEAP_LOCK); /* release other threads */
__END_TRY_FINALLY

return;
}

And that’s where it fails, the
“_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));”

The thing is, like I said, all runs fine if I don’t run the debugger. Is
anyone using the std:vector<> with success or has a clue what might be going
wrong?

Cheers,
Thijs

http://www.nano-soundworks.com

#33325
Aug 20, 2007 at 2:46pm

Been using various stl containers including vector in externals for a
while on Mac, not yet on Windows. Are you sure you are always
accessing it from the same thread? Stl is not thread safe. Just an
idea.

On Aug 20, 2007, at 4:26 AM, Thijs Koerselman wrote:

> I’ve written an external using C++ which makes use of std::vector.
> When I run it without the debugger attached everything seems to be
> fine and stable. When I run the debugger I get an exception on
> vector functions like resize() and clear().
>
> It points to a failing assertion in the msvcr71d.dll
>
> For example if I call “x->segList.clear();” on my vector, it
> triggers the following function calls in the STD library:
>
> void clear()
> { // erase all
> _Tidy();
> }
>
> *************************
>
> void _Tidy()
> { // free all storage
> if (_Myfirst != 0)
> { // something to free, destroy and deallocate it
> _Destroy(_Myfirst, _Mylast);
> this->_Alval.deallocate(_Myfirst, _Myend – _Myfirst);
> }
> _Myfirst = 0, _Mylast = 0, _Myend = 0;
> }
>
> *************************
>
> void deallocate(pointer _Ptr, size_type)
> { // deallocate object at _Ptr, ignore size
> operator delete(_Ptr);
> }
>
> ************************
> void operator delete( void *pUserData)
> {
> _CrtMemBlockHeader * pHead;
>
> RTCCALLBACK(_RTC_Free_hook, (pUserData, 0));
>
> if (pUserData == NULL)
> return;
>
> _mlock(_HEAP_LOCK); /* block other threads */
> __TRY
>
> /* get a pointer to memory block header */
> pHead = pHdr(pUserData);
>
> /* verify block type */
> _ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));
>
> _free_dbg( pUserData, pHead->nBlockUse );
>
> __FINALLY
> _munlock(_HEAP_LOCK); /* release other threads */
> __END_TRY_FINALLY
>
> return;
> }
>
>
> And that’s where it fails, the “_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead-
> >nBlockUse));”
>
> The thing is, like I said, all runs fine if I don’t run the
> debugger. Is anyone using the std:vector<> with success or has a
> clue what might be going wrong?
>
>
> Cheers,
> Thijs
>
>
> —
> http://www.nano-soundworks.com

grrr waaa
http://www.grahamwakefield.net

#110868
Aug 20, 2007 at 5:03pm

On 8/20/07, Graham Wakefield

wrote:
>
> Been using various stl containers including vector in externals for a
> while on Mac, not yet on Windows. Are you sure you are always accessing it
> from the same thread? Stl is not thread safe. Just an idea.
>
>
Hi Graham,

That’s good news. I don’t do anything with threads at the moment. I did some
more investigating and realize I must be doing something wrong regarding the
allocation of the vector.

If I allocate a local vector I can do whatever, it works fine. If I try to
do the same on the vector that is a member of my class, it doesn’t work.
What am I doing wrong? (code below)

Cheers,
Thijs

class Foo
{
public:
t_jit_object ob;
static void *_class;

std::vector mSegList;

static void *alloc(void);
static void free(Foo *x);
static void callProcess(Foo *x);
};

void* Foo::alloc(void)
{
int i;
Foo *x;
t_atom atom;

if (x = (Foo *)jit_object_alloc(Foo::_class))
{
// works fine
std::vector segList;
segList.resize(512);

// throws exception
x->mSegList.resize(512);
}
return x;
}

// function triggered by a max message
void Foo::callProcess(Foo *x)
{
// works fine
std::vector segList;
segList.clear();

// throws exception
x->mSegList.clear();
}

http://www.nano-soundworks.com

#110869
Aug 20, 2007 at 5:21pm

You’re using a C allocator not a C++ constructor. It doesn’t build
any of the class objects, just allocates memory. You can call the
constructor on allocated memory by using the placement new operator:

void *memory = alloc(Foo::_class);
Foo *f = new(memory) Foo;

http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10

but this will probably mung the t_object stuff that alloc takes care
of. There may be a way around this, but definitely you can make
mSegList a pointer and allocate it after alloc(Foo::_class);

wes

On 8/20/07, Thijs Koerselman

wrote:
>
> On 8/20/07, Graham Wakefield

wrote:
> >
> >
> > Been using various stl containers including vector in externals for a
> while on Mac, not yet on Windows. Are you sure you are always accessing it
> from the same thread? Stl is not thread safe. Just an idea.
> >
>
> Hi Graham,
>
> That’s good news. I don’t do anything with threads at the moment. I did some
> more investigating and realize I must be doing something wrong regarding the
> allocation of the vector.
>
> If I allocate a local vector I can do whatever, it works fine. If I try to
> do the same on the vector that is a member of my class, it doesn’t work.
> What am I doing wrong? (code below)
>
> Cheers,
> Thijs
>
>
> class Foo
> {
> public:
> t_jit_object ob;
> static void *_class;
>
> std::vector mSegList;
>
> static void *alloc(void);
> static void free(Foo *x);
> static void callProcess(Foo *x);
> };
>
> void* Foo::alloc(void)
> {
> int i;
> Foo *x;
> t_atom atom;
>
> if (x = (Foo *)jit_object_alloc(Foo::_class))
> {
> // works fine
> std::vector
segList;
> segList.resize(512);
>
> // throws exception
> x->mSegList.resize(512);
> }
> return x;
> }
>
> // function triggered by a max message
> void Foo::callProcess(Foo *x)
> {
> // works fine
> std::vector
segList;
> segList.clear();
>
> // throws exception
> x->mSegList.clear();
>
> }
>
> –
> http://www.nano-soundworks.com
>
>
>

#110870
Aug 20, 2007 at 6:15pm

Thanks Wes! I think I prefer the pointer approach. Although I understand how
to get around this issue now I’m still a bit unsure if I understand the
nature of the problem.

If I understand this correctly the line “std::vector mSegList;”
inside a c++ function both allocates memory AND calls the vector
constructor.
The same line of code placed in my class declaration only specifies mSegList
as a member of type std::vector<> and allocates sufficient memory for it.
Because my Foo class is allocated by the Max/C allocation (instead of
calling the Foo() constructor like you normally would on a C++ class) the
constructor of mSegList never gets called and the std::vector object for
mSegList isn’t created. Am I right about this?

I’m still curious to what Grahams approach is to constructing vectors in his
C++ code…?

All the best,
Thijs

http://www.nano-soundworks.com

On 8/20/07, Wesley Smith wrote:
>
> You’re using a C allocator not a C++ constructor. It doesn’t build
> any of the class objects, just allocates memory. You can call the
> constructor on allocated memory by using the placement new operator:
>
> void *memory = alloc(Foo::_class);
> Foo *f = new(memory) Foo;
>
> http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10
>
>
> but this will probably mung the t_object stuff that alloc takes care
> of. There may be a way around this, but definitely you can make
> mSegList a pointer and allocate it after alloc(Foo::_class);
>
> wes
>
>
> On 8/20/07, Thijs Koerselman

wrote:
> >
> > On 8/20/07, Graham Wakefield

wrote:
> > >
> > >
> > > Been using various stl containers including vector in externals for a
> > while on Mac, not yet on Windows. Are you sure you are always accessing
> it
> > from the same thread? Stl is not thread safe. Just an idea.
> > >
> >
> > Hi Graham,
> >
> > That’s good news. I don’t do anything with threads at the moment. I did
> some
> > more investigating and realize I must be doing something wrong regarding
> the
> > allocation of the vector.
> >
> > If I allocate a local vector I can do whatever, it works fine. If I try
> to
> > do the same on the vector that is a member of my class, it doesn’t work.
> > What am I doing wrong? (code below)
> >
> > Cheers,
> > Thijs
> >
> >
> > class Foo
> > {
> > public:
> > t_jit_object ob;
> > static void *_class;
> >
> > std::vector mSegList;
> >
> > static void *alloc(void);
> > static void free(Foo *x);
> > static void callProcess(Foo *x);
> > };
> >
> > void* Foo::alloc(void)
> > {
> > int i;
> > Foo *x;
> > t_atom atom;
> >
> > if (x = (Foo *)jit_object_alloc(Foo::_class))
> > {
> > // works fine
> > std::vector
segList;
> > segList.resize(512);
> >
> > // throws exception
> > x->mSegList.resize(512);
> > }
> > return x;
> > }
> >
> > // function triggered by a max message
> > void Foo::callProcess(Foo *x)
> > {
> > // works fine
> > std::vector
segList;
> > segList.clear();
> >
> > // throws exception
> > x->mSegList.clear();
> >
> > }
> >
> > –
> > http://www.nano-soundworks.com
> >
> >
> >
>

#110871
Aug 20, 2007 at 6:21pm

On 8/20/07, Thijs Koerselman

wrote:
>
>
> The same line of code placed in my class declaration only specifies
> mSegList as a member of type std::vector<> and allocates sufficient memory
> for it.

I meant to say:
The same line of code placed in my class declaration only specifies mSegList
as a member of type std::vector<> and jit_object_alloc() allocates
sufficient memory for it.

#110872
Aug 20, 2007 at 6:24pm

> If I understand this correctly the line “std::vector mSegList;”
> inside a c++ function both allocates memory AND calls the vector
> constructor.
> The same line of code placed in my class declaration only specifies mSegList
> as a member of type std::vector<> and allocates sufficient memory for it.
> Because my Foo class is allocated by the Max/C allocation (instead of
> calling the Foo() constructor like you normally would on a C++ class) the
> constructor of mSegList never gets called and the std::vector object for
> mSegList isn’t created. Am I right about this?

yes
wes

#110873
Aug 22, 2007 at 2:35am

I’m doing some template hackery to make max externals feel like C++
classes, with constructors and all so that member variables
initialize properly.

It’s a bit rough around the edges, but here’s the jist. When (if) I
clean this up nicely and document it, I may post it online as a very
very lightweight alternative to flext for simple C++ max externals.
Basically I’m using overloaded new and delete operators to allow Max
to manage memory while still calling my own constructors. I’m also
doing a couple of other things that make the user code look much
simpler (and I’m a fan of that).

HTH

My test class external:

#include
#include “max.cpp.h”

class TestExternal : public MaxCpp
{
public:

TestExternal(t_symbol * s, long ac, t_atom * av);
~TestExternal();

protected:

std::vector mIntVector;

};

TestExternal::TestExternal(t_symbol *s, long ac, t_atom *av)
: MaxCpp()
{
// do stuff
}

TestExternal :: ~TestExternal()
{
// undo stuff
}

// called when library is loaded
extern “C” int main(void)
{
t_class * c = MaxVessel::maxMakeClass(“testexternal”);

// register this as a class for Max UI & store static reference
class_register(CLASS_BOX, c);
}

And here’s the magic, in max.cpp.h:

#ifndef MAXMSP_CPP_H
#define MAXMSP_CPP_H

#include

#include “ext.h”
#include “ext_common.h”
#include “commonsyms.h”
#include “ext_obex.h”
#include “ext_path.h”
#include “z_dsp.h”
#include “buffer.h”

// abstraction to provide inlet & outlet signals in a safer way
typedef std::vector signalArray;

template
class MaxCpp
{

protected:
// for Max external-loading & Obex:
t_pxobject maxOb; // Max ‘instance’ pointer
void * maxObex; // Max Obex container
static void * maxClass; // Max ‘class’ pointer
static char * maxClassName; // our own reference

private:
static void * maxAlloc(t_symbol *s, long argc, t_atom *argv);
static void maxFree(T *x);
static void * operator new(size_t);
static void operator delete(void *);

public:

// called from ‘main’ to create class ptr
static t_class * maxMakeClass(char * name); // use this for Max objects

// use normal C++ constructors & destructors:
MaxCpp();
~MaxCpp();

// ‘dump’ obex outlet:
t_object * maxObexOutlet;
};

// singleton class object
template
void * MaxCpp::maxClass = 0;
template
char * MaxCpp::maxClassName;

template
inline t_class * MaxCpp::maxMakeClass(char * name)
{
t_class * c = class_new(
name,
(method)T::maxAlloc,
(method)T::maxFree,
(short)sizeof(T),
0L,
A_GIMME,
0);

maxClassName = name;

// initialize the common symbols, since we want to use them
common_symbols_init();

// register the byte offset of obex with the class
class_obexoffset_set(c, calcoffset(T, maxObex));

// add methods for dumpout and quickref
class_addmethod(c, (method)object_obex_dumpout, “dumpout”, A_CANT, 0);
class_addmethod(c, (method)object_obex_quickref,”quickref”, A_CANT,
0);

// store static ref
MaxCpp::maxClass = c;

return c;
}

template
inline void * MaxCpp :: maxAlloc(t_symbol * sym, long ac, t_atom *
av)
{
return new T(sym, ac, av);
}

template
inline void MaxCpp :: maxFree(T * x)
{
delete x;
}

template
inline void * MaxCpp :: operator new(size_t t)
{
return object_alloc((t_class *)MaxCpp
::maxClass);
}

template
inline void MaxCpp :: operator delete(void *)
{
// nothing to do – Max will manage the memory
}

template
inline MaxCpp::MaxCpp()
{
// add a generic outlet
maxObexOutlet = (t_object *)outlet_new(this, 0L);
object_obex_store(this, _sym_dumpout, maxObexOutlet); // also set it
as dumpout
}

template
inline MaxCpp :: ~MaxCpp()
{
}

#endif

On Aug 20, 2007, at 11:15 AM, Thijs Koerselman wrote:

> Thanks Wes! I think I prefer the pointer approach. Although I
> understand how to get around this issue now I’m still a bit unsure
> if I understand the nature of the problem.
>
> If I understand this correctly the line “std::vector
> mSegList;” inside a c++ function both allocates memory AND calls
> the vector constructor.
> The same line of code placed in my class declaration only specifies
> mSegList as a member of type std::vector<> and allocates sufficient
> memory for it. Because my Foo class is allocated by the Max/C
> allocation (instead of calling the Foo() constructor like you
> normally would on a C++ class) the constructor of mSegList never
> gets called and the std::vector object for mSegList isn’t created.
> Am I right about this?
>
> I’m still curious to what Grahams approach is to constructing
> vectors in his C++ code…?
>
> All the best,
> Thijs
>
>
> —
> http://www.nano-soundworks.com
>
> On 8/20/07, Wesley Smith wrote: You’re
> using a C allocator not a C++ constructor. It doesn’t build
> any of the class objects, just allocates memory. You can call the
> constructor on allocated memory by using the placement new operator:
>
> void *memory = alloc(Foo::_class);
> Foo *f = new(memory) Foo;
>
> http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10
>
>
> but this will probably mung the t_object stuff that alloc takes care
> of. There may be a way around this, but definitely you can make
> mSegList a pointer and allocate it after alloc(Foo::_class);
>
> wes
>
>
> On 8/20/07, Thijs Koerselman < thijskoerselman@gmail.com> wrote:
> >
> > On 8/20/07, Graham Wakefield

wrote:
> > >
> > >
> > > Been using various stl containers including vector in externals
> for a
> > while on Mac, not yet on Windows. Are you sure you are always
> accessing it
> > from the same thread? Stl is not thread safe. Just an idea.
> > >
> >
> > Hi Graham,
> >
> > That’s good news. I don’t do anything with threads at the moment.
> I did some
> > more investigating and realize I must be doing something wrong
> regarding the
> > allocation of the vector.
> >
> > If I allocate a local vector I can do whatever, it works fine.
> If I try to
> > do the same on the vector that is a member of my class, it
> doesn’t work.
> > What am I doing wrong? (code below)
> >
> > Cheers,
> > Thijs
> >
> >
> > class Foo
> > {
> > public:
> > t_jit_object ob;
> > static void *_class;
> >
> > std::vector mSegList;
> >
> > static void *alloc(void);
> > static void free(Foo *x);
> > static void callProcess(Foo *x);
> > };
> >
> > void* Foo::alloc(void)
> > {
> > int i;
> > Foo *x;
> > t_atom atom;
> >
> > if (x = (Foo *)jit_object_alloc(Foo::_class))
> > {
> > // works fine
> > std::vector
segList;
> > segList.resize(512);
> >
> > // throws exception
> > x->mSegList.resize(512);
> > }
> > return x;
> > }
> >
> > // function triggered by a max message
> > void Foo::callProcess(Foo *x)
> > {
> > // works fine
> > std::vector
segList;
> > segList.clear();
> >
> > // throws exception
> > x->mSegList.clear ();
> >
> > }
> >
> > –
> > http://www.nano-soundworks.com
> >
> >
> >
>

grrr waaa
http://www.grahamwakefield.net

#110874
Aug 22, 2007 at 10:22am

On 8/22/07, Graham Wakefield

wrote:
>
> I’m doing some template hackery to make max externals feel like C++
> classes, with constructors and all so that member variables
> initialize properly.
>
> It’s a bit rough around the edges, but here’s the jist. When (if) I
> clean this up nicely and document it, I may post it online as a very
> very lightweight alternative to flext for simple C++ max externals.
> Basically I’m using overloaded new and delete operators to allow Max
> to manage memory while still calling my own constructors. I’m also
> doing a couple of other things that make the user code look much
> simpler (and I’m a fan of that).

Thanks a lot Graham! That’s beautiful. I’ll have to adjust it to make it
compatible with the Jitter externals I’m writing, but I get the concept. I
started using pointers to my member classes, but its polluting my code. Your
solution is very clean and lightweight.

Cheers,
Thijs

#110875
Oct 22, 2007 at 11:21pm

Hi all,

Thought I’d share this, since it’s been useful for me already. I
improved the C++ template I’ve been using to build MaxMSP objects
somewhat – now includes wrapping of max messages to instance methods,
MSP callbacks and text editors for example. The basic idea was to
make a very simple bridge – just a single .h file – to make external
development a bit simpler for C++ oriented developers. I threw
together a quick page of documentation that should give the general
idea:

http://www.mat.ucsb.edu/~wakefield/maxcpp.htm

If anyone makes use of this template, please let me know, and
especially if you spot something that could be improved or added to
it! AFAIK it should compile fine for Windows – but I haven’t tested
it yet so do let me know if you have trouble.

Best,

Graham

PS. A much more comprehensive and feature packed C++ alternative is
to use the Flext library (http://grrrr.org/ext/flext/), which also
builds for PD etc. In contrast to Flext, I tried to minimize the use
of macros by employing templates instead (partly as a learning
exercise).

On Aug 22, 2007, at 3:22 AM, Thijs Koerselman wrote:

>
> On 8/22/07, Graham Wakefield

wrote:
> I’m doing some template hackery to make max externals feel like C++
> classes, with constructors and all so that member variables
> initialize properly.
>
> It’s a bit rough around the edges, but here’s the jist. When (if) I
> clean this up nicely and document it, I may post it online as a very
> very lightweight alternative to flext for simple C++ max externals.
> Basically I’m using overloaded new and delete operators to allow Max
> to manage memory while still calling my own constructors. I’m also
> doing a couple of other things that make the user code look much
> simpler (and I’m a fan of that).
>
> Thanks a lot Graham! That’s beautiful. I’ll have to adjust it to
> make it compatible with the Jitter externals I’m writing, but I get
> the concept. I started using pointers to my member classes, but its
> polluting my code. Your solution is very clean and lightweight.
>
> Cheers,
> Thijs
>
>
>

grrr waaa
http://www.grahamwakefield.net

#110876
Oct 23, 2007 at 2:06am

sounds great, and i will try it out as soon as i get time.
i wonder if you notice any differences in compile time when you are using all those templates.

thanks for sharing.

#110877
Oct 23, 2007 at 2:10am

It’s not that significant a difference. Anyway, it’s best to let the
compiler do more work for you if you can.

wes

On 10/22/07, Robert Ramirez wrote:
>
> sounds great, and i will try it out as soon as i get time.
> i wonder if you notice any differences in compile time when you are using all those templates.
>
> thanks for sharing.
>

#110878
Oct 23, 2007 at 3:35am

Didn’t notice it… However the nice thing is that the runtime
performance should be entirely unaffected.

On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:

>
> sounds great, and i will try it out as soon as i get time.
> i wonder if you notice any differences in compile time when you are
> using all those templates.
>
> thanks for sharing.

grrr waaa
http://www.grahamwakefield.net

#110879
Oct 26, 2007 at 5:10am

Very cool templated-code; well, there is no run-time hit since there is no
polymorphism in the code (from what I could see anyway) and the compiler is
doing all of the work here :)

Will play around with this, thanks!

Brandon

On 10/22/07, Graham Wakefield

wrote:
>
> Didn’t notice it… However the nice thing is that the runtime
> performance should be entirely unaffected.
>
> On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:
>
>
> sounds great, and i will try it out as soon as i get time.
> i wonder if you notice any differences in compile time when you are using
> all those templates.
>
> thanks for sharing.
>
>
> grrr waaa
> http://www.grahamwakefield.net
>
>
>
>
>
>
>

#110880
Nov 2, 2007 at 11:32pm

Let me know if you have any helpful additions/corrections!

On Oct 25, 2007, at 10:10 PM, Brandon Nickell wrote:

> Very cool templated-code; well, there is no run-time hit since
> there is no polymorphism in the code (from what I could see anyway)
> and the compiler is doing all of the work here :)
>
> Will play around with this, thanks!
>
> Brandon
>
> On 10/22/07, Graham Wakefield

wrote:
> Didn’t notice it… However the nice thing is that the runtime
> performance should be entirely unaffected.
>
> On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:
>
>>
>> sounds great, and i will try it out as soon as i get time.
>> i wonder if you notice any differences in compile time when you
>> are using all those templates.
>>
>> thanks for sharing.
>
> grrr waaa
> http://www.grahamwakefield.net
>
>
>
>
>
>
>

grrr waaa
http://www.grahamwakefield.net

#110881

You must be logged in to reply to this topic.