std::vector conflicting memory alloc?


    Aug 20 2007 | 11:26 am
    I've written an external using C++ which makes use of std::vector. When I run it without the debugger attached everything seems to be fine and stable. When I run the debugger I get an exception on vector functions like resize() and clear().
    It points to a failing assertion in the msvcr71d.dll
    For example if I call "x->segList.clear();" on my vector, it triggers the following function calls in the STD library:
    void clear() { // erase all _Tidy(); }
    *************************
    void _Tidy() { // free all storage if (_Myfirst != 0) { // something to free, destroy and deallocate it _Destroy(_Myfirst, _Mylast); this->_Alval.deallocate(_Myfirst, _Myend - _Myfirst); } _Myfirst = 0, _Mylast = 0, _Myend = 0; }
    *************************
    void deallocate(pointer _Ptr, size_type) { // deallocate object at _Ptr, ignore size operator delete(_Ptr); }
    ************************ void operator delete( void *pUserData) { _CrtMemBlockHeader * pHead;
    RTCCALLBACK(_RTC_Free_hook, (pUserData, 0));
    if (pUserData == NULL) return;
    _mlock(_HEAP_LOCK); /* block other threads */ __TRY
    /* get a pointer to memory block header */ pHead = pHdr(pUserData);
    /* verify block type */ _ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));
    _free_dbg( pUserData, pHead->nBlockUse );
    __FINALLY _munlock(_HEAP_LOCK); /* release other threads */ __END_TRY_FINALLY
    return; }
    And that's where it fails, the "_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse));"
    The thing is, like I said, all runs fine if I don't run the debugger. Is anyone using the std:vector<> with success or has a clue what might be going wrong?
    Cheers, Thijs

    • Aug 20 2007 | 2:46 pm
      Been using various stl containers including vector in externals for a while on Mac, not yet on Windows. Are you sure you are always accessing it from the same thread? Stl is not thread safe. Just an idea.
      On Aug 20, 2007, at 4:26 AM, Thijs Koerselman wrote:
      > I've written an external using C++ which makes use of std::vector. > When I run it without the debugger attached everything seems to be > fine and stable. When I run the debugger I get an exception on > vector functions like resize() and clear(). > > It points to a failing assertion in the msvcr71d.dll > > For example if I call "x->segList.clear();" on my vector, it > triggers the following function calls in the STD library: > > void clear() > { // erase all > _Tidy(); > } > > ************************* > > void _Tidy() > { // free all storage > if (_Myfirst != 0) > { // something to free, destroy and deallocate it > _Destroy(_Myfirst, _Mylast); > this->_Alval.deallocate(_Myfirst, _Myend - _Myfirst); > } > _Myfirst = 0, _Mylast = 0, _Myend = 0; > } > > ************************* > > void deallocate(pointer _Ptr, size_type) > { // deallocate object at _Ptr, ignore size > operator delete(_Ptr); > } > > ************************ > void operator delete( void *pUserData) > { > _CrtMemBlockHeader * pHead; > > RTCCALLBACK(_RTC_Free_hook, (pUserData, 0)); > > if (pUserData == NULL) > return; > > _mlock(_HEAP_LOCK); /* block other threads */ > __TRY > > /* get a pointer to memory block header */ > pHead = pHdr(pUserData); > > /* verify block type */ > _ASSERTE(_BLOCK_TYPE_IS_VALID(pHead->nBlockUse)); > > _free_dbg( pUserData, pHead->nBlockUse ); > > __FINALLY > _munlock(_HEAP_LOCK); /* release other threads */ > __END_TRY_FINALLY > > return; > } > > > And that's where it fails, the "_ASSERTE(_BLOCK_TYPE_IS_VALID(pHead- > >nBlockUse));" > > The thing is, like I said, all runs fine if I don't run the > debugger. Is anyone using the std:vector<> with success or has a > clue what might be going wrong? > > > Cheers, > Thijs > > > -- > http://www.nano-soundworks.com
      grrr waaa www.grahamwakefield.net
    • Aug 20 2007 | 5:03 pm
      On 8/20/07, Graham Wakefield wrote: > > Been using various stl containers including vector in externals for a > while on Mac, not yet on Windows. Are you sure you are always accessing it > from the same thread? Stl is not thread safe. Just an idea. > > Hi Graham,
      That's good news. I don't do anything with threads at the moment. I did some more investigating and realize I must be doing something wrong regarding the allocation of the vector.
      If I allocate a local vector I can do whatever, it works fine. If I try to do the same on the vector that is a member of my class, it doesn't work. What am I doing wrong? (code below)
      Cheers, Thijs
      class Foo { public: t_jit_object ob; static void *_class;
      std::vector mSegList;
      static void *alloc(void); static void free(Foo *x); static void callProcess(Foo *x); };
      void* Foo::alloc(void) { int i; Foo *x; t_atom atom;
      if (x = (Foo *)jit_object_alloc(Foo::_class)) { // works fine std::vector segList; segList.resize(512);
      // throws exception x->mSegList.resize(512); } return x; }
      // function triggered by a max message void Foo::callProcess(Foo *x) { // works fine std::vector segList; segList.clear();
      // throws exception x->mSegList.clear(); }
    • Aug 20 2007 | 5:21 pm
      You're using a C allocator not a C++ constructor. It doesn't build any of the class objects, just allocates memory. You can call the constructor on allocated memory by using the placement new operator:
      void *memory = alloc(Foo::_class); Foo *f = new(memory) Foo;
      but this will probably mung the t_object stuff that alloc takes care of. There may be a way around this, but definitely you can make mSegList a pointer and allocate it after alloc(Foo::_class);
      wes
      On 8/20/07, Thijs Koerselman wrote: > > On 8/20/07, Graham Wakefield wrote: > > > > > > Been using various stl containers including vector in externals for a > while on Mac, not yet on Windows. Are you sure you are always accessing it > from the same thread? Stl is not thread safe. Just an idea. > > > > Hi Graham, > > That's good news. I don't do anything with threads at the moment. I did some > more investigating and realize I must be doing something wrong regarding the > allocation of the vector. > > If I allocate a local vector I can do whatever, it works fine. If I try to > do the same on the vector that is a member of my class, it doesn't work. > What am I doing wrong? (code below) > > Cheers, > Thijs > > > class Foo > { > public: > t_jit_object ob; > static void *_class; > > std::vector mSegList; > > static void *alloc(void); > static void free(Foo *x); > static void callProcess(Foo *x); > }; > > void* Foo::alloc(void) > { > int i; > Foo *x; > t_atom atom; > > if (x = (Foo *)jit_object_alloc(Foo::_class)) > { > // works fine > std::vector segList; > segList.resize(512); > > // throws exception > x->mSegList.resize(512); > } > return x; > } > > // function triggered by a max message > void Foo::callProcess(Foo *x) > { > // works fine > std::vector segList; > segList.clear(); > > // throws exception > x->mSegList.clear(); > > } > > -- > http://www.nano-soundworks.com > > >
    • Aug 20 2007 | 6:15 pm
      Thanks Wes! I think I prefer the pointer approach. Although I understand how to get around this issue now I'm still a bit unsure if I understand the nature of the problem.
      If I understand this correctly the line "std::vector mSegList;" inside a c++ function both allocates memory AND calls the vector constructor. The same line of code placed in my class declaration only specifies mSegList as a member of type std::vector<> and allocates sufficient memory for it. Because my Foo class is allocated by the Max/C allocation (instead of calling the Foo() constructor like you normally would on a C++ class) the constructor of mSegList never gets called and the std::vector object for mSegList isn't created. Am I right about this?
      I'm still curious to what Grahams approach is to constructing vectors in his C++ code...?
      All the best, Thijs
      On 8/20/07, Wesley Smith wrote: > > You're using a C allocator not a C++ constructor. It doesn't build > any of the class objects, just allocates memory. You can call the > constructor on allocated memory by using the placement new operator: > > void *memory = alloc(Foo::_class); > Foo *f = new(memory) Foo; > > http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10 > > > but this will probably mung the t_object stuff that alloc takes care > of. There may be a way around this, but definitely you can make > mSegList a pointer and allocate it after alloc(Foo::_class); > > wes > > > On 8/20/07, Thijs Koerselman wrote: > > > > On 8/20/07, Graham Wakefield wrote: > > > > > > > > > Been using various stl containers including vector in externals for a > > while on Mac, not yet on Windows. Are you sure you are always accessing > it > > from the same thread? Stl is not thread safe. Just an idea. > > > > > > > Hi Graham, > > > > That's good news. I don't do anything with threads at the moment. I did > some > > more investigating and realize I must be doing something wrong regarding > the > > allocation of the vector. > > > > If I allocate a local vector I can do whatever, it works fine. If I try > to > > do the same on the vector that is a member of my class, it doesn't work. > > What am I doing wrong? (code below) > > > > Cheers, > > Thijs > > > > > > class Foo > > { > > public: > > t_jit_object ob; > > static void *_class; > > > > std::vector mSegList; > > > > static void *alloc(void); > > static void free(Foo *x); > > static void callProcess(Foo *x); > > }; > > > > void* Foo::alloc(void) > > { > > int i; > > Foo *x; > > t_atom atom; > > > > if (x = (Foo *)jit_object_alloc(Foo::_class)) > > { > > // works fine > > std::vector segList; > > segList.resize(512); > > > > // throws exception > > x->mSegList.resize(512); > > } > > return x; > > } > > > > // function triggered by a max message > > void Foo::callProcess(Foo *x) > > { > > // works fine > > std::vector segList; > > segList.clear(); > > > > // throws exception > > x->mSegList.clear(); > > > > } > > > > -- > > http://www.nano-soundworks.com > > > > > > >
    • Aug 20 2007 | 6:21 pm
      On 8/20/07, Thijs Koerselman wrote: > > > The same line of code placed in my class declaration only specifies > mSegList as a member of type std::vector<> and allocates sufficient memory > for it.
      I meant to say: The same line of code placed in my class declaration only specifies mSegList as a member of type std::vector<> and jit_object_alloc() allocates sufficient memory for it.
    • Aug 20 2007 | 6:24 pm
      > If I understand this correctly the line "std::vector mSegList;" > inside a c++ function both allocates memory AND calls the vector > constructor. > The same line of code placed in my class declaration only specifies mSegList > as a member of type std::vector<> and allocates sufficient memory for it. > Because my Foo class is allocated by the Max/C allocation (instead of > calling the Foo() constructor like you normally would on a C++ class) the > constructor of mSegList never gets called and the std::vector object for > mSegList isn't created. Am I right about this?
      yes wes
    • Aug 22 2007 | 2:35 am
      I'm doing some template hackery to make max externals feel like C++ classes, with constructors and all so that member variables initialize properly.
      It's a bit rough around the edges, but here's the jist. When (if) I clean this up nicely and document it, I may post it online as a very very lightweight alternative to flext for simple C++ max externals. Basically I'm using overloaded new and delete operators to allow Max to manage memory while still calling my own constructors. I'm also doing a couple of other things that make the user code look much simpler (and I'm a fan of that).
      HTH
      My test class external:
      #include #include "max.cpp.h"
      class TestExternal : public MaxCpp { public:
      TestExternal(t_symbol * s, long ac, t_atom * av); ~TestExternal();
      protected:
      std::vector mIntVector;
      };
      TestExternal::TestExternal(t_symbol *s, long ac, t_atom *av) : MaxCpp() { // do stuff }
      TestExternal :: ~TestExternal() { // undo stuff }
      // called when library is loaded extern "C" int main(void) { t_class * c = MaxVessel::maxMakeClass("testexternal");
      // register this as a class for Max UI & store static reference class_register(CLASS_BOX, c); }
      And here's the magic, in max.cpp.h:
      #ifndef MAXMSP_CPP_H #define MAXMSP_CPP_H
      #include
      #include "ext.h" #include "ext_common.h" #include "commonsyms.h" #include "ext_obex.h" #include "ext_path.h" #include "z_dsp.h" #include "buffer.h"
      // abstraction to provide inlet & outlet signals in a safer way typedef std::vector signalArray;
      template class MaxCpp {
      protected: // for Max external-loading & Obex: t_pxobject maxOb; // Max 'instance' pointer void * maxObex; // Max Obex container static void * maxClass; // Max 'class' pointer static char * maxClassName; // our own reference
      private: static void * maxAlloc(t_symbol *s, long argc, t_atom *argv); static void maxFree(T *x); static void * operator new(size_t); static void operator delete(void *);
      public:
      // called from 'main' to create class ptr static t_class * maxMakeClass(char * name); // use this for Max objects
      // use normal C++ constructors & destructors: MaxCpp(); ~MaxCpp();
      // 'dump' obex outlet: t_object * maxObexOutlet; };
      // singleton class object template void * MaxCpp::maxClass = 0; template char * MaxCpp::maxClassName;
      template inline t_class * MaxCpp::maxMakeClass(char * name) { t_class * c = class_new( name, (method)T::maxAlloc, (method)T::maxFree, (short)sizeof(T), 0L, A_GIMME, 0);
      maxClassName = name;
      // initialize the common symbols, since we want to use them common_symbols_init();
      // register the byte offset of obex with the class class_obexoffset_set(c, calcoffset(T, maxObex));
      // add methods for dumpout and quickref class_addmethod(c, (method)object_obex_dumpout, "dumpout", A_CANT, 0); class_addmethod(c, (method)object_obex_quickref,"quickref", A_CANT, 0);
      // store static ref MaxCpp::maxClass = c;
      return c; }
      template inline void * MaxCpp :: maxAlloc(t_symbol * sym, long ac, t_atom * av) { return new T(sym, ac, av); }
      template inline void MaxCpp :: maxFree(T * x) { delete x; }
      template inline void * MaxCpp :: operator new(size_t t) { return object_alloc((t_class *)MaxCpp::maxClass); }
      template inline void MaxCpp :: operator delete(void *) { // nothing to do - Max will manage the memory }
      template inline MaxCpp::MaxCpp() { // add a generic outlet maxObexOutlet = (t_object *)outlet_new(this, 0L); object_obex_store(this, _sym_dumpout, maxObexOutlet); // also set it as dumpout }
      template inline MaxCpp :: ~MaxCpp() { }
      #endif
      On Aug 20, 2007, at 11:15 AM, Thijs Koerselman wrote:
      > Thanks Wes! I think I prefer the pointer approach. Although I > understand how to get around this issue now I'm still a bit unsure > if I understand the nature of the problem. > > If I understand this correctly the line "std::vector > mSegList;" inside a c++ function both allocates memory AND calls > the vector constructor. > The same line of code placed in my class declaration only specifies > mSegList as a member of type std::vector<> and allocates sufficient > memory for it. Because my Foo class is allocated by the Max/C > allocation (instead of calling the Foo() constructor like you > normally would on a C++ class) the constructor of mSegList never > gets called and the std::vector object for mSegList isn't created. > Am I right about this? > > I'm still curious to what Grahams approach is to constructing > vectors in his C++ code...? > > All the best, > Thijs > > > -- > http://www.nano-soundworks.com > > On 8/20/07, Wesley Smith wrote: You're > using a C allocator not a C++ constructor. It doesn't build > any of the class objects, just allocates memory. You can call the > constructor on allocated memory by using the placement new operator: > > void *memory = alloc(Foo::_class); > Foo *f = new(memory) Foo; > > http://www.parashift.com/c++-faq-lite/dtors.html#faq-11.10 > > > but this will probably mung the t_object stuff that alloc takes care > of. There may be a way around this, but definitely you can make > mSegList a pointer and allocate it after alloc(Foo::_class); > > wes > > > On 8/20/07, Thijs Koerselman < thijskoerselman@gmail.com> wrote: > > > > On 8/20/07, Graham Wakefield wrote: > > > > > > > > > Been using various stl containers including vector in externals > for a > > while on Mac, not yet on Windows. Are you sure you are always > accessing it > > from the same thread? Stl is not thread safe. Just an idea. > > > > > > > Hi Graham, > > > > That's good news. I don't do anything with threads at the moment. > I did some > > more investigating and realize I must be doing something wrong > regarding the > > allocation of the vector. > > > > If I allocate a local vector I can do whatever, it works fine. > If I try to > > do the same on the vector that is a member of my class, it > doesn't work. > > What am I doing wrong? (code below) > > > > Cheers, > > Thijs > > > > > > class Foo > > { > > public: > > t_jit_object ob; > > static void *_class; > > > > std::vector mSegList; > > > > static void *alloc(void); > > static void free(Foo *x); > > static void callProcess(Foo *x); > > }; > > > > void* Foo::alloc(void) > > { > > int i; > > Foo *x; > > t_atom atom; > > > > if (x = (Foo *)jit_object_alloc(Foo::_class)) > > { > > // works fine > > std::vector segList; > > segList.resize(512); > > > > // throws exception > > x->mSegList.resize(512); > > } > > return x; > > } > > > > // function triggered by a max message > > void Foo::callProcess(Foo *x) > > { > > // works fine > > std::vector segList; > > segList.clear(); > > > > // throws exception > > x->mSegList.clear (); > > > > } > > > > -- > > http://www.nano-soundworks.com > > > > > > >
      grrr waaa www.grahamwakefield.net
    • Aug 22 2007 | 10:22 am
      On 8/22/07, Graham Wakefield wrote: > > I'm doing some template hackery to make max externals feel like C++ > classes, with constructors and all so that member variables > initialize properly. > > It's a bit rough around the edges, but here's the jist. When (if) I > clean this up nicely and document it, I may post it online as a very > very lightweight alternative to flext for simple C++ max externals. > Basically I'm using overloaded new and delete operators to allow Max > to manage memory while still calling my own constructors. I'm also > doing a couple of other things that make the user code look much > simpler (and I'm a fan of that).
      Thanks a lot Graham! That's beautiful. I'll have to adjust it to make it compatible with the Jitter externals I'm writing, but I get the concept. I started using pointers to my member classes, but its polluting my code. Your solution is very clean and lightweight.
      Cheers, Thijs
    • Oct 22 2007 | 11:21 pm
      Hi all,
      Thought I'd share this, since it's been useful for me already. I improved the C++ template I've been using to build MaxMSP objects somewhat - now includes wrapping of max messages to instance methods, MSP callbacks and text editors for example. The basic idea was to make a very simple bridge - just a single .h file - to make external development a bit simpler for C++ oriented developers. I threw together a quick page of documentation that should give the general idea:
      www.mat.ucsb.edu/~wakefield/maxcpp.htm
      If anyone makes use of this template, please let me know, and especially if you spot something that could be improved or added to it! AFAIK it should compile fine for Windows - but I haven't tested it yet so do let me know if you have trouble.
      Best,
      Graham
      PS. A much more comprehensive and feature packed C++ alternative is to use the Flext library (http://grrrr.org/ext/flext/), which also builds for PD etc. In contrast to Flext, I tried to minimize the use of macros by employing templates instead (partly as a learning exercise).
      On Aug 22, 2007, at 3:22 AM, Thijs Koerselman wrote:
      > > On 8/22/07, Graham Wakefield wrote: > I'm doing some template hackery to make max externals feel like C++ > classes, with constructors and all so that member variables > initialize properly. > > It's a bit rough around the edges, but here's the jist. When (if) I > clean this up nicely and document it, I may post it online as a very > very lightweight alternative to flext for simple C++ max externals. > Basically I'm using overloaded new and delete operators to allow Max > to manage memory while still calling my own constructors. I'm also > doing a couple of other things that make the user code look much > simpler (and I'm a fan of that). > > Thanks a lot Graham! That's beautiful. I'll have to adjust it to > make it compatible with the Jitter externals I'm writing, but I get > the concept. I started using pointers to my member classes, but its > polluting my code. Your solution is very clean and lightweight. > > Cheers, > Thijs > > >
      grrr waaa www.grahamwakefield.net
    • Oct 23 2007 | 2:06 am
      sounds great, and i will try it out as soon as i get time. i wonder if you notice any differences in compile time when you are using all those templates.
      thanks for sharing.
    • Oct 23 2007 | 2:10 am
      It's not that significant a difference. Anyway, it's best to let the compiler do more work for you if you can.
      wes
      On 10/22/07, Robert Ramirez wrote: > > sounds great, and i will try it out as soon as i get time. > i wonder if you notice any differences in compile time when you are using all those templates. > > thanks for sharing. >
    • Oct 23 2007 | 3:35 am
      Didn't notice it... However the nice thing is that the runtime performance should be entirely unaffected.
      On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote:
      > > sounds great, and i will try it out as soon as i get time. > i wonder if you notice any differences in compile time when you are > using all those templates. > > thanks for sharing.
      grrr waaa www.grahamwakefield.net
    • Oct 26 2007 | 5:10 am
      Very cool templated-code; well, there is no run-time hit since there is no polymorphism in the code (from what I could see anyway) and the compiler is doing all of the work here :)
      Will play around with this, thanks!
      Brandon
      On 10/22/07, Graham Wakefield wrote: > > Didn't notice it... However the nice thing is that the runtime > performance should be entirely unaffected. > > On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote: > > > sounds great, and i will try it out as soon as i get time. > i wonder if you notice any differences in compile time when you are using > all those templates. > > thanks for sharing. > > > grrr waaa > www.grahamwakefield.net > > > > > > >
    • Nov 02 2007 | 11:32 pm
      Let me know if you have any helpful additions/corrections!
      On Oct 25, 2007, at 10:10 PM, Brandon Nickell wrote:
      > Very cool templated-code; well, there is no run-time hit since > there is no polymorphism in the code (from what I could see anyway) > and the compiler is doing all of the work here :) > > Will play around with this, thanks! > > Brandon > > On 10/22/07, Graham Wakefield wrote: > Didn't notice it... However the nice thing is that the runtime > performance should be entirely unaffected. > > On Oct 22, 2007, at 7:06 PM, Robert Ramirez wrote: > >> >> sounds great, and i will try it out as soon as i get time. >> i wonder if you notice any differences in compile time when you >> are using all those templates. >> >> thanks for sharing. > > grrr waaa > www.grahamwakefield.net > > > > > > >
      grrr waaa www.grahamwakefield.net