t_linklist and deadlock / crash


    Jan 27 2009 | 12:31 pm
    Hello All,
    (Something of a noob disclaimer applies here as I'm in the midst of my
    first serious external making attempt).
    vital stats: Max 4.6.3, PPC G4, OS 10.4.11
    I'm trying to use a t_linklist in an msp external and am getting crashes
    (normally in the gui thread) whenever I re-instantiate my object with
    dsp on. Given that the crashes are always in some other thread and they
    only started when I started using t_linklist, I'm assuming its a
    threading problem...
    Having looked at this message by JKC:
    I've tried to use a lightweight locking scheme as he suggested, but all
    I've achieved with this so far is making my crashes more reproducible.
    In my bang method, the list is appended to and sorted and I have
    something like:
    while(myObject->list_being_read); //block if list in perform
    myObject->listValid = false;
    //do stuff
    myObject->listValid = true;
    and in perform():
    if(!myObject->listValid)
    {
    //output zeroes, or something else that doesn't involve reading the
    //list and return
    }
    myObject->list_being_read = true;
    //retreive head, use as basis of output
    myObject->list_being_read = false;
    Frankly, I'm a bit stumped as to how to hunt down the cause of this.
    Given that I only want the linked list for sorting a smallish ( number of things, would I be better off doing something else altogether?
    Thanks for any sage advice,
    Owen

    • Jan 27 2009 | 12:57 pm
      Quote: owen wrote on Tue, 27 January 2009 05:31
      ----------------------------------------------------
      > I'm trying to use a t_linklist in an msp external and am getting crashes
      > (normally in the gui thread) whenever I re-instantiate my object with
      > dsp on. Given that the crashes are always in some other thread and they
      > only started when I started using t_linklist, I'm assuming its a
      > threading problem...
      Any chance of a crash report and ideally more code (especially the object struct and new routine)? Is the crash in your code, or in cycling's code? Also your subject suggests that sometimes it's not actually crashing - what exactly happens? Without a bit more info it's hard to give the best advice.
      > I've tried to use a lightweight locking scheme as he suggested, but all
      > I've achieved with this so far is making my crashes more reproducible.
      >
      > In my bang method, the list is appended to and sorted and I have
      > something like:
      This sounds fairly sensible, but are you getting crashes whilst your bang routine, or not? - above you say it's when you instantiate your object that the problem occurs...
      Regards,
      Alex
    • Jan 27 2009 | 1:08 pm
      Quote: owen wrote on Tue, 27 January 2009 05:31
      ----------------------------------------------------
      > Frankly, I'm a bit stumped as to how to hunt down the cause of this.
      > Given that I only want the linked list for sorting a smallish ( > number of things, would I be better off doing something else altogether?
      >
      If all you want to do is sort things there are certainly other ways to do this - whether it would be preferable or not is a different matter - what are you sorting exactly? is there a definite maximum list size? Do you need to keep track of the original order of the objects, or be able to remove, insert them at will?
      A.
    • Jan 27 2009 | 2:17 pm
      Hi Alex,
      Thanks. Somewhat maddeningly, behaviour has changed for reasons utterly
      beyond my ken. Now it just deadlocks (spinning beachball of death) when
      I load a patch with the object in, even if I disable the scheduler.
      If I step through in the debugger, it goes through new() just fine, and
      *then* chokes after a couple of bangs.
      > Any chance of a crash report and ideally more code (especially the
      > object struct and new routine)?
      Of course. Here's a couple of crashes, as was:
      Thread 0 Crashed:
      0 <> 0xffff8a50 __memcpy + 688 (cpu_capabilities.h:189)
      1 ...ple.CoreServices.CarbonCore 0x90408c20 CompressFromMemory + 212
      2 ...ple.CoreServices.CarbonCore 0x90408ab4 DoActualCompact + 132
      3 ...ple.CoreServices.CarbonCore 0x90400b34 UpdateTheFile + 516
      4 ...ple.CoreServices.CarbonCore 0x90402c90 UpdateResFileCommon + 96
      5 ...ple.CoreServices.CarbonCore 0x90402998 CloseResFileCommon + 288
      6 com.cycling74.MaxMSP46 0x00090078 tempfile_close + 96
      (tempfile.c:73)
      7 com.cycling74.MaxMSP46 0x000396bc main + 728 (main.c:422)
      8 com.cycling74.MaxMSP46 0x00002f44 _start + 760
      9 com.cycling74.MaxMSP46 0x00002c48 start + 48
      Thread 5 Crashed:
      0 libjvm.dylib 0x9ea24394 JVM_GetLastErrorString + 9332
      1 libjvm.dylib 0x9e8ea15c JNI_CreateJavaVM_Impl + 49276
      2 libjvm.dylib 0x9e8ea0b4 JNI_CreateJavaVM_Impl + 49108
      3 libjvm.dylib 0x9e8e9fac JNI_CreateJavaVM_Impl + 48844
      4 libSystem.B.dylib 0x9002b908 _pthread_body + 96
      So not in my code, no. (I've no mxj action going on either).
      Here's my structs:
      //--------------------------------------------------------------------
      //@struct t_oscillator Defines a single oscillator
      typedef struct {
      float phi_at_pulse; //phase at latest pulse
      float phiVel_at_pulse;// phase velocity at onset
      float errFunc; // error function, calculated at onset
      float etaLong; // strength of long-term adaptation
      float etaShort; // strength of short-term adaptation
      float gamma; // gain parameter
      float output; // current output value of the oscillator
      float phi; // running phase - serves as a mapping between //absolute
      and musical time (so gets bigger, and bigger, and bigger...)
      float period; // period, and hence frequency
      long startTime; // uh...
      float initialTempo; // attribute for initial tempo
      float resonance; // measure of 'resonance' for osc ranking
      float lastPhi;
      int active; //
      float output_at_LastEvent; //the oscillator output at the time of the
      //last event is used to measure its continuing
      synchrony
      float phi_at_LastEvent; //ditto phase
      } t_oscillator;
      //--------------------------------------------------------------------
      ///@struct t_event Defines an event, taken to be a cluster of onsets
      typedef struct {
      long time;
      long numOnsets;
      } t_event;
      //---------------------------------------------------------------------
      //@struct t_adaptosc The struct for our MSP object...
      typedef struct adaptosc {
      t_pxobject p_ob;
      long kAdaptArraySize; // size of adpatation lookup tables
      float longTerm[ARRAYSIZE]; // lookup for long term adaptation
      float shortTerm[ARRAYSIZE]; // lookup for short term adaptation
      double timeIncPerSample; // calculated increment of time per
      long lastIoi; // last inter-onset interval
      long lastPulseTime; // time of last pulse
      float clicktable[ARRAYSIZE];
      float last_phi; // the previous wrapped phase value
      float eventDelta; // time threshold for event
      t_event lastEvent; // the last event (a grouped cluster of onsets)
      t_event thisEvent; // the current event
      long numOutlets; //the number of signal outlets
      long startTime;
      //lightweight locking a la JKC:
      //
      int listValid; //lightweight lock on list mods
      int list_in_read; //lightweight lock on reads in perform
      long numOscs; //how many oscillators
      long numActive;
      t_oscillator *oscillators; // An array of oscillators
      t_linklist *osclist; // a sortable linked list for ranking
      void *p_outlet;
      } t_adaptosc;
      //--------------------------------------------------------------
      and new():
      void *adaptosc_new(t_symbol *s, long argc, t_atom *argv) {
      t_adaptosc *oscillator;
      if (oscillator = (t_adaptosc *)object_alloc(adaptosc_class))
      {
      dsp_setup((t_pxobject *)oscillator, 0);
      oscillator->p_outlet = floatout(oscillator);
      oscillator->numOutlets = 1; //for now
      outlet_new(oscillator, "signal");
      }
      long n = 0;
      if(argc > 0)
      {
      if(argv->a_type == A_LONG) n = atom_getlong(argv);
      }
      post("have we got this far? n = %ld", n);
      if(n > 0)
      {
      oscillator->numOscs = n;
      } else
      {
      oscillator->numOscs = 1;
      }
      post( "num of oscis= %ld" , oscillator->numOscs);
      oscillator->oscillators=(t_oscillator*)sysmem_newptr(sizeof(t_oscillator));
      adaptosc_reset(oscillator);
      oscillator->osclist = linklist_new();
      calculateAdaptations(oscillator); //populate adaptation lookup tables
      return(oscillator);
      }
      The linked list is gradually populated with oscillator instances from
      the array in my main object. This happens in my bang method.
      The reset() method doesn't do anything controversial, except iterating
      over the oscillators array and initializing vars. Come to think of it,
      given that it's called from dsp() also, it probably /is/ controversial,
      isn't it?
      > Also your subject suggests that sometimes it's not
      > actually crashing - what exactly happens? Without a bit more info
      > it's hard to give the best advice.
      Sorry. Well, now, as I said, it's not crashing at all, just locking up
      (although I'm pretty sure nothing of significance changed in my code).
      What it /was/ doing was sort of holding its breath, maybe having a quick
      beachball moment, and then crashing. Sometimes it would carry on
      operating audibly, but the display wouldn't be refreshed and it would
      carry on like that until you moved a window, then it would crash.
      > This sounds fairly sensible, but are you getting crashes whilst your
      > bang routine, or not? - above you say it's when you instantiate your
      > object that the problem occurs...
      It was when I re-instantiated the object by typing new arguments. (Now
      it doesn't give me the chance to do that). There didn't seem to be any
      way of attributing the crash to a particular routine of mine one way or
      the other, given that the logs always showed the crashes as being in
      other code.
      Cheers,
      Owen
    • Jan 27 2009 | 2:36 pm
      Quote: owen wrote on Tue, 27 January 2009 07:17
      ----------------------------------------------------
      > > Any chance of a crash report and ideally more code (especially the
      > > object struct and new routine)?
      >
      > Of course. Here's a couple of crashes, as was:
      > So not in my code, no. (I've no mxj action going on either).
      Crashes not in my code almost always turn out to be memory corruption I've caused due to pointer errors. Guard Malloc in XCode (in the debug menu) can be used to identify where these problems occur, but it does make the startup slow...
      > oscillator->oscillators=(t_oscillator*)sysmem_newptr(sizeof(t_oscillator));
      I'll try to look more at this later, but on a quick scan it looks like you only allocate one oscillator here, whatever oscillator->numOscs is equal to. Did you mean to do that? I think a pointer error is a prime contender here for these kinds of crashes...
      Also, as far as I can see you only have an empty linklist here, you don't seem to have added anything to it yet. Is that correct, or doee that happen in the reset routine (which isn't included here)? If you no longer crashing here I'd guess that either the problem is in your other routines, or the possible allocation error above is what's causing the problem. Is it the case that you have problems with no object argument, or an argument of 1, or only when you have more than one oscillator?
      > The reset() method doesn't do anything controversial, except iterating
      > over the oscillators array and initializing vars. Come to think of it,
      > given that it's called from dsp() also, it probably /is/ controversial,
      > isn't it?
      Well, it could be. Are you running with scheduler in audio interrupt or not? If you are and it's still crashing then I'd be looking for something other than a threading issue first before I consider that option...
      > There didn't seem to be any
      > way of attributing the crash to a particular routine of mine one way or
      > the other, given that the logs always showed the crashes as being in
      > other code.
      As I said - try Guard Malloc - hopefully that'll produce an exception when debugging that is actually in your code.
      A.
    • Jan 27 2009 | 2:37 pm
      You're really going to need to post/send the complete code for all of the functions being referenced. It wouldn't hurt to have:
      main()
      xx_new()
      xx_free()
      xx_reset()
      CalculateAdaptations()
      plus any code called from those functions. The perform method would be good too. In fact, why not just send the whole object? Until we can see what your object is doing, and what it's doing wrong, there's not much help we can provide.
      jb
    • Jan 27 2009 | 2:48 pm
      Alex Harker wrote:
      >> oscillator->oscillators=(t_oscillator*)sysmem_newptr(sizeof(t_oscillator));
      >
      > I'll try to look more at this later, but on a quick scan it looks
      > like you only allocate one oscillator here, whatever
      > oscillator->numOscs is equal to. Did you mean to do that? I think a
      > pointer error is a prime contender here for these kinds of crashes...
      Oh, jesus wept, that's embarrassing. And it seems to have fixed it also.
      I'm going to stand in the corner with a dunce hat. Sorry for wasting
      everyone's time, and many thanks.
      --
      O
    • Jan 27 2009 | 2:51 pm
      Jeremy Bernstein wrote:
      > You're really going to need to post/send the complete code for all of
      > the functions being referenced. It wouldn't hurt to have:
      (Kind of moot now as Alex has exposed my crapulence, but).
      Yeah, sorry - I'd be delighted to in general, but am / was unsure about
      just overloading helpful yet busy people. Will remember for next time
      (and will try not to have such a sucky mistake as the cause next time).
      Thanks anyway,
      --
      O
    • Jan 27 2009 | 3:04 pm
      Quote: owen wrote on Tue, 27 January 2009 07:48
      ----------------------------------------------------
      > Alex Harker wrote:
      > >> oscillator->oscillators=(t_oscillator*)sysmem_newptr(sizeof(t_oscillator));
      > >
      > > I'll try to look more at this later, but on a quick scan it looks
      > > like you only allocate one oscillator here, whatever
      > > oscillator->numOscs is equal to. Did you mean to do that? I think a
      > > pointer error is a prime contender here for these kinds of crashes...
      >
      > Oh, jesus wept, that's embarrassing. And it seems to have fixed it also.
      Well - glad that that's seemed to fix it. If that changes then post the rest of the code. And no need for embarrassment - you've no idea how many hours I've spent trying to find errors like this, or even more basic than this - they're often the hardest to spot in your own code...
      A.