Debugging jit.gl.slab pipelines


    Oct 16 2014 | 5:19 pm
    I decided that to debug shaders, I would use jit.matrix with a small test case (such as a 3x3 or 4x4 matrix) but I am having some problems, due to some confusion I may have on the relationships among objects such as slab, texture, render, and videoplane in gl. I have attached a small example. Here are my assumptions on how the objects work, followed by expected results.
    jit.gl.render and jit.gl.videoplane (these seem to be connected somehow but it is unclear). The first renders to the GPU using a "rendering context." The documentation for videoplane says that this is yet another way to render but with many controls (messages, attrui). A jit.matrix defines a matrix -- this one seems clear enough.
    A slab is a buffer on the GPU where shaders may be applied. In what I attach, I am including the known blank shader which just passes everything through (and so performs no manipulation). What I expect is to see the same matrix at the bottom that is fed in through the top.
    The use of the latching mechanism (zl reg) was in the GameOfLife example and prior use of it seems to work for a single delay, but I have tried creating a 2nd or 3rd order delay by chaining them together without this working.
    Comments welcome from the shader/slab folks.

    • Oct 17 2014 | 5:05 pm
      if you want to see the exact same thing in the gl window as in the matrix window, you have to turn off filtering for every stage of the texture pipeline. each object has a different method for doing that (texture, slab, videoplane), and i've demonstrated in the patch below.
      additionally, textures are converted to 4 plane rgba matrices, unless explicitly told not to do that, therefore the matrix at the output is a 4 plane argb matrix.
      a possibly better solution to disabling filtering, is to upsample without filtering at the matrix stage. i've shown how to do that in the second patch.
    • Oct 20 2014 | 3:05 am
      @Rob: Thank you for this detailed response. I have been looking carefully over your patches during the weekend, and had some observations and questions:
      1) It would seem that there is a bigger difference between [jit.window] and [jit.pwindow] than simply one window being in the patch and the other as a separate window. This is because when [jit.pwindow] is used, I don't see any filtering that you describe. The checkerboard looks fine. Is this your understanding as well?
      2) The entire filtering scenario confuses me somewhat: normally, I would expect that the default would be "no filter" unless I specify one explicitly via a shader or an option, or message to a jitter object. So, it seems peculiar that one has to turn something off that should be theoretically off by default? By having filtering on, it also seems as though the filtering has to be turned off in special ways depending on the object (you have 3 modes - "filter none", "sendoutput filter none" and "sendtexture filter none".
      3) In your second patch, you downsample by writing to a 320x240 matrix. That is fine, but the problem is trying to upsample as you do for the matrix in the lower right of the patch: note that even with the plane set to 1, the values are incorrect in [jit.cellblock] even though they are visually correct in the [jit.pwindow] to the right of the cellblock.
      Thanks for any additional clues on the above. I find it harder to create a clear mental model of what some of these jitter objects are doing. For example, with [jit.gl.videoplane], I have seen some indications that one needs one videoplane per texture. Maybe this is in the documentation. I don't have a good grasp of the technical difference between a gl.texture (residing in a buffer on the GPU) and a videoplane (is that also on the GPU?).
    • Oct 20 2014 | 2:27 pm
      In my opinion, the author's idea is advisable.I will make advertisement freely for your post,recommend it to my frields.Most of my frields are fall in love with rs. And under this circumstance,I also like to play online game,such as RuneScape.I always buy the items online.So I am familiar with the safest and convenient way to buy [url=http://www.rsgoldfast.com/]Runescape Gold[/url] online.There are: http://www.rsgoldfast.com/
    • Oct 23 2014 | 4:40 pm
      1) It would seem that there is a bigger difference between [jit.window] and [jit.pwindow] than simply one window being in the patch and the other as a separate window. This is because when [jit.pwindow] is used, I don’t see any filtering that you describe. The checkerboard looks fine. Is this your understanding as well?
      nope that should not be the case.
      2) The entire filtering scenario confuses me somewhat
      that's just how it works. filter linear is the more common scenario.
      3) In your second patch, you downsample by writing to a 320×240 matrix. That is fine, but the problem is trying to upsample as you do for the matrix in the lower right of the patch: note that even with the plane set to 1, the values are incorrect in [jit.cellblock] even though they are visually correct in the [jit.pwindow] to the right of the cellblock.
      the resolution of the matrix is now 320x240. you would have to stretch the cellblock out pretty wide before you start seeing some values that are not 0.
    • Oct 24 2014 | 5:27 am
      I follow your explanation and created a simpler patch that I am trying to grok. Some of it makes sense but the behavior doesn't match what I follow from the documentation.
      Here are my assumptions:
      STEP 1: sets up the video rendering using OpenGL. The filtering has been removed as per your example. I also changed the texture size to a power of 2.
      STEP 2: pressing the message fills a 2x2 matrix which creates a 2x2 texture called "texture1". I get a red checkerboard for render context "a9" but a black texture even though the input matrix (and texture) are 2x2, and the matrix has a green checkerboard (which is expected since plane 2 is set).
      More interestingly, jit.window a9 is red and so is the output from jit.gl.slab. The only thing I can think of here is that jit.gl.slab is a memory latch of sorts of size (one video frame) and one has to wait a frame cycle to see whatever is input to the slab.
      STEP 3: By pressing the same message [message 0. 0.4 0.4 0.] again, refilling the matrix identically to the previous click, the output from [jit.gl.texture] is blue rather than black. And a green checkerboard (correct) now pops out of the [jit.gl.slab] latch.
      @Rob: if you or some other reader can shed light on this, it would be great. I can't make heads or tails of this Jitter logic.
    • Oct 24 2014 | 3:52 pm
      Did some more on this, and it seems that [jit.gl.texture] must receive CHAR (8 bit) as the data type for each of ARGB. Using float32 sends it into the ozone. I could not find any documentation on [jit.gl.texture] requiring CHAR-based matrices when a matrix is used to create a texture. Perhaps it is in the docs somewhere.
      The interesting behavior documented above could be due to IEEE754 and how a floating point number converts to ARGB using CHAR. Still unsure about this since the 2nd refilling of the input 2x2 matrix is doing something to the [jit.gl.slab] output.
    • Oct 26 2014 | 1:29 am
      hi metaphorz. thanks for the patch. it does look like there's some inconsistencies with converting float32 matrix data to gl-textures. we will investigate.
    • Oct 26 2014 | 4:56 pm
      @Rob: Regarding the first issue encountered (filtering, which needed to be turned off to avoid filtering occurring at the end of the pipeline), I used your method in a scaled-down version of the Cycling-provided Game Of Life, but the interpolation still seems to occur. I didn't see any filtering options for [jit.noise] and [jit.gl.render]. The patch is attached below.
      Also, and I am probably hijacking my own thread here a bit, but the behavior of this patch is counter-intuitive probably because I am making a bad assumption:
      1) jit.noise seems to create only red, green, and blue whereas all other examples of [jit.noise] I could find fill the color spectrum for A, R, G, and B planes. Not sure why this is happening. 2) in [jit.gl.pix], I am assuming that the second argument to the "sample()" function is the vertex coordinate (vec2) of the pixel, and that the "v00, v01, etc" are of type vec4. This makes the "sum" also a vec4 and so relational operations such as "sum but not sure where and there seems to be no way to easily debug these codeboxes (or is there?). I can start a new thread if desired on these issues with the Game of Life patch.
    • Oct 26 2014 | 11:32 pm
      Working on some assumptions, and to make the Game of Life code a bit more educational (and debug friendly), I created a patch. I found a past post by Andrew, while looking for something different, that included the message needed to stop interpolation. There is a peculiar thing going on with having to bang the noise generator twice to get a 16x16 matrix