Experiment: flatter strands API

[davepagurek]

This partially addresses #7994 and #7992.

The core issue is that a single p5.strands shader involves two callback functions, which is kind of a lot. The outermost one is there because we need to isolate the code that we need to transpile, so that isn't immediately going away (a short term option could be to let people load that from a file rather than a function; a longer term option could be to transpile the whole file via a custom script tag type.) This PR doesn't address that outer one. In this I'm trying to chip away at the inner callback that currently is used for each hook.

Changes:

In general, instead of doing a callback for a hook, you can use begin/end. e.g.:

Callback	Flat
baseMaterialShader().modify(() => { let myNormal = sharedVec3() getPixelInputs((inputs) => { myNormal = inputs.normal return inputs }) getFinalColor.begin((inputs) => { return mix( [1, 1, 1, 1], inputs.color, abs(dot(myNormal, [0, 0, 1])) ) }) });	baseMaterialShader().modify(() => { let myNormal = sharedVec3() getPixelInputs.begin() myNormal = getPixelInputs.normal getPixelInputs.end() getFinalColor.begin() getFinalColor.result = mix( [1, 1, 1, 1], getFinalColor.color, abs(dot(myNormal, [0, 0, 1])) ); getFinalColor.end() });

Live:
https://editor.p5js.org/davepagurek/sketches/oTsFO63lk

Both forms still work for backwards compatibility.

The rules currently are this:

• You can replace a callback with a .begin()/.end() block
• To access the inputs of the hook:
  • If the input is a single object, then you can access its properties on the hook object
  • Otherwise, the name of each argument can be accessed on the hook object
• To output a value:
  • If the hook takes in and returns the same object type (allowing you to access or modify it), you can just reassign the input properties
  • If the hook returns a new value as output, you can assign to the .result property of the hook

Some other potential thoughts that we could try out:

• Rather than accessing properties on the hook itself, e.g. getPixelnputs.normal, should we make a global inputs that aliases the hook within its begin/end so you could write inputs.normal?
• Should we auto-alias get*-prefixed hooks, like getPixelInputs, to pixelInputs so it reads more clearly in this form?
• With the current rules, since a filter shader's getColor hook has two arguments, inputs and canvasContent, the properties of inputs aren't directly accessible, e.g. you'd have to do getColor.inputs.texCoord rather than just getColor.texCoord. So maybe we'd need to update the rules to give direct access to properties when there's only one object input? (I just don't want name clashes if someone makes a hook that takes two object arguments of the same type.)

PR Checklist

• npm run lint passes
• Inline reference is included / updated
• Unit tests are included / updated

[ksen0]

Thanks for this @davepagurek !

A few thoughts:

If the hook returns a new value as output, you can assign to the .result property of the hook

After fiddling with the sketch a little, I'm wondering if there's a more p5-feeling alternative for assigning .result? Though there's the begin/end pattern in framebuffer, I'm not remembering anything like .result = anywhere.

getFinalColor.begin()
let myNewColor = mix(
  [1, 1, 1, 1],
  getFinalColor.color,
  abs(dot(myNormal, [0, 0, 1]))
);
getFinalColor.set(myNewColor);
getFinalColor.end()

Is one idea, what do you think?

Should we auto-alias get*-prefixed hooks, like getPixelInputs, to pixelInputs so it reads more clearly in this form?

Not sure I understand this, is the code below interpreting the alias idea correctly?

baseMaterialShader().modify(() => {
  let myNormal = sharedVec3()
  
  pixelInputs.begin()
  myNormal = pixelInputs.normal
  pixelInputs.end()
  
  finalColor.begin()
  finalColor.result = mix(
    [1, 1, 1, 1],
    finalColor.color,
    abs(dot(myNormal, [0, 0, 1]))
  );
  finalColor.end()
});

If yes, then I support it, because it more closely matches the begin/end pattern on framebuffer.

Rather than accessing properties on the hook itself, e.g. getPixelnputs.normal, should we make a global inputs that aliases the hook within its begin/end so you could write inputs.normal? ...

Same here, is the code below interpreting the alias idea correctly?

  let myNormal = sharedVec3()
  
  getPixelInputs.begin()
  myNormal = inputs.normal // is this what you meant?
  getPixelInputs.end() // if so, is begin/end here this needed?
  
  getFinalColor.begin()
  getFinalColor.result = mix(
    [1, 1, 1, 1],
    inputs.color, // does this make sense too or no?
    abs(dot(myNormal, [0, 0, 1]))
  );
  getFinalColor.end()
});

Putting it together, it seems much more p5-like:

let myNormal = sharedVec3()

pixelInputs.begin()
myNormal = inputs.normal
pixelInputs.end()

finalColor.begin()

let myNewColor = mix(
  [1, 1, 1, 1],
  inputs.color,
  abs(dot(myNormal, [0, 0, 1]))
);

finalColor.set(myNewColor);
finalColor.end()

[davepagurek]

Returning results

I like the .set() idea! I'll take a crack at implementing that later.

Renaming get* prefixes

That is what it would look like, yes!

Global inputs

That's also how it would work, yep! The possible downsides to consider would be:

• Is inputs too common of a name / would this be clashing with variables users declare?

• Is there any confusion around reusing of property names across hooks? e.g. currently, it's expected that normal is different in these two contexts, which I like:

  objectInputs.begin()
  // The normal at this stage has not had any transformations applied
  objectInputs.position += objectInputs.normal * 2
  objectInputs.end()
  
  pixelInputs.begin()
  // The normal at this stage has had all transformations applied now
  pixelInputs.color = abs(pixelInputs.normal)
  pixelInputs.end()

  vs with a single global inputs, it may be a bit less clear that they are different values, but the code is a bit simpler:

  objectInputs.begin()
  // The normal at this stage has not had any transformations applied
  inputs.position += inputs.normal * 2
  objectInputs.end()
  
  pixelInputs.begin()
  // The normal at this stage has had all transformations applied now
  inputs.color = abs(inputs.normal)
  pixelInputs.end()

  I guess you'd just have to know that calling .begin() on a hook will update inputs? As far as precedent goes, I think this would be a new pattern. e.g. even though framebuffer.begin() begins capturing drawing output, it doesn't change globals like width and height -- you'd still access and set properties of the framebuffer directly like framebuffer.width.

[ksen0]

Thanks @davepagurek ! Based on the discord chat, here is another potential idea which assumes that we wouldn't want to support interleaving or nesting of the hooks, and that there's just one hook at a time, and that aside from hook definitions, there's maybe some declarations in the beginning but no other code.

Starting with this begin/end example:

baseMaterialShader().modify(() => {

    let myNormal = sharedVec3()
    
    pixelInputs.begin()
    myNormal = inputs.normal
    pixelInputs.end()
    
    finalColor.begin()
    
    let myNewColor = mix(
      [1, 1, 1, 1],
      inputs.color,
      abs(dot(myNormal, [0, 0, 1]))
    );
    
    finalColor.set(myNewColor);
    finalColor.end()

});

What about something like:

baseMaterialShader().modify(() => {

    let myNormal = sharedVec3()
    
    Strands.hookMode(PIXEL_INPUTS);
    myNormal = Strands.inputs.normal // Potential for confusion?
    
    Strands.hookMode(FINAL_COLOR);
    // instead of begin/end, it's a mode? but it's basically both end and begin?
    
    let myNewColor = mix(
      [1, 1, 1, 1],
      inputs.color,
      abs(dot(myNormal, [0, 0, 1]))
    );
    
    Strands.finalColor.set(myNewColor); 
});

In the example above I was also thinking about "Is inputs too common of a name / would this be clashing with variables users declare?" and wondering if there's value to a Strands class, which would also improve strands visibility in the docs. Is this going too far?

[davepagurek]

I think using a mode does make the changes in input's interpretation more consistent, e.g. how we interpret an angle number also changes based on a mode. Although because it's additionally affecting control flow it reads a bit more confusing to me than usage of angleMode, which does less. I guess the control flow aspect is sort of unavoidable given that you're not actually writing a single shader program, but rather are modifying specific hooks in a base shader?

Looking back at one of the original inspirations, the Luma Gaussian Splats API (see the Custom Shaders section here https://lumalabs.ai/luma-web-library), that still feels a lot clearer to me, but maybe also because it's so upfront about the fact that it's just letting you modify little bits of a shader. It feels like when you're juggling inputs from multiple parts of the shader, having that control flow be more explicit helps with readability? But then for other cases where you're not doing that (e.g. a filter shader where you're always filling out only the getColor hook) the control flow feels redundant. Not sure where that leaves me yet -- maybe there should be a different syntax for modifying a shader with just one hook? I'll continue to give it some thought.