How to create a 2d clipping mask in webgpu?

Question

I've been investigating a webgpu way to create a clipping mask.

Here is what I tried:

const pipeline1 = device.createRenderPipeline({
  vertex: {
    module: basicShaderModule,
    entryPoint: 'vertex_main',
    buffers: [{
      attributes: [{
        shaderLocation: 0,
        offset: 0,
        format: 'float32x2'
      }],
      arrayStride: 8,
      stepMode: 'vertex'
    }],
  },
  fragment: {
    module: basicShaderModule,
    entryPoint: 'fragment_main',
    targets: [{ format }]
  },
  primitive: {
    topology: 'triangle-strip',
  },
  layout: 'auto',
})
passEncoder.setPipeline(pipeline1);
const uniformValues1 = new Float32Array(4)
uniformValues1.set([1, 0, 0, 1], 0)
const uniformBuffer1 = device.createBuffer({
  size: uniformValues1.byteLength,
  usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
});
device.queue.writeBuffer(uniformBuffer1, 0, uniformValues1)
passEncoder.setBindGroup(0, device.createBindGroup({
  layout: pipeline1.getBindGroupLayout(0),
  entries: [
    {
      binding: 0, resource: {
        buffer: uniformBuffer1
      }
    },
  ],
}));
const vertices1 = new Float32Array([-1, -1, 1, -1, 1, 1])
const verticesBuffer1 = device.createBuffer({
  size: vertices1.byteLength,
  usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST,
})
device.queue.writeBuffer(verticesBuffer1, 0, vertices1, 0, vertices1.length)
passEncoder.setVertexBuffer(0, verticesBuffer1);
passEncoder.draw(3);

const pipeline2 = device.createRenderPipeline({
  vertex: {
    module: basicShaderModule,
    entryPoint: 'vertex_main',
    buffers: [{
      attributes: [{
        shaderLocation: 0,
        offset: 0,
        format: 'float32x2'
      }],
      arrayStride: 8,
      stepMode: 'vertex'
    }],
  },
  fragment: {
    module: basicShaderModule,
    entryPoint: 'fragment_main',
    targets: [{ format }]
  },
  primitive: {
    topology: 'line-strip',
  },
  layout: 'auto',
})
passEncoder.setPipeline(pipeline2);
const uniformValues2 = new Float32Array(4)
uniformValues2.set([0, 1, 0, 1], 0)
const uniformBuffer2 = device.createBuffer({
  size: uniformValues2.byteLength,
  usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
});
device.queue.writeBuffer(uniformBuffer2, 0, uniformValues2)
passEncoder.setBindGroup(0, device.createBindGroup({
  layout: pipeline2.getBindGroupLayout(0),
  entries: [
    {
      binding: 0, resource: {
        buffer: uniformBuffer2
      }
    },
  ],
}));
const vertices2 = new Float32Array([0, -1, 1, -1, -1, 1, 0, -1])
const verticesBuffer2 = device.createBuffer({
  size: vertices2.byteLength,
  usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST,
})
device.queue.writeBuffer(verticesBuffer2, 0, vertices2, 0, vertices2.length)
passEncoder.setVertexBuffer(0, verticesBuffer2);
passEncoder.draw(4);

passEncoder.end();
device.queue.submit([commandEncoder.finish()]);

The code above draws a path and a content, I expect the content clipped by the path.

Here is current result:

Here is expected result:

I ignored some common code because stackoverflow complains "It looks like your post is mostly code; please add some more details."

Answer 1

The are infinite ways to clip. A few off the top of my head

• Clip via vertex math intersection
• Clip with a depth texture
• Clip with a stencil texture
• Clip with an alpha mask
• Clip with area intersection (like SDF, or CSG)

Via alpha mask has the advantage that your mask can blend.

In any case, clip via stencil texture means making a stencil texture, rendering the mask to it, then rendering the other things set to draw only where the mask is.

In particular, the pipeline that sets the mask would be set to something like

  const maskMakingPipeline = device.createRenderPipeline({
    ...
    fragment: {
      module,
      entryPoint: 'fs',
      targets: [],
    },
    // replace the stencil value when we draw
    depthStencil: {
      format: 'stencil8',
      depthCompare: 'always',
      depthWriteEnabled: false,
      stencilFront: {
        passOp:'replace',
      },
    },
  });

There's no targets in the fragment because we're only drawing to the stencil texture. We've set so when front facing triangles are drawn to this texture and the pass the depth test (which is set to 'always' pass), then 'replace' the stencil with the stencil reference value (we set that later)

The pipeline for drawing the 2nd triangle (the one being masked) looks like this

  const maskedPipeline = device.createRenderPipeline({
    ...
    fragment: {
      module,
      entryPoint: 'fs',
      targets: [{format: presentationFormat}],
    },
    // draw only where stencil value matches
    depthStencil: {
      depthCompare: 'always',
      depthWriteEnabled: false,
      format: 'stencil8',
      stencilFront: {
        compare: 'equal',
      },
    },
  });

The fragment.targets are now set because we want a color rendered. The depthStencil is set so pixels in front facing triangles will only draw if the stencil is 'equal' to the stencil reference value.

At draw time first we render the mask to the stencil texture

  {
    const pass = encoder.beginRenderPass({
      colorAttachments: [],
      depthStencilAttachment: {
        view: stencilTexture.createView(),
        stencilClearValue: 0,
        stencilLoadOp: 'clear',
        stencilStoreOp: 'store',
      }
    });
    // draw the mask
    pass.setPipeline(maskMakingPipeline);
    pass.setVertexBuffer(0, maskVertexBuffer);
    pass.setStencilReference(1);
    pass.draw(3);
    pass.end();
  }

The stencil was set to clear to 0 and the stencil reference is set to 1 so when this pass is done there will be 1s where we want to allow rendering

Then we render the 2nd triangle masked

  {
    const pass = encoder.beginRenderPass({
      colorAttachments: [{
        view: context.getCurrentTexture().createView(),
        clearValue: [0, 0, 0, 1],
        loadOp: 'clear',
        storeOp: 'store',
      }],
      depthStencilAttachment: {
        view: stencilTexture.createView(),
        stencilLoadOp: 'load',
        stencilStoreOp: 'store',
      }
    });
    // draw only the mask is
    pass.setPipeline(maskedPipeline);
    pass.setStencilReference(1);
    pass.setVertexBuffer(0, toBeMaskedVertexBuffer);
    pass.draw(3);

    pass.end();
  }

Here we 'load' the stencil texture back in before rendering, and set the stencil reference to 1 so we'll only draw where there are 1s in the stencil texture.

const code = `
struct VSIn {
  @location(0) pos: vec4f,
};

struct VSOut {
  @builtin(position) pos: vec4f,
};

@vertex fn vs(vsIn: VSIn) -> VSOut {
  var vsOut: VSOut;
  vsOut.pos = vsIn.pos;
  return vsOut;
}

@fragment fn fs(vin: VSOut) -> @location(0) vec4f {
  return vec4f(1, 0, 0, 1);
}
`;

(async() => {
  const adapter = await navigator.gpu?.requestAdapter();
  const device = await adapter?.requestDevice();
  if (!device) {
    alert('need webgpu');
    return;
  }

  const canvas = document.querySelector("canvas")
  const context = canvas.getContext('webgpu');
  const presentationFormat = navigator.gpu.getPreferredCanvasFormat();
  context.configure({
      device,
      format: presentationFormat,
      alphaMode: 'opaque',
  });

  const module = device.createShaderModule({code});
  const maskMakingPipeline = device.createRenderPipeline({
    label: 'pipeline for rendering the mask',
    layout: 'auto',
    vertex: {
      module,
      entryPoint: 'vs',
      buffers: [
        // position
        {
          arrayStride: 2 * 4, // 2 floats, 4 bytes each
          attributes: [
            {shaderLocation: 0, offset: 0, format: 'float32x2'},
          ],
        },
      ],
    },
    fragment: {
      module,
      entryPoint: 'fs',
      targets: [],
    },
    // replace the stencil value when we draw
    depthStencil: {
      format: 'stencil8',
      depthCompare: 'always',
      depthWriteEnabled: false,
      stencilFront: {
        passOp:'replace',
      },
    },
  });

  const maskedPipeline = device.createRenderPipeline({
    label: 'pipeline for rendering only where the mask is',
    layout: 'auto',
    vertex: {
      module,
      entryPoint: 'vs',
      buffers: [
        // position
        {
          arrayStride: 2 * 4, // 2 floats, 4 bytes each
          attributes: [
            {shaderLocation: 0, offset: 0, format: 'float32x2'},
          ],
        },
      ],
    },
    fragment: {
      module,
      entryPoint: 'fs',
      targets: [{format: presentationFormat}],
    },
    // draw only where stencil value matches
    depthStencil: {
      depthCompare: 'always',
      depthWriteEnabled: false,
      format: 'stencil8',
      stencilFront: {
        compare: 'equal',
      },
    },
  });

  const maskVerts = new Float32Array([-1, -1, 1, -1, 1, 1]);
  const toBeMaskedVerts = new Float32Array([0, -1, 1, -1, -1, 1]);

  const maskVertexBuffer = device.createBuffer({
    size: maskVerts.byteLength,
    usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST,
  });
  device.queue.writeBuffer(maskVertexBuffer, 0, maskVerts);
  const toBeMaskedVertexBuffer = device.createBuffer({
    size: toBeMaskedVerts.byteLength,
    usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST,
  });
  device.queue.writeBuffer(toBeMaskedVertexBuffer, 0, toBeMaskedVerts);

  const stencilTexture = device.createTexture({
    format: 'stencil8',
    size: [canvas.width, canvas.height],
    usage: GPUTextureUsage.RENDER_ATTACHMENT,
  });

  const encoder = device.createCommandEncoder();
  {
    const pass = encoder.beginRenderPass({
      colorAttachments: [],
      depthStencilAttachment: {
        view: stencilTexture.createView(),
        stencilClearValue: 0,
        stencilLoadOp: 'clear',
        stencilStoreOp: 'store',
      }
    });
    // draw the mask
    pass.setPipeline(maskMakingPipeline);
    pass.setVertexBuffer(0, maskVertexBuffer);
    pass.setStencilReference(1);
    pass.draw(3);
    pass.end();
  }
  {
    const pass = encoder.beginRenderPass({
      colorAttachments: [{
        view: context.getCurrentTexture().createView(),
        clearValue: [0, 0, 0, 1],
        loadOp: 'clear',
        storeOp: 'store',
      }],
      depthStencilAttachment: {
        view: stencilTexture.createView(),
        stencilLoadOp: 'load',
        stencilStoreOp: 'store',
      }
    });
    // draw only the mask is
    pass.setPipeline(maskedPipeline);
    pass.setStencilReference(1);
    pass.setVertexBuffer(0, toBeMaskedVertexBuffer);
    pass.draw(3);

    pass.end();
  }

  device.queue.submit([encoder.finish()]);


})();

<canvas></canvas>

Just like we set the stencil compare to 'equal'. We could also mask using the depth compare and a depth texture.

Steps:

1. Clear a depth texture to 1.0.

2. Draw the mask into a depth texture with its Z value set to something, for example 0.0 (which is what we were already doing).

   This will end up with 0s in the depth texture where the first thing we drew is and 1s everywhere else.

3. Draw the thing we want masked with the depth compare set to 'equal' and its Z value also 0.0 (again, what we were already doing).

   We'll end up only drawing where 0.0 is in the depth texture

const code = `
struct VSIn {
  @location(0) pos: vec4f,
};

struct VSOut {
  @builtin(position) pos: vec4f,
};

@vertex fn vs(vsIn: VSIn) -> VSOut {
  var vsOut: VSOut;
  vsOut.pos = vsIn.pos;
  return vsOut;
}

@fragment fn fs(vin: VSOut) -> @location(0) vec4f {
  return vec4f(1, 0, 0, 1);
}
`;

(async() => {
  const adapter = await navigator.gpu?.requestAdapter();
  const device = await adapter?.requestDevice();
  if (!device) {
    alert('need webgpu');
    return;
  }

  const canvas = document.querySelector("canvas")
  const context = canvas.getContext('webgpu');
  const presentationFormat = navigator.gpu.getPreferredCanvasFormat();
  context.configure({
      device,
      format: presentationFormat,
      alphaMode: 'opaque',
  });

  const module = device.createShaderModule({code});
  const maskMakingPipeline = device.createRenderPipeline({
    label: 'pipeline for rendering the mask',
    layout: 'auto',
    vertex: {
      module,
      entryPoint: 'vs',
      buffers: [
        // position
        {
          arrayStride: 2 * 4, // 2 floats, 4 bytes each
          attributes: [
            {shaderLocation: 0, offset: 0, format: 'float32x2'},
          ],
        },
      ],
    },
    fragment: {
      module,
      entryPoint: 'fs',
      targets: [],
    },
    // replace the depth value when we draw
    depthStencil: {
      format: 'depth24plus',
      depthCompare: 'always',
      depthWriteEnabled: true,
    },
  });

  const maskedPipeline = device.createRenderPipeline({
    label: 'pipeline for rendering only where the mask is',
    layout: 'auto',
    vertex: {
      module,
      entryPoint: 'vs',
      buffers: [
        // position
        {
          arrayStride: 2 * 4, // 2 floats, 4 bytes each
          attributes: [
            {shaderLocation: 0, offset: 0, format: 'float32x2'},
          ],
        },
      ],
    },
    fragment: {
      module,
      entryPoint: 'fs',
      targets: [{format: presentationFormat}],
    },
    // draw only where stencil value matches
    depthStencil: {
      format: 'depth24plus',
      depthCompare: 'equal',
      depthWriteEnabled: false,
    },
  });

  const maskVerts = new Float32Array([-1, -1, 1, -1, 1, 1]);
  const toBeMaskedVerts = new Float32Array([0, -1, 1, -1, -1, 1]);

  const maskVertexBuffer = device.createBuffer({
    size: maskVerts.byteLength,
    usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST,
  });
  device.queue.writeBuffer(maskVertexBuffer, 0, maskVerts);
  const toBeMaskedVertexBuffer = device.createBuffer({
    size: toBeMaskedVerts.byteLength,
    usage: GPUBufferUsage.VERTEX | GPUBufferUsage.COPY_DST,
  });
  device.queue.writeBuffer(toBeMaskedVertexBuffer, 0, toBeMaskedVerts);

  const depthTexture = device.createTexture({
    format: 'depth24plus',
    size: [canvas.width, canvas.height],
    usage: GPUTextureUsage.RENDER_ATTACHMENT,
  });

  const encoder = device.createCommandEncoder();
  {
    const pass = encoder.beginRenderPass({
      colorAttachments: [],
      depthStencilAttachment: {
        view: depthTexture.createView(),
        depthClearValue: 1,
        depthLoadOp: 'clear',
        depthStoreOp: 'store',
      }
    });
    // draw the mask
    pass.setPipeline(maskMakingPipeline);
    pass.setVertexBuffer(0, maskVertexBuffer);
    pass.draw(3);
    pass.end();
  }
  {
    const pass = encoder.beginRenderPass({
      colorAttachments: [{
        view: context.getCurrentTexture().createView(),
        clearValue: [0, 0, 0, 1],
        loadOp: 'clear',
        storeOp: 'store',
      }],
      depthStencilAttachment: {
        view: depthTexture.createView(),
        depthLoadOp: 'load',
        depthStoreOp: 'store',
      }
    });
    // draw only the mask is
    pass.setPipeline(maskedPipeline);
    pass.setVertexBuffer(0, toBeMaskedVertexBuffer);
    pass.draw(3);

    pass.end();
  }

  device.queue.submit([encoder.finish()]);


})();

<canvas></canvas>