[spine-ts / webgl] optimize rendering multiple instances

MikalDev

In the Construct 3 plugin which uses spine-ts / webgl, I implemented a batch rendering of all skeleton instances, to greatly reduce context switch cost between C3 renderer and the Spine renderer.

I wanted to see if anyone had suggestions on improving the render/draw loop I implemented. My first idea was to try to make the polygon batcher batch all the skeletons, since we are using the same basic parameters for all the skeleton instances. However the issue is that we will likely change the mvp for each instance based on some resizing we are doing and the instances could be from a mix of different original skeletons. I know I am just scratching the surface of spine-ts / webgl, so I imagine there are at least a few ways to improve the current method. Thanks in advance for the help.

// Per instance render
for (const uid in skeletonInstances)
{
    const skeletonInstance = skeletonInstances[uid];
    if (skeletonInstance.initialized)
    {
        const bounds = skeletonInstance.skeletonInfo.bounds;
        // Render to our targetTexture by binding the framebuffer to the SpineFB texture
        gl.bindFramebuffer(gl.FRAMEBUFFER, skeletonInstance.spineFB);

    // Set viewport
    gl.viewport(0, 0, bounds.size.x, bounds.size.y);

    // Set proper webgl blend for Spine render
    gl.enable(gl.BLEND);
    gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA);

    gl.bindTexture(gl.TEXTURE_2D, null);        
    gl.bindBuffer(gl.ARRAY_BUFFER, null);
    gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, null);

    // Bind the shader and set the texture and model-view-projection matrix.
    this.shader.bind();
    this.shader.setUniformi(spine.webgl.Shader.SAMPLER, 0);
    // Resize 
    this.resize(bounds, skeletonInstance.skeletonScale);
    this.shader.setUniform4x4f(spine.webgl.Shader.MVP_MATRIX, this.mvp.values);
    
    // Start the batch and tell the SkeletonRenderer to render the active skeleton.
    this.batcher.begin(this.shader);
    
    // Apply vertex effect
    this.renderer.vertexEffect = null;

    gl.clearColor(0, 0, 0, 0);
    gl.clear(gl.COLOR_BUFFER_BIT);

    // Render
    this.renderer.premultipliedAlpha = this.premultipliedAlpha;
    this.renderer.draw(this.batcher, skeletonInstance.skeletonInfo.skeleton);
    this.batcher.end();
    this.shader.unbind();
}
}

Also, the improvement from just changing to this batcher/loop was significant _4X higher performance for the C3 plugin.

Example below:

Mario

Having to resize the skeletons via the MVP is a bit unfortunate, and what's worse is the need to render each instance into its own FBO. I guess that's simply how it's done in Construct? You could still do a few optimizations.

You only need to bind the share once, outside the loop.
If all skeletons share the same texture atlas, and the atlas only has a single page, you only need to bind the corresponding texture once.

MikalDev

Thanks for the feedback, I think there may be a way around doing the resize via the MVP, I will need to take a look at that. In terms of the FBO, that's how I am doing it right now, because it's expensive to save/restore the webgl state in C3. If we rendered directly to the canvas, we'd need to do it in order of the C3 draw, so we could be switching back and forth between C3 render of generic sprites and Spine render of characters. As a work around we render to a texture, which the C3 render can then use to render in the correct order (and also add C3 effects, like special shaders, etc.) There are definitely tradeoffs in this approach, so I've been trying some different methods along the way.

Sometimes there may be multiple atlas pages (e.g. multiple png textures). A question about that, it seems like multiple textures are loaded via:

	assetManager.loadTexture(DEMO_NAME, textureLoader, "atlas1.png");
	assetManager.loadTexture(DEMO_NAME, textureLoader, "atlas12.png");
	assetManager.loadText(DEMO_NAME, "atlas1.atlas");
	assetManager.loadJson(DEMO_NAME, "demos.json");

The atlas1.atlas then refers to the different pages by name. In my case the name for atlas1.png and atlas12.png will be a blob in C3 preview mode (for example: blob:https://preview.construct.net/006fb36f-1dd7-4afc-906e-b6d19837b2dc) Will this cause an issue with multiple pages? I am not sure how the Spine renderer will identify which texture to reference from the atlas (unless it's order based, not name based.) I have been experimenting, but no luck so far.

Mario

Hm, the WebGL state is "global" to the context, so using an FBO doesn't help with eliminating/reducing the number of states that need to be restored. But ya, rendering to an FBO which you can later plugin into the C3 rendering pipeline probably makes sense.

Quick side note: you can use assetManager.loadTextureAtlas() which will load the texture atlas pages automatically.

The resolution of image resources is done by name, relative to the .atlas file. E.g. spine-runtimes/owl-pma.atlas at 3.8

The owl-pma.atlas references owl-pma.png. The AssetManager will thus check for owl-pma.png in the "parent directory" of owl-pma.atlas.

You can also just set raw data URIs for a path on AssetManager, see spine-runtimes/AssetManager.ts at 3.8

MikalDev

Thanks for the ideas, I will review the Spine atlas info.

In terms of rendering to FBO vs state, I didn't really provide the details and perhaps 'state savings' is not really correct. What I was originally doing was having Spine render to a separate canvas for each Spine object (and gl context) and then transferring that canvas to a texture in the C3 canvas context (to integrate in with the C3 render), that process took a long time, so I switched over to rendering to a texture using the same gl context as the final canvas and it gave a nice boost in performance. I imagine the worse original perf was due to a few of the different operations required.

The real 'state' savings I get is just from batching all the Spine rendering vs doing it just when the Spine object needs to be rendered in the C3 draw order. Each time I switch between Spine render and C3 render I need to do a bunch of getParameter calls (which cause a CPU/GPU sync) to save the C3 render state. So, changing from doing that once per frame vs once per object was a pretty big improvement. I am thinking about where I can do the Spine render in the C3 general pipeline, to see if I can cache or remove those once per frame getParameter calls.

Mario

Ah, that makes sense. It's a bit weird that C3 would let you insert custom drawing but not make sure the state is (re-)set properly before and after your custom drawing calls. Invoking getParameter() will definitely kill anything under the sun. C3 likely internally caches the GL state, so it itself doesn't have to reset state that's already set appropriately. Maybe that can be polled somehow instead of going to the WebGL context and asking for the state?

MikalDev

That's a good idea to see if I can find if C3 is caching the webgl state somewhere, I will have to hunt for that! In terms of C3 letting me insert custom drawing, well it really doesn't 🙂 I am basically 'interrupting' the C3 engine to make this happen, it's not a direct part of the C3 SDK to allow for this (though there is direct suppor for a dynamic texture and updating a texture for a C3 object/instance.) The C3 developer has given some unofficial support in terms of ideas about what to save off and how to handle it, but it's not a typical C3 addon/plugin.