MikalDev

I am looking at changing the runtime shader to a custom shader (indexed color per character at first then per slot later). It will just be a change to the fragment shader with a largish set of uniforms for just the fragment shader (static per pixel, so I do not need to set them on the vertices, like is done with tint.)

My thought right now is to copy and add to the two color tint shader defined in https://github.com/EsotericSoftware/spine-runtimes/blob/3.8/spine-ts/webgl/src/Shader.ts . I will then use setUniform3f to set the uniforms on a per character basis (they will be vec3 RGB values.) I may also look at using a second texture as a color table (which will still require a custom shader, but now I will also need to create and load the the color table texture and set the color table texture uniform.)

I am wondering if you have other suggestions, so that I don't need to change the spine-ts runtime source (so I can stay current with the runtime.)

---

I did a hack version of Shader.ts to test it out and it seems to work ok. My test shader is using a bunch of branching, but I may move to using a second 1D texture as a color lookup depending on perf bottlenecks.

SpineIndex.gif


I am still hoping for some suggestions on how to integrate this smoothly with the spine runtime, instead of myself changing the runtime (I might just create a new shader and add a PR, but I am not sure if it will get accepted to add a new index color shader. I would like to hear comments on that...) I guess I could add another class file which extends the spine shader class...

My changes:
(this is my draw code)
// Set palette in shader
const PALETTE_ENTRIES = 8;
if (tickCount%30 == 0)
{
for (let i=0;i<PALETTE_ENTRIES;i++)
{
let uniformName = 'color'+i;
this.shader.setUniform4f(uniformName, Math.random(), Math.random(), Math.random(), 1.0);
}
}

// Start the batch and tell the SkeletonRenderer to render the active skeleton.
this.batcher.begin(this.shader);

// Apply vertex effect
this.renderer.vertexEffect = null;

gl.clearColor(0, 0, 0, 0);
gl.clear(gl.COLOR_BUFFER_BIT);

// Render
this.renderer.premultipliedAlpha = premultipliedAlpha;
this.renderer.draw(this.batcher, skeletonInstance.skeletonInfo.skeleton);
These are my changes to Shader.ts:
let fs = `
#ifdef GL_ES
#define LOWP lowp
precision mediump float;
#else
#define LOWP
#endif
varying LOWP vec4 v_light;
varying LOWP vec4 v_dark;
varying vec2 v_texCoords;
uniform sampler2D u_texture;
uniform vec4 color0;
uniform vec4 color1;
uniform vec4 color2;
uniform vec4 color3;
uniform vec4 color4;
uniform vec4 color5;
uniform vec4 color6;
uniform vec4 color7;

void main () {
vec4 texColor = texture2D(u_texture, v_texCoords);
int index = int(texColor.r * 7.0);
if (texColor.a == 0.0)
{
// transparent, ignore index
} else if (index == 0)
{
texColor = color0;
} else if (index == 1)
{
texColor = color1;
} else if (index == 2)
{
texColor = color2;
} else if (index == 3)
{
texColor = color3;
} else if (index == 4)
{
texColor = color4;
} else if (index == 5)
{
texColor = color5;
} else if (index == 6)
{
texColor = color6;
} else
{
texColor = color7;
}
gl_FragColor.a = texColor.a * v_light.a;
gl_FragColor.rgb = ((texColor.a - 1.0) * v_dark.a + 1.0 - texColor.rgb) * v_dark.rgb + texColor.rgb * v_light.rgb;
}
`;


---

Now I'm looking at changing the palette colors (fs uniforms) on a per slot basis and that's definitely more complicated. I see the dark/light colors are applied on a slot basis by adding them to the vertices, but I am definitely open to suggestions on the best way to apply the palette color uniforms (for the fragment shader) on a per slot basis (it looks like I might need to dig into the poly batcher and mesh classes?)
Bu mesaja eklenen dosyaları görüntülemek için gerekli yetkilere sahip değilsiniz.
MikalDev
  • Mesajlar: 75

Harald

Please note that instead of branches and 1D texture lookup you could also use a vec4 color[MAX_COLORS] array to store the color at each index, then you can access it via color[index]. This would require to change the code and add the method to set the uniform array however.

The best improvement performance-wise would anyway be to move the color lookup from the fragment shader to the vertex shader. Even when coloring attachments separately, all vertices of an attachment share the same color.
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

Thanks, I was trying the array, but in my webgl context under construct 3, arrays using non loop index variables for indices are not possible (though I could do a looping workaround, which might be ok to do.) https://stackoverflow.com/questions/6247572/variable-array-index-not-possible-in-webgl-shaders

I am using index color for the source texture on a per pixel basis (e.g. output color based on palette color referred to by the source texture color), so I think that needs to be done on the FS, it can't be done on the VS, since it will be different for each pixel/color.
MikalDev
  • Mesajlar: 75

Harald

Sorry, I misunderstood your use-case. I assumed you were only setting colors for a few attachments, like coloring the hair black or brown, the trousers black, blue or green, and so on. If you really want to map single colors of an attachment texture one by one, then a LUT texture will most likely be the best way.
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

Thanks, yes I understand that may be best. My real question is not about the shader itself, but how to best integrate it into the runtime. Right now I just plan on editing the Shader.ts for the shader change and separately adding a new palette class / object to the slots I want to apply this to. I thought I could add an object to the slot itself, but that seems to be recreated elsewhere (perhaps in update?.) So I will create an object with slot names as keys and the palette objects as values. I will create the class outside of the runtime and just add the palette object during runtime. I will also dive into skeleton renderer code and before each slot is rendered, I will apply the slot palette uniforms (either color values or a LUT texture) before the slot is rendered.

For example here at the bottom of this code from SkeletonRenderer.ts
for (var i = 0, n = drawOrder.length; i < n; i++) {
var clippedVertexSize = clipper.isClipping() ? 2 : vertexSize;
var slot = drawOrder[i];
if (!slot.bone.active) {
clipper.clipEndWithSlot(slot);
continue;
}
if (slotRangeStart >= 0 && slotRangeStart == slot.data.index) {
inRange = true;
}
if (!inRange) {
clipper.clipEndWithSlot(slot);
continue;
}
if (slotRangeEnd >= 0 && slotRangeEnd == slot.data.index) {
inRange = false;
}
var attachment = slot.getAttachment();
var texture = null;
slotPalettes[slot.data.name].apply(batcher.shader); // [i]CHANGE[/i] apply palette data to the shader (I will get a reference to the shader and apply uniforms as needed. I need to figure out a way to include slotPalettes with each skeleton instance.


---

I got per slot palette color working. The one thing I did have to do was to add a batcher.flush() before changing the uniform, so I am breaking the batch for the mesh.draw(). However, I don't see a way around that, I need to (potentially) change the shader uniforms before each draw (with either the colors of the palette or a different texture LUT). Perhaps I could encode the entire palette into vertices though, like is done with color and darkColor, would require more changes to support more vertices, hmmm, something to think about.

Per slot 8 color palette working with Construcct 3 ACEs (enable, set default color palette, set palette index color). Functionality is in, now looking at perf. Each slot has a separate palette which I am setting randomly based on keypress.

SpineSlotIndex.gif


---

First look at performance is not great. It looks like having a separate dedicated palette per slot which requires FS uniforms to be updated per slot requires a flush of the Spine draw batcher (so the new palette can be set.) So, I think I will probably need to go the route of having a larger palette lookup texture per instance, that can store multiple palettes and then encode which one of those palettes is used per slot through a vertex, so I don’t need to do a batcher flush each time. We should be able to have enough palettes through this technique for 50+ slots w/ 16 color palettes.
Bu mesaja eklenen dosyaları görüntülemek için gerekli yetkilere sahip değilsiniz.
MikalDev
  • Mesajlar: 75

Harald

MikalDev yazdı:I got per slot palette color working. The one thing I did have to do was to add a batcher.flush() before changing the uniform, so I am breaking the batch for the mesh.draw().
This is not a good idea. Each skeleton will then issue e.g. 50 draw calls instead of a single one.
MikalDev yazdı:First look at performance is not great.
This is not a big surprise. I didn't assume that you wanted to use separate pallettes per slot.
MikalDev yazdı:So, I think I will probably need to go the route of having a larger palette lookup texture per instance, that can store multiple palettes and then encode which one of those palettes is used per slot through a vertex, so I don’t need to do a batcher flush each time. We should be able to have enough palettes through this technique for 50+ slots w/ 16 color palettes.
Yes, this is the way to go. A 256x256 LUT already allows for 65.536 colors, so you could easily have 256 slots with 256 colors.
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

Harald - thanks for the reply. Learning through experimentation here and I appreciate the comments and review. This is an evolving feature for our game, so we started with per skeleton instance palette and now we are looking to do per slot palette, so it is a change from what I first was looking at.

More background: the main reason for it is to allow for player customization of skins through changing color palette per slot (our game is a MMO, so lots of player customizations, skin types and colors), right now we are using two color tint which is good, but we are looking for more flexibility.
MikalDev
  • Mesajlar: 75

Harald

You're welcome. That sounds really nice! It reminds me of some D&D RPGs by SSI in the era of ~1990 where you could customize the characters of your party by setting colors for hair, body, trousers, armour, and so on - just with the difference that in those 16x16 sprites it ended up changing only 5 pixels :nerd:. Looking forward to seeing what you make of this feature, would be very cool if you could later share some screenshots once you're ready! :cooldoge:
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

SSI - that's great blast from the past!

Here's a quick example where we are using two color tint per slot currently, but we are imagining we can do more interesting customization via customized palettes per slot.

All the characters are Spine instances. Hair color, shields, skin, eyes, clothing, etc. all can have different two color tint per slot.

Screen Shot 2021-03-11 at 11.02.45 AM.png
Bu mesaja eklenen dosyaları görüntülemek için gerekli yetkilere sahip değilsiniz.
MikalDev
  • Mesajlar: 75

Harald

Awesome, this looks very cool already! Thanks for sharing! 8)
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

Got it all working, looking for optimizations. Each slot can use one of 64 palettes. Total of 64x32 palette (LUT texture) used (smaller is better for transferring palette data along with player data.) Using 'vertex' to control shader (if palette mode or not and which palette of the 64 to use.) Full batching can be used again. Also have default palettes that can be used.

ACEs:
Enable palette color (for Spine instance)
Set all palette colors from string (set the entire 64x32 palettes from a hex string)
Set palette to default colors (set a palette entry to default colors)
Set slot palette (set a slotName to use palette entry)
Set palette color (set a palette entry index color to RGBA value)
AllPaletteColorString (get the hex string of the entire 64x32 palettes)

Example random palette used on each character (w/ live updates of palettes.)

Screen Shot 2021-03-13 at 12.24.13 AM.png
Bu mesaja eklenen dosyaları görüntülemek için gerekli yetkilere sahip değilsiniz.
MikalDev
  • Mesajlar: 75

Harald

Cool, looks very promising! Thanks for sharing! 8)
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

SpineSwordPalette.gif

Got some palette color cycling working with per slot palette (slowed down attack animation).
Bu mesaja eklenen dosyaları görüntülemek için gerekli yetkilere sahip değilsiniz.
MikalDev
  • Mesajlar: 75

Harald

Very cool, thanks for sharing! 8) That's some high-tech blade the king is wielding. Now we need some waterfall-creatures! :nerd:
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451

MikalDev

No waterfall yet, but how about fire crown?

FireCrown.gif
Bu mesaja eklenen dosyaları görüntülemek için gerekli yetkilere sahip değilsiniz.
MikalDev
  • Mesajlar: 75

Harald

Haha, even better! Thats very cool indeed! :D
Kullanıcı avatarı
Harald

Harri
  • Mesajlar: 4451


Dön Runtimes