Texture packer - sRGB?

Ffossy · 12 Tem 2018

When exporting to png, what color space does texture packer export to?

There is no sRGB chunk, nor any ICC chunk.

If it is linear (not sRGB), which seems to be the case: then RGBA8888 has too little precision. (Will it be possible to export to eg RGBA at 16bpp?)

If it is non-linear (sRGB): how is alpha premultiplied?

To be correct, it should be:

f(R * a), f(G * a),f(B * a),a

That is, it should be multiplied by f(a) not by a. Where f takes linear value to sRGB. (Otherwise it would mix linear and non-linear color space and blending would be off).

Nate · 12 Tem 2018

The RGB is linear. Are you speculating or do you have a concrete example of a precision problem? If so, do you see the problem when exporting an image without texture packing?

FWIW, the texture packer in Spine is the same I contributed to libgdx. The part that applies PMA is here:
https://github.com/libgdx/libgdx/blob/master/extensions/gdx-tools/src/com/badlogic/gdx/tools/texturepacker/TexturePacker.java#L294
This is using Java's AWT API (canvas is a java.awt.image.BufferedImage).

Ffossy · 12 Tem 2018

You can see examples and details of why it is bad to store linear data in 8bpp: https://blog.demofox.org/2018/03/10/dont-convert-srgb-u8-to-linear-u8/

Would it be possible to add a 16bpp mode (or float)? (I don't mind having it exported linear, but not at 8bpp)

Nate · 12 Tem 2018

Before we try to fix something, we need to see it is broken else we can't verify we've fixed it. Can you share a Spine project which shows data loss when exported? I don't see a problem with this project. The input is a frame from the GIF on the page you linked, saved as PNG using Photoshop Save for Web with Convert to sRGB checked. Here is an image that is an export of the skeleton's pose (rendered to an FBO then saved as PNG). Here is an image that is an atlas page (created by Spine's texture packer).

Ffossy · 13 Tem 2018

How is it possible, that no information is lost, when exporting linear 8 bit? For example: pixel value 1 in sRGB cannot be represented at all in 8 bit linear, it is impossible. Yet.. it looks correct?

Well, now I know why, it is because neither TexturePacker nor Spine does gamma correction!

Spine just assumes it is handed linear data regardless of gamma information in the png files (haven't tested other formats).

I'm attaching a Spine project to prove this.

There is ramp_linear.png and ramp_srgb.png. They are badly named as they are not ramps. In any case: their pixel data is tiles of either value 1 or 0, but one is linear (gamma = 1) and one is sRGB (roughly 2.2, but definitely not exactly, one should not approximate it with 2.2). If spine did things correctly, it would linearize the sRGB data and then pixel value 1 in the sRGB would NOT be pixel value 1 in linear space.

Furthermore, I also included this picture: https://superuser.com/questions/579216/why-does-this-png-image-display-differently-in-chrome-firefox-than-in-safari-a/579220
And it shows as an apple in Spine

I think it would be good if: Spine showed things gamma correctly, and that TexturePacker also handles gamma correctly, and once it does so - offers a 16bit export mode.

Oh, and to clarify: someone who is feeding sRGB images to Spine, will get sRGB data from TexturePacker. But, since TexturePacker assumes it is working in linear space, it will multiply the alpha linearly - with something that is non-linear. Someone who is feeding linear data to Spine, will get precision loss (when they convert their sRGB properly to linear prior to sending it to Spine). There is no easy way around it, afaik.

Nate · 14 Tem 2018

The apple/pear image is really neat! :nerd: I'm all for improving Spine, but I am still wondering if there is an actual problem you are having that you would like solved? Since test images are required it seems that you are pursuing this solely on principle? That's perfectly fine of course, but we have a lot to work on and try to prioritize based on users needs.

Spine loads all images using sRGB, eg:

If changed to use linear RGB then it's definitely wrong:

I did find that we are loading the image data, then applying PMA. You may be right that this is incorrect. It sounds logical, though it has never caused a noticeable problem. Do you have any test images that show PMA is incorrect? This would allow us to validate the fix. The images that Spine displays in the editor have PMA applied, so it should show any problems there. We have changed how PMA is applied in the editor to avoid problems. The texture packer has not yet been updated.

The ramp_srgb image has an sRGB chunk set to "perceptual":

That image should be rendered correctly by your browser as a black square. Spine loads and display it correctly, since it always uses sRGB:

The ramp_linear image has a gAMA chunk with 1:

Spine displays it incorrectly as a black square. The image loader is ignoring the gAMA chunk and blindly using sRGB:

The pear/apple image has a gAMA chunk with 0.02:

This has the same problem as the ramp_linear image, it is loaded using sRGB, not 0.02 gamma:

0.02 gamma is of course an extreme case. Using anything other than sRGB is likely rare. Photoshop Save for web doesn't write a gAMA or sRGB chunk even when Convert to sRGB is checked and Metadata is "All". It doesn't seem possible to get Photoshop to write linear RGB at all. Also, unlike a web browser, Spine is consuming images the user provides (and likely created), so the user is in control of the images. Still, we'll look into respecting the gAMA chunk.

We'll look at fixing up the texture packer to be sure there is not a problem applying PMA to sRGB. A test image showing a problem would be very helpful.

I haven't dug into output, but I believe I was wrong earlier

Spine's output is almost certainly sRGB.

I doubt many users would use 16bpp to avoid the imprecision at small RGB values using 8bpp. You are the first to ever bring this up, and even you don't seem to have a problem in a real world project. The imprecision is in small values. If PMA is what caused values to become small, then those pixels have low opacity and can barely be seen, so the tone being incorrect is unlikely to be noticeable.

If you aren't using PMA, AFAICT there isn't a problem. Likely only game projects want PMA and 16bpp wastes storage space, which is something most games want to avoid. Even for games, if it is really a problem then PMA can be avoided by using the Bleed setting, which provides nearly identical blending (though affects batching if slot additive blending is used).

Spine is using JRE APIs to load images, so it is a convoluted process, but Spine should now respect PNG sRGB and gAMA chunks (it renders slightly differently on screen versus in the screenshot):

You'll need 3.7.20-beta (not yet released). For various reasons you'll also need the latest Spine launcher, version 3.7.20 (available in a few hours, download and reinstall Spine), else you'll see an apple even with 3.7.20-beta.

Now let's see who complains that their images look funny!

I'm not sure why Spine shows only a pear while Firefox and Chrome show a pear with a faint apple. Interestingly, Windows Image Viewer shows only a pear, but when I screenshot it, I get an image like Firefox/Chrome. :o Spine's screenshot looks different (dimmer), but doesn't show a faint apple.

FWIW, I noticed Slack, ImageGlass, and Windows thumbnails don't do gamma correction.

Ffossy · 19 Tem 2018

Thanks Nate for looking into this and fixing it.

I wasn't sure about what format Spine was exporting in. Now I know it will always be sRGB. Thanks.

As for PMA. That is less of a problem really. Since it is exporting in sRGB I'm not loosing any information, and so I'm just doing the PMA at load-time (correctly with sRGB * linear_to_srgb(alpha)).

Now that you do gamma correction, beware of not accidentally altering images that are already at 2.2 gamma (because sRGB is on average 2.2 gamma but it is definitely NOT uniformly 2.2. It is linear for the low-range colors, and non-linear with an exponent of 2.4 for higher range colors). I think libpng only does gamma correction if the gamma differs "substantially" from 2.2. The exact formulas for sRGB are:

(see e.g. https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_sRGB.txt)

sRGB to linear

     {  0.0,                          cl        <= 0
     {  12.92 * c,                    0         <  cl < 0.0031308
cs = {  1.055 * cl^0.41666 - 0.055,   0.0031308 <= cl < 1
     {  1.0,                                       cl >= 1

Linear to sRGB

     {  0.0,                          cl        <= 0
     {  12.92 * c,                    0         <  cl < 0.0031308
cs = {  1.055 * cl^0.41666 - 0.055,   0.0031308 <= cl < 1
     {  1.0,                                       cl >= 1

In case you want an example image of PMA done correctly (sRGB * to_srgb(alpha)) vs incorrectly (sRGB * alpha). I have attached it here. To me there is a rather noticeable difference.

update: I can see that the helmet on the character does not look as it should when doing srgbcs(a). I'll investigate why. The formula pma=cs(rgb)cs(a) is the correct one though. Going to linear cl(pma) = rgba (correct). While cl(cs(rgb)a) = rgb*cl(a) is incorrect.

update2: it was the image that needed to be corrected (it had been drawn/adapted to how it looked when it was not correct).

However, one thing I noted: you said "That image should be rendered correctly by your browser as a black square.". That image is not a black square - it is a mosaic (darkest possible grey with pitch black). Perhaps your monitor has changed contrast/brightness and not been calibrated?

OK, about the PMA not looking "right" despite that I did it correctly: in the end, it was the image itself that needed fixing. (i.e. PMA should be cs(rgb)*cs(a) (if you send sRGB data to OpenGL:s sRGB textures)).

Most software doesn't do this correctly. So it is hard to know how to compare. And it is easy to adapt art to the wrong equations (but you will get quirks when you add lighting and blur effects etc).

Nate · 6 Oca 2019

No worries, Spine uses the proper sRGB formula rather than a 2 (or 2.2) gamma approximation.

Aye, I noticed that later. I'm not sure why I was seeing something off.

Indeed, it's quite a minefield, especially when it may not be readily apparent that the results are incorrect.

Thanks for the test images! We'll check the texture packer output is correct next time we get a chance to work on this. Also thanks for digging into this tricky stuff, it's good that Spine now respects the gAMA and sRGB PNG chunks. :beer:

This came up again and required some fixes, see:
3.7 project discoloration