DirectXTex DirectX::Compress with premultipled alpha

Jan 18, 2016 at 8:55 AM
Edited Jan 20, 2016 at 7:03 PM
Hi,

I'm having issues with artifacts when using ::Compress on textures that are stored with premultipled alpha. I've searched around, but haven't really found any info on that. The recommendation sees to be that premultipled is the way to go, but that does not seem to work for me. The following image shows the artifacts I'm talking about:

https://www.dropbox.com/s/g2sndwezvcxkpmy/premul%20compression%20artifacts.png?dl=0

The left side is the leaves-texture, compressed with data in premultipled alpha rendered with premul alpha blend, and the right is compressed with data in standard alpha rendered with standard alpha blend.

I checked the code and the compressor does not care about the TEX_ALPHA_MODE_PREMULTIPLIED flag. However, maybe the issue is not with the compressor, but rather how the texture is sampled?
Coordinator
Jan 18, 2016 at 6:18 PM
What is the source image and how did you convert it? What was the Image structure you passed to Compress? How did you view the results?

The BC algorithm doesn't really care about PM vs. not, which is why in DXGI DXT2/DXT3 were collapsed to just BC2, and DXT4/DXT5 are just BC3. The issue is entirely about the source data and the blend modes used to set them.

You may also be having problems with sRGB vs. non-sRGB colorspaces, so keep that in mind.
Jan 18, 2016 at 7:55 PM
Edited Jan 20, 2016 at 7:03 PM
Thanks for the response.
The BC algorithm doesn't really care about PM vs. not, which is why in
DXGI DXT2/DXT3 were collapsed to just BC2, and DXT4/DXT5 are just BC3.
Great info. I searched for that piece of info without success.
You may also be having problems with sRGB vs. non-sRGB colorspaces, so keep that in mind.
I tried the available flags for this, but it had no effect on the artifacts, just the color-space in general.
What is the source image and how did you convert it?
It's all general textures for our game. Every texture with a soft alpha edge has the issue more or less.
How did you view the results?
Rendered in our proprietary engine.

Not very helpful answers, I know. :/
What was the Image structure you passed to Compress?
Nothing special. Currently I call:
DirectX::Compress(srcImages, nimages, metadata, DXGI_FORMAT_BC3_UNORM, TEX_COMPRESS_DITHER, 0.5f, bcImage)

with for example the following structs:
srcImages[0]
    width   528
    height  400
    format  DXGI_FORMAT_B8G8R8A8_UNORM
    rowPitch    2112
    slicePitch  844800
    pixels

nimages 7

metadata
    width   528
    height  400
    depth   1
    arraySize   1
    mipLevels   7
    miscFlags   0
    miscFlags2  2
    format  DXGI_FORMAT_B8G8R8A8_UNORM
    dimension   TEX_DIMENSION_TEXTURE2D

Here are more variations of the screens of the issue:
https://www.dropbox.com/s/9n9lsuereexbrzf/compression_artifacts.zip?dl=0

Any suggestions you might have would be very appreciated. Otherwise, next step for me will be to try AMD's texture compression library (http://developer.amd.com/tools-and-sdks/graphics-development/amdcompress/) and see if I end up with the same issue with that.
Coordinator
Jan 18, 2016 at 11:51 PM
Edited Jan 18, 2016 at 11:51 PM
Ok, so the final miscFlags2 of '2' means it was set as DDS_ALPHA_MODE_PREMULTIPLIED, presumably because you called "::PremultiplyAlpha" at some point, yes? Have you tried doing that after the compression? Does it matter? Have you confirmed the image is 'correct' before compression as both pmalph and straight?

Have you looked at the source for texconv?

You might also try setting up a simple repro with your processed DDS files so you can confirm if the problem is with the blend settings, the source image, or the processing. See the DirectX Tool Kit tutorial. You can try subbing in your texture files for the various forms of 'cat'.
Jan 20, 2016 at 10:25 AM
Edited Jan 20, 2016 at 10:27 AM
Ok, so the final miscFlags2 of '2' means it was set as DDS_ALPHA_MODE_PREMULTIPLIED, presumably because you called "::PremultiplyAlpha" at some point, yes?
No, I set that flag manually in case ::Compress used it (which it doesn't). I do my own pre-multiply conversion (which looks great as long as I don't compress the texture).
Have you tried doing that after the compression?
Calling ::PremultiplyAlpha after compression is not allowed in DirectXTex. (and it makes no sense in the BC3 format). Setting the flag does not matter, since I discard the DirectXTex structs after copying the data.
Have you confirmed the image is 'correct' before compression as both pmalph and straight?
Yes.

So, digging further into the problem, I'm leaning towards that the issue is that the BC3 format is not a great fit for pre-multipled alpha because the color and the alpha is interpolated with independent lookup-tables.

But I'll send you a case using the DirectX Tool Kit tutorial that will highlight the problem, which will also force me to make sure that I'm not missing anything.
Jan 20, 2016 at 7:20 PM
Edited Jan 20, 2016 at 7:31 PM
Alright. I built a new project using the DirectX Tool Kit, and the issue persists.

I used this texture:
https://www.dropbox.com/s/rkvpxn2o82wpr1a/texture.png?dl=0

With the following conversion calls:
texconv texture.png -m 1 -f B8G8R8A8_UNORM -o standard_uncompressed
texconv texture.png -m 1 -f BC3_UNORM -o standard_bc3
texconv texture.png -pmalpha -m 1 -f B8G8R8A8_UNORM -o premul_uncompressed
texconv texture.png -pmalpha -m 1 -f BC3_UNORM -o premul_bc3

And the following rendering calls:
m_spriteBatch->Begin(SpriteSortMode_Deferred, m_states->NonPremultiplied());
m_spriteBatch->Draw(m_StandardAlpha_Uncompressed.Get(), RECT{ 0, 0, 320, 400 });
m_spriteBatch->Draw(m_StandardAlpha_BC3.Get(), RECT{ 320, 0, 640, 400 });
m_spriteBatch->End();

m_spriteBatch->Begin(SpriteSortMode_Deferred);
m_spriteBatch->Draw(m_PremultipledAlpha_Uncompressed.Get(), RECT{ 640, 0, 960, 400 });
m_spriteBatch->Draw(m_PremultipledAlpha_BC3.Get(), RECT{ 960, 0, 1280, 400 });
m_spriteBatch->End();
After rendering it looks like this:
https://www.dropbox.com/s/2lvn8rgy9g2efaw/screen.png?dl=0

There are some tiny non-premultiply artifacts on two circles to the left. This because I'm not running a color-expanding pass on the transparent pixels, so the bi-linear blending blends towards the fully transparent/white pixels.

However, the main issue is of course the right-most premultiplied-alpha BC3, which has ugly artifacts all around.

As I mentioned in the previous post. I think this is due to how BC3 treats alpha and color in independent interpolation, and this combined with pre-multipled alpha where it's crucial that the alpha is in sync with the color to end up with the correct final blend color.

Full source here:
https://www.dropbox.com/s/zi6qpzqf6087k1u/premul_bc3_artifacts.zip?dl=0

I've been thinking if it would be possible to detect blocks that are hurt by this in the compression step and handle them a bit differently (use synced lookup-tables for alpha and color), but that is a bit out of my league. Otherwise, my option will be to instead use non-premultipled alpha textures, as I have this issue all throughout our game. It's a shame, since there are a few other unavoidable artifacts for non-premul alpha.

If you have any other suggestions, I'd love to hear them.
Coordinator
Jan 31, 2016 at 6:04 AM
Did you try playing around with -bcuniform or -bcdither to see if that impacts your issues? The flags are TEX_COMPRESS_RGB_DITHER, TEX_COMPRESS_A_DITHER, TEX_COMPRESS_DITHER and TEX_COMPRESS_UNIFORM.
Feb 21, 2016 at 8:49 AM
Edited Feb 21, 2016 at 8:51 AM
Sorry for the delayed response. Yes, I tried that as well, it made no noticeable difference. Here is the same texture rendered with the different dithering flags. All with premultipled alpha, so they all have similar artifacts:
https://www.dropbox.com/s/m1rxrrqeh2pabju/TEX_COMPRESS_UNIFORM.png?dl=0
https://www.dropbox.com/s/su9oakriv4je3e5/TEX_COMPRESS_RGB_DITHER.png?dl=0
https://www.dropbox.com/s/ip5dnvixyc3thnm/TEX_COMPRESS_A_DITHER.png?dl=0
https://www.dropbox.com/s/x3julnaie0o7oo0/TEX_COMPRESS_DITHER.png?dl=0

Also, I conferred with a game-dev friend (Double Fine) who confirmed he gets the same issue with premul alpha and BC3 using a completely different pipeline:
https://twitter.com/p1xelcoder/status/690187985472585728

Fyi. I've switched our engine to default to non-premul alpha, so this is not really an issue for us any longer.