I spend severals day to debug this god of war issue. I think I have an understanding of most of the issue to draw correctly the fog.
I really don't know how to fix it properly. But I think it would be nice to share the crazy behavior of GoW with you
* Start of the frame
various input texture and primitives
output a 32 bits RT @ 0x0 + a 24 bits depth buffer @0xd00
Then it will compute a special effect based on the depth. Screen is split in 8 vertial bands. And effect is apply 8 times (that a shame it would be free to do it once for the full screen with current GPU)
Step1 of the effect:
input texture: 24 bits (with depth swizzling) @ 0xd00
output RT: 24 bits @ 0x2800
depth (depth disabled): 24 bits @0xd00
RT will receive the opposite of texture (3 channels), i.e. a color of 4 will return 251.
GSdx limitation1: blending will clamp not wrap
=> solution use sw blending => accurate_colclip
GSdx limitation2: depth texture is float 32 bits on GPU. So it need to be converted. And we need to reuse it as input texture.
=> half solution: add some shader to convert it
=> add various hack to copy the depth into the local memory and disable the texture cache. Temporary until I find a better solution (and that it works this way).
GSdx limitation3: RT is 24 bits so alpha channel mustn't be written
=> solution not yet implemented but doable
Yes so much issue for a single draw call! Anyway I manage to have a texture. Initially it was completely dark now you have that
Step2 of the effect:
no input texture
output RT: 24 bits @ 0x2800
depth (depth enabled): 24 bits @ 0x2800
GSdx limitation: support of RT/Depth as same address.
RT will receive the color of RT (so no change)
Depth will be updated with the primitive test)
Note: due to GSdx limitation alpha value will be wrongly copied
Step3 of the effect:
input texture: 16 bits (depth swizzling) @ 0x2800
output RT: 16 bits @ 0x0
depth (depth disabled): 24 bits @ 0xd00
Rt will receive texture with a mask of ~0x3FFF, which mean that red channels is left untouched. And green partially masked
Gsdx limitation: GPU only supports mask of Full channel. You can't do it halfy.
GSdx limitation: initial RT was 32 bits now it is 16 bits ! Texture cache won't like it.
Actually what happen is that depth information is more or less stored in the alpha channel of the RT. If I get it correctly
Z[14:8] is copyied into A[6:0]
0 is copyied into A[7] (due to aem stuff)
Current screenshot of the situation, due to wrong format conversion we draw silly thing:
Once the effect is done 8 times for the full screen. There is a final post-processing step that multiply the the alpha channel of the RT (which is reused as 32 bits). On my example the factor is 91 (i.e. 0.71%)
input texture: 32 bits @ 0x0
output RT: 32 bits @ 0x0
I really don't know how to fix it properly. But I think it would be nice to share the crazy behavior of GoW with you

* Start of the frame
various input texture and primitives
output a 32 bits RT @ 0x0 + a 24 bits depth buffer @0xd00
Then it will compute a special effect based on the depth. Screen is split in 8 vertial bands. And effect is apply 8 times (that a shame it would be free to do it once for the full screen with current GPU)
Step1 of the effect:
input texture: 24 bits (with depth swizzling) @ 0xd00
output RT: 24 bits @ 0x2800
depth (depth disabled): 24 bits @0xd00
RT will receive the opposite of texture (3 channels), i.e. a color of 4 will return 251.
GSdx limitation1: blending will clamp not wrap
=> solution use sw blending => accurate_colclip
GSdx limitation2: depth texture is float 32 bits on GPU. So it need to be converted. And we need to reuse it as input texture.
=> half solution: add some shader to convert it
=> add various hack to copy the depth into the local memory and disable the texture cache. Temporary until I find a better solution (and that it works this way).
GSdx limitation3: RT is 24 bits so alpha channel mustn't be written
=> solution not yet implemented but doable
Yes so much issue for a single draw call! Anyway I manage to have a texture. Initially it was completely dark now you have that
Step2 of the effect:
no input texture
output RT: 24 bits @ 0x2800
depth (depth enabled): 24 bits @ 0x2800
GSdx limitation: support of RT/Depth as same address.
RT will receive the color of RT (so no change)
Depth will be updated with the primitive test)
Note: due to GSdx limitation alpha value will be wrongly copied
Step3 of the effect:
input texture: 16 bits (depth swizzling) @ 0x2800
output RT: 16 bits @ 0x0
depth (depth disabled): 24 bits @ 0xd00
Rt will receive texture with a mask of ~0x3FFF, which mean that red channels is left untouched. And green partially masked
Gsdx limitation: GPU only supports mask of Full channel. You can't do it halfy.
GSdx limitation: initial RT was 32 bits now it is 16 bits ! Texture cache won't like it.
Actually what happen is that depth information is more or less stored in the alpha channel of the RT. If I get it correctly
Z[14:8] is copyied into A[6:0]
0 is copyied into A[7] (due to aem stuff)
Current screenshot of the situation, due to wrong format conversion we draw silly thing:
Once the effect is done 8 times for the full screen. There is a final post-processing step that multiply the the alpha channel of the RT (which is reused as 32 bits). On my example the factor is 91 (i.e. 0.71%)
input texture: 32 bits @ 0x0
output RT: 32 bits @ 0x0