Why is software mode that precise?

[gregory]

Texture uses normalized integer (i.e. float that range from 0.0 to 1.0).
If you use true Integer, you can't use anymore the fixed function unit of the GPU (aka the blending unit). It is partially possible to do the blending in the shader (see accurate* option). But doing all draw calls is not yet possible (maybe with future hw). Shader blending will also cost extra operations.

Another solution is to use true float (not clamped between 0.0 to 1.0) often call HDR stuff. But it will be more memory intensive (16 bits/(32 bits?) by channel instead of 8). There is also the issue to clamp the value in 0.0 in 255.0.

The most interesting solution will be to uses full integer everywhere with a GPU that support programmable blending.

[dabore]

good point todo all integer. for texture coordinates this' not really working tho. as seen. the post pro buffers and stuff. isn't there always that pixel center offset bugs with filtering. it should work for colors tho. it's like bytes. integer. if weird fractional portions are not used the driver's shader translator might probably optimize the default ranges (1.0, 256, 65536) to straight binary anyway. on the gpu it ends up being the same pieces of bytes and bits. just the sign is a weird wrap.

i still have that experiment to test how values curve if i'd integrate a straight byte as a shifted exp. with fractional portion. this sort of floating binary math. i never did that.

[gregory]

good point todo all integer. for texture coordinates this' not really working tho. as seen. the post pro buffers and stuff. isn't there always that pixel center offset bugs with filtering

I'm not sure I understood what you say.

There is a half pixel bug due to upscaling. If you upscale with a factor of 2x, first pixel will be 0.5*2 = 1 instead of 0 hence a bad offset. I think there are also sometimes a bad offset due to rounding of texture coordinate. For the first point, I think we need to round UV coordinate in the vertex shader when doing a post-processing effect. I think it will work better than the current half offset hack.

[gregory]

SW mode has that Silent Hill flashlight bug where the flashlight is just a white spiky non-transparent blob. Hardware actually emulates the flashlight properly for some reason.

Do you have the issue if you change the rounding/clamping mode of the VU? Is it an old issue?

I have a solution to avoid the issue in GSdx but it could be also a vu bug.

Edit: in the mean time I pushed a temporary hack fix in GSdx.

[K.F]

Do you have the issue if you change the rounding/clamping mode of the VU? Is it an old issue?

It is an old issue and changing the VU settings does not fix it.

[refraction]

Have you tried using SuperVU and VU Interpreters to see if they make a difference?

[gregory]

Or the EE interpreter (yes take your time )

[K.F]

I tried everything for 15 minutes while reloading from memory card each time to make sure it is not one of those initialization errors like some VU errors I found in the past. Of course that means I can't try every single combinations, cause that would take hours. If you have a specific settings combination, I'll try it.

Edit: by the way, there are multiple bugs in the software mode in that game, not just the flashlight.

[prafull]

Have you tried using SuperVU and VU Interpreters to see if they make a difference?

Ref, These flashlight effects were broken in software mode around the same time that Lupin game was broken. flashlights are ok with this old gsdx I use for testing every game these days.

http://forums.pcsx2.net/attachment.php?aid=54407

[ssakash]

@Gregory

Why not emulate the GS memory format and fixed function units of the GS in a shader ? that way it could be suited according to the GS behavior.