Why is software mode that precise?

[willkuer]

GS can easily uses AVX whereas it is more difficult on the EE/VU. You can't compare both.

Maybe that's the most important thing... I was often wondering why we have instruction set specific gs plugins but not core 'plugins'. This sounds indeed like higher optimization level on the gs plugin.

Do you know that one can not use more often AVX/Higher SSE instructions in core would you just guess?

[Dr_Hycodan]

SW mode has that Silent Hill flashlight bug where the flashlight is just a white spiky non-transparent blob. Hardware actually emulates the flashlight properly for some reason.

[Ōkami Amaterasu]

I thought both opengl and directx 10 could do integer math. The dolphin team said it was one of the main reasons they dropped dx9.

[refraction]

I thought both opengl and directx 10 could do integer math. The dolphin team said it was one of the main reasons they dropped dx9.

They can, and a lot of the new fixes have been thanks to this integer math.

[Nijohc]

Does anyone know what causes the character shadows in Soul Calibur III to be non-existant, in either Software or hardware, is it the mistake in VU calculation or is it something else causing it?

[refraction]

It could be an internal thing on PCSX2, or it could be generally bad GS emulation

[K.F]

Short answer is that it has to do more with the nature of the CPU and GPU, than it has for the nature of the PS2. The CPU can do almost any function with almost any type, very fast threads - branch prediction, much faster caches ,etc -, but just a few cores per CPU. The GPU has fixed functions for fixed types, very slow threads compared to the CPU , but it has thousands of them per GPU.

So yeah, the software mode is accurate because CPU cores are more versatile, but it is slow since there are just a few of them. This is true for all renderers, PS2 or not. For example, most modern ray-tracing renderers -like Vray and Mental ray- come it 2 versions, one for the CPU and one for the GPU. The GPU version is much faster, but is lacking many features that the CPU version has and had for decades. Similar thing is now happening for the finite element scene - which is a very slow and hard industry to change, I hate it - where there are GPU solvers now that has the same exact problem, GPU simulations are fast but missing almost all features mainly because the problem is hard to parallelize efficiently, so it is almost only used for explicit time integration problems. It is not just a PS2 emulation thing.

Did't I say it is a short answer

[gregory]

I thought both opengl and directx 10 could do integer math. The dolphin team said it was one of the main reasons they dropped dx9.

Quoting myself

Except upscaling which is a hack anyway. The only real issue is speed, otherwise lots of things can be fixed. You can do a kind of SW renderer in the shader directly

I never said it was not possible. I don't know if Dolphin uses integer for color or only the coordinate (or both). Anyway I try to replace float color computation with integer computation => speed hit 10% on SotC (4x/6x). To be fair, float uses native (therefore bad) rounding.

A basic shader are only 4/5 instructions.
Float shader
1/ no texture color conversion
2/ Clamp(x, 0.0f, 1.0f) is 0 instructions in HW (yes 0!)
3/ you can directly output the color

Vs integer shader
1/ float to int conversion (1/2 instructions)
2/ clamp(x, 0, 255) is 2 instructions.
3/ you need to convert back the integer to float (1/2 instructions)

In short a float shader is 4/5 instructions, the equivalent is 8/9 instructions. It seems small but you need to multiply those extra instructions by the number of time the shader are executed (hundred of millions of times by seconds).

Edit: Oh, forget to say, using integer (at least correct rounding) help to fix suikoden bug.

[Dr_Hycodan]

Wow then that means the dolphin team is doing a really good job at keeping the hardware renderers up to task with the software renderer. Pretty much every effect in the software renderer is emulated with its hardware renderers.

[Ge-Force]

Vs integer shader
1/ float to int conversion (1/2 instructions)
2/ clamp(x, 0, 255) is 2 instructions.
3/ you need to convert back the integer to float (1/2 instructions)

Is their any reason why you can't keep the output as an int? or have whatever calls the shader pass an int instead of a float?