[blog] PS2 VU (Vector Unit) Documentation Part 1

[refraction]

The problems are rounding errors. i believe that the numbers are constantly rounded on SSE where it only happens with the result with the VUs. The method of rounding is "chop".on the VU's

so say for example you had 2 numbers which were rounded to whole integers, but the input values were

1.6
1.4

due to the rounding methods, the answer the VU's want here after adding them together would be 3, however due to SSE rounding (this wouldnt actually work like this, im just making it look simpler), it would do 1 + 1 as it rounds down before hand, so our answer would be 2.

This is exactly what happens with Valkrie Profile, one result is out by 1, causing the encryption to fail (yep, stupid game decrypts the ELF file with the VU's) and the game crashes before it can even load.

[cottonvibes]

Cottonvibes, could you give me an example of some numbers, that when subtracted, have a different result on SSE and the vu? I'm having some difficulty in understanding whats going on/replicating this. Thanks

Although there are some precision problems like ref said (so that numbers can differ by very little between emulated results and real VU results), the nastier problem is the limited range you get with processors that handle floats according to the IEEE standard.

With SSE any float equal or greater to 0x7f800000 (positive infinity's binary representation) is an Infinity or a NaN. Or if the number is less than 0xff800000 (negative infinity) it is Negative Infinity or a NaN.
In other words, if the floating point's exponent byte is 0xff, then it is a Nan or Inf.

so essentially if you have:

Code:

u32 f1_temp = 0x7f800000; // Postive inf binary representation
float f1 = (float&)f1_temp; // f1 is equal to pos inf

cout << (f1 - 100000.0f) << endl;

The number will be positive infinity on x86 processors or anything following the IEEE standard supporting Infinities and NaNs.

The number on the VU's will be the correct floating point approximated number because its number range is larger since it doesn't support Infs or NaNs (i don't know the resulting value of it off the top of my head).

If instead you had the following code:

Code:

u32 f1_temp = 0x7ffffffff; // A NaN's binary representation
float f1 = (float&)f1_temp; // f1 is equal to a NaN value

cout << (f1 - 100000.0f) << endl;

On SSE NaN's propagate to the result of expressions. So the answer will be a NaN (i can't remember if its guaranteed to be the same NaN as you passed in or not, i don't think it is).

On the VU's again you will get the correct approximation since it treats the NaN like it does any other number.

Almost every game that uses the VUs end up getting a NaN or Inf value at some point in its results.
If more operations are performed once you get a NaN, the resulting operations keep generating NaNs, and so you get a terribly wrong result.
You can clamp values to the max representable floating point number that isn't a NaN or Inf (which is what pcsx2's clamping modes do), but this obviously isn't as accurate as having the full range.

Also you don't know if the NaN's you're getting are because the game loaded in these values, or because some calculation in pcsx2 somewhere generated the NaN.
Most of the time i assume its the games loading the NaN values.

I would imagine for example that games would sometimes load in values like FLOAT_MAX (0x7fffffff) or FLOAT_MIN (0xffffffff), which on the VU's will be valid numbers, but would be NaNs with SSE.

Anyways hope that helps