TEX0_1.CBP Addressing Woes

[undercovergrill]

Hi,

TLDR; 
How does the CBP address convert back it's pixel location in the PSMCT32? (See below for my calc).
If I have a PSMT8 texture with CBP referencing back to a PSMCT32 how does that work?
Perhaps some other REGS that can offset effective result of CBP?


Currently reversing the Xenosaga series, in particular the first game. I'm really interested in their multi CLUT/palette textures. For example.

For Shion's character, there is a single .xtx (texture format) file that has both a PSMCT32 image and a PSMT8 image composed together. Meaning the rgba32 data is both a texture, and a palette for the same PSMCT32 data but as an PSMT8 image. Neat right.  

So, I have an 472x128(WxH) PSMCT32 image. Textures must be a power of 2 so it gets expanded to 512x128 PSMCT32 .
A second texture is created referencing that same 512x128 PSMCT32 data but formatted as PSMT8, which causes the PSMCT32 data to expand to a 1024x256 texture,
per the result of the documentation (8.2 Page Arrangement in Buffer).

Following the VIF command list + VifUnpack + disassembly from @chaoticgd vutracer (I can't post link), I was able to see an XGKICK setting TEX0_1 to a known value from their 3d file format .lex(slight value differences but that's expected, see note later on).

Here's some examples for TEX0_1.

#1 Model 1 : VU1 TEX0_1
#2 Model 2 : VU1 TEX0_1
#3 Model 3 : Suspected TEX0_1 from disc

Code:

#1 FROM VU1
TEX0_1 : 0x20071D0629343800
    TBP0 : 14336  (3800h)
    TBW  : 16     (10h)     16*64=1024
    PSM  : 19     (13h)     PSMT8
    TW   : 10     (ah)      1024
    TH   : 8      (8h)      256
    TCC  : 1      (1h)      RGBA
    TFX  : 0      (0h)      MODULATE
    CBP  : 14568  (38e8h)   (CBP * 64)=E3A00
    CPSM : 0      (0h)      PSMCT32
    CSM  : 0      (0h)      CSM1
    CSA  : 0      (0h)      0
    CLD  : 1      (1h)

#2 FROM VU1
TEX0_1 : 0x20071D0629343800
    TBP0 : 14336  (3800h)
    TBW  : 8      (8h)     8*64=512
    PSM  : 0      (0h)     PSMCT32
    TW   : 9      (9h)     512
    TH   : 7      (7h)     128
    TCC  : 1      (1h)     RGBA
    TFX  : 0      (0h)     MODULATE
    CBP  : 14335  (37ffh)  (CBP * 64)=DFFC0
    CPSM : 1      (1h)     ??? :(
    CSM  : 0      (0h)     CSM1
    CSA  : 0      (0h)     0
    CLD  : 1      (1h)
CLAMP1 : 0x000000007F5003FF
    WMS  : 3   (3h)        REGION_REPEAT
    WMT  : 3   (3h)        REGION_REPEAT
    MINU : 63  (3fh)       -> UMSK
    MAXU : 320 (140h)      -> UFIX
    MINV : 127 (7fh)       -> VMSK
    MAXV : 0   (0h)        -> VFIX

#3 ORIGINAL VALUE ON DISC
TEX0_1 : 0x20006E8629340000
    TBP0 : 0     (0h)
    TBW  : 16    (10h)     1024
    PSM  : 19    (13h)     PSMT8
    TW   : 10    (ah)      1024
    TH   : 8     (8h)      256
    TCC  : 1     (1h)      RGBA
    TFX  : 0     (0h)      MODULATE
    CBP  : 884 (374h)      (CBP * 64)=DD00
    CPSM : 0     (0h)      PSMCT32
    CSM  : 0     (0h)      CSM1
    CSA  : 0     (0h)
    CLD  : 1     (1h)

I have two questions; 

How does the CBP address convert back it's pixel location in the PSMCT32?
Is the CBP address 0xDD00 from the OnDisc version really a CBP, or is it something that needs to be processed and I should go look for that?

*TBP0 being 0 on OnDisc is expected, but is the difference in CBP expected too?

The closest I got was the Y coord being correct, while the X coord is off by 32;
*palette location... should... be correct, I checked by incrementing the Index8 data and matching a palette in the PSMCT32

Sample if you want to try to calculate it.
I can't share game assets due to copyright. Though I can make a mock-up, I've marked the palette location.

You should use 0x20006E8629340000 for TEX0_1; goal : { x : 240, y : 96 };
Tex0 definition can be found here : /PCSX2/pcsx2/blob/master/pcsx2/GS/GSRegs.h

   

Thanks for your time.

**I have a bunch of scripts for this game I'll be releasing once they are finished. I'll post them to this forum if anyone wants to make character/texture mods. Right now I have an obj exporter and patcher, but it can only do deltas on vertices. I hope to have that, this, and rigging support sorted for release. Stretch goal targets their VM that runs most of the gameplay code . But the swizzeling on textures is giving me burn out. 

[refraction]

CBP doesn't convert to the pixel location, it is the location of a palette in memory to be loaded in to the internal CLUT buffer (which can hold 256 32bit colours)

so the CBP is a part of TEX0 which says "next time the CLUT is reloaded (based on CLD) load the data from this location", and that decision is made when TEX0 is written to, using the new data. If the CLUT reloads, next time you try to draw with a PSMT8/PSMT4 texture, it will use the index from each of those pixels as a lookup for the palette.


if it's taking a 32bit image and referencing it directly as 8bit (I'm sure this isn't what you mean, but I'll mention it anyway), what it is doing is known as a "channel shuffle" where it's reading off the R G B A channels, depending on which one it's looking at. Games will fill the CLUT with a greyscale, then read each of these colour channels, and mask off the destination to put that colour value in the right place.

if it's just uploading a PSMCT32 texture and a PSMT8 texture, it's likely the PSMCT32 is the palette to go with the compressed/indexed texture, which is loaded in to the CLUT buffer as described above.

[undercovergrill]

CBP doesn't convert to the pixel location, it is the location of a palette in memory to be loaded in to the internal CLUT buffer (which can hold 256 32bit colours)

While the CBP is an arbitrary address to load data for the CLUT buffer, in this case the addr of the CLUT data and INDEX8 data overlap over in a PSMCT32.
You can see this below.

Code:

TEX0_1 : 0x20071D0629343800
    TBP0 : 14336  (3800h)
    CBP  : 14568  (38e8h)

TEX0_1 : 0x20076E8629343800
    TBP0 : 14336  (3800h)
    CBP  : 15220  (3b74h)

TEX0_1 : 0x20071C0629343800
    TBP0 : 14336  (3800h)
    CBP  : 14560  (38e0h)

TEX0_1 : 0x20073C0629343800
    TBP0 : 14336  (3800h)
    CBP  : 14816  (39e0h)

So this render INDEX8 with a CBP, but note the TBP0 and CBP address.
TBP0 and CBP are right on top of each other. Texture length is 3B02.
You can see CBP is always TBP0+n. Where n can be found using the OnDisc data. For example.

Code:

TEX0_1 : 0x20076E8629343800 = VU1 -> TEX0_1
    TBP0 : 14336  (3800h)
    CBP  : 15220  (3b74h)     CBP - TBP0 = 884

TEX0_1 : 0x20006E8629340000   OnDisc
    TBP0 : 0      (0h)
    CBP  : 884  (374h)        CBP Matches : So TBP0 + CBP = CBP = PALETTE

So for rendering, you grab the OnDisc(884) value and add TBP0(14336) of the loaded texture in memory. This gives you the CBP(15220) for the palette at runtime.

Offline would be TBP0 + CBP, where TBP0 is zero, So CBP = palette.

Shouldn't the pixel offset be given by this. (Given the stored PSMCT32 for the palettes is linear on disc)

Code:

l = unswizzle(CBP << 8);
x = l % 512;
y = l % 128;

I know my unswizzle code is inncorrect now, so I've moved away from elegant bit shifts, and am embarrassed by my attempts. So

This gives correct results for 3 CBP I've tested. But I can't explain good portions. I can explain why in general this makes sense. But the details are beyond me.
And it's probably wrong.

Code:

/*  (⩺_⩹)  */

// CBP can be viewed as a block index into the texture here.
// So to recover pixel coords we use CBP to derive a pageIndex and blockIndex
// Then following the pixels sizes of Pages and Blocks we can get pixel coords.

// 884 -> 240, 96;
// 864 -> 192, 96;
// 480 -> 448, 16;

int startBlock = 884;
int blockIndex = startBlock * 2; //
                                 // I can't explain away the * 2
                                 // Probably wrong

// 8.3.1. PSMCT32/PSMCT24/PSMZ32/PSMZ24
int[] blockLayout = new[]
{
    0,   1,  4,  5, 16, 17, 20, 21,
    2,   3,  6,  7, 18, 19, 22, 23,
    8,   9, 12, 13, 24, 25, 28, 29,
    10, 11, 14, 15, 26, 27, 30, 31,
};
int blocksInPage = blockLayout.Length;

int pageIndex = (blockIndex / blocksInPage);
int localBlockOffset = blockLayout[startBlock % blocksInPage];

// Can't explain, probably wrong
localBlockOffset =localBlockOffset > 0 ? localBlockOffset - 8 : 0;

// Probably need pixelLayout too? For coloum offset.
int localPixelOffset = (blockIndex * 64) % 16;

// Why no good reason just a palette I know starts at y : 16
// Probably wrong, should be 64, 32.
int pagePixelWidth = 32;
int pagePixelHeight = 16;

// My textures width.
int w = 512;

// Standard 2D array coords from single value index
// x = index % width;
// y = index / width;
// Here we respect wrapping at our texture width, and Y gets incremented
// by pageHeight for each width reached.
// Perhaps that is wrong.
int pagePixelX = pageIndex % (w / pagePixelWidth) * pagePixelWidth;
int pagePixelY = pageIndex / (w / pagePixelWidth) * pagePixelHeight;

int pixelX = pagePixelX + (localBlockOffset);
int pixelY = pagePixelY;

I haven't done any sort of thorough testing. But you see what I am going for.

Thanks again for your time. 

*This is rendering a from all the same model, TEX_FLUSH is set to invalidate the CLUT, along with the new TEX_0 each "mesh part".

[refraction]

well the size TEX0 is set to in the first example is 1024x256, which is a very strange size, but it does indeed overlap. However there's nothing stopping the gamedevs lying about the size, the texture could indeed be a lot smaller.

Also keep in mind that the main image is PSMT8, so the page size is 128x64, with a width of 1024 pixels (aka 8 pages)

so the math to get the number of pages would be 256 / 64 = 4 * 8 = 32 pages of space and we multiply that by 32 to get the block count, so going in to hex we do 0x20 * 0x20 = 0x400, so the end of the first texture will actually be 0x3C00.

But regardless, yes it still overlaps, but I'm not 100% convinced *all* of that is valid texture data, they might have snuck the palette in as part of the texture, it's entirely possible. There's no hard and fast rule that data cannot overlap, the GS is just numbers, it doesn't care what it originally was, it just takes whatever you tell it as gospel. You could upload a Z16 1024x1024 texture, tell it part of it is a PSMCT32 CLUT buffer, another part is PSMT4 and another bit is Z32, if you wanted to, the GS really doesn't care, what matters is the order in which the data is retrieved from memory and if it's being interpreted the way you want it to.

[undercovergrill]

Interesting you can lie about the texture size  

Ah and thanks for PSMT8 advice, I missed that and page size and GS info, I have re-turn the cranks in my head.

As far as gospel my data is far from it, the first TEX0 may be erroneous, I can't find it again. IDK sorry. 

In regards to not being 100% convinced, if you saw the PSMCT32 linearly laid out as 472x128 it is very clear that this is what is going on. There is a big portion of obvious character hair, and blocks of 16*16 laid out nicely in 128x128 blocks. Besides that the file is complete noise, scatterings of colors (INDEX8). I can post a portion of it if you'd like.

In total I count 9 palettes and the hair is 4 128x128 blocks.

What made me want to look this up was the git pull request, /pull/5547, for the GS texture dumping mentioning Xenosaga's use of multi-palette textures. 
In fact if you run the texture dumper you'll see the base source PSMCT32 512x128 and a bunch of 1024x256 INDEX8 of that same PSMCT32, just with CLUT.
If you do run the dumper to go the first tutorial room, skipping cutscene. The game has HD versions of characters, which include models and textures.

Also I have savestates addresses the works; I have another post I was working on that has some more data and info.

But thanks again for the your continued thoughts. 

[undercovergrill]

if it's just uploading a PSMCT32 texture and a PSMT8 texture, it's likely the PSMCT32 is the palette to go with the compressed/indexed texture, which is loaded in to the CLUT buffer as described above.

*This is just here for completeness. 

I've narrowed the renderer to just a single character, no lights, shadows, HUD and run vutracer again.

Texture data is uploaded once per frame. This would be the PSMCT32 data (the INDEX8 data and CLUT data).

Code:

[VIF] VIF1 Tag 01070830_30003b02 size=15106, id=3, madr=1070830, tadr=e00410 ; start texture
[VIF]     VIF1 SrcChain TTE=1, data = 0x00000000.00000000
[VIF] New VifCMD 0 tagsize 0 irq 0
[VifCode] Nop
[VIF] New VifCMD 0 tagsize 0 irq 0
[VifCode] Nop
[VIF] vif1Interrupt: 23db8f60 chcr 30000145, done 0, qwc 3b02
[VIF] VIF1chain size=15106, madr=1070830, tadr=e00420
[VIF] New VifCMD 0 tagsize 0 irq 0
[VifCode] Nop
[VIF] New VifCMD 0 tagsize 0 irq 0
[VifCode] Nop
[VIF] New VifCMD 0 tagsize 0 irq 0
[VifCode] Nop
[VIF] New VifCMD 50 tagsize 0 irq 0
[VifCode] Direct

And an example of the relevant VIF1 packets with TEX0_1 + extras like TEX2,CLAMP1 etc.

Code:

[VifCode] Unpack V4_32 (unmasked) @ 0x03D5 (cl=1  wl=1  num=0x06)
[VIF] Unpack VIF1, QWC 6 tagsize 18
[VIF] vif1Interrupt: 23dd266e chcr 10000145, done 0, qwc 7
[VIF] VIF1chain size=7, madr=e08ae0, tadr=e08ae0
[VIF] New VifCMD 14 tagsize 0 irq 0
[VifCode] MSCAL

TEX_0 will be set at 0x03D70.
Data setup happens in 002384e8 you can follow the data to see where the actually VIF packets are created and sent.

TEX_0 values

Code:

Addr       Disc               VU1                name
01001E50 - 2007FFE5E4020000 - 200EFFE5E4023800 - son_hair, son_matu_col
01016600 - 20001C0629340000 - 20071C0629343800 - son_kmono
01019210 - 20006F0629340000 - 20076F0629343800 - son_kutu
0101E710 - 20006E0629340000 - 20076E0629343800 - son_taitu
010330F0 - 20006C8629340000 - 20076C8629343800 - son_yubi
010404B0 - 20006C0629340000 - 20076C0629343800 - son_kuku
0104C470 - 20004E8629340000 - 20074E8629343800 - son_mune
0104FCD0 - 20006F8629340000 - 20076F8629343800 - son_cho
01051B00 - 20001D0629340000 - 20071D0629343800 - son_face
010549E0 - 20006E8629340000 - 20076E8629343800 - son_uwagimae, son_sode
010DD3C0 - 20003C0629340000 - 20073C0629343800 - son_eye

All disc TEX_0.TBPO/CBP are incremented by the same amount for each mesh part. This is done because of the dynamic nature of their runtime, the TBPO and the resulting CBP need to be updated to where the PSMCT32 was loaded at. 

Meaning for each VU1 TEX0 you can CBP - TBP0 and that will give you Disc.TEX0.CBP neat right 

[refraction]

pretty cool yea

[Everdred]

When you finish your project I'd be greatly interested in seeing your scripts. I'm working through a similar problem trying to track down palette locations. I feel like I'm close with vutrace (fantastic tool), but the values for TBP don't seem to actually point to anything in-memory. For instance I see an XGKICK where TEX0_1's TBP is 0x620. Multiply that by 0x100 and I get 0x62000 which is... not even a valid region of memory.

[jesalvein]

The most recent post in this thread is more than 8 months old. Please create a new thread and refrain from posting in threads older than 8 months in the future. Please also review the forum rules. Thank you.