Custom Shaders for GSdx

[Asmodean]

Good, finally somebody actually using the options lol

Some suggestions;

Gamma: If you set the gamma to 2.2, and tone amount to 2.2 also, you should get the same result your getting now, but with a better dynamic range.

Sharpen: If you don't like the deeper look of the sharpening, set the sharpen bias to 1.0, and you'll get a nice clean subtle texture sharpening, that will counter some of GSdx's heavy tex filtering (I probably should have 1.0 bias as default tbh)

Contrast: either set it to anything but 0 (0.01 even) or you may as well turn it off, because 0.0 is default contrast levels, so it's doing nothing/

Edit: you enabled me to find a bug in the sharpening btw lol, nice one. When FXAA is disabled, the luma calculations are off, and the sharpen wasn't working right >.<

I'll fix it and update the archive.
----

Edit2: Fixed a ninja bug in the texture sharpening, updated the filmic optional tonemapping, and mellowed down the default settings a bit, to make the initial setup a bit more neutral.

Re-uploaded the zip file here.

[Asmodean]

Made some improvements to the texture sharpening. If anyone is interested. Screens below of it off and on. The difference is subtle but it really shows on the texture clarity of the huts, and canopies/crates etc.


I've increased the strength of it a bit in the shots, just to make it a bit more apparent in a screenshot.

If you want to view original high quality versions, right-click the image, and hit 'view original'.
Original
Texture sharpening

[refraction]

That makes quite a huge difference to a lot of things (especially the cliffs at the back and of course the main character)

seems to be quite selective in what it improves though, ground textures seem to go untouched, but definitely excellent!

[Asmodean]

Yeah lol, I'm not sure why that is tbh. it must be the luminance of the scene, because that's how I'm detecting edges. Colour might be better, if I could get it working. I tried once before to use colour edge detection, but it didn't seem to like it.

Edit: that big blurry patch on the cliff, is the in-game heat blur btw. The rest is the sharpening though

Now that I think of it. that was a pretty stupid place to take that comparison screen lmao

[refraction]

lol a little xD


I just tried your most recent shader.fx and i don't see any difference between turning the shader on or off, but HQ Sharpening is on in the ini along with fxaa, am i being a dumbass and have to turn off normal filtering or something? lol

I'm pretty sure the shader is working aside from that as the blooming etc was happening before i disabled it. Or is it that it is always on if texture filtering is on? I mean the textures do look pretty sharp anyways

[Asmodean]

Try the version attached below. It's the version I'm working on right now.

All you should need is fxaa enabled (by the GSdx dialog checkbox on latest version) or fxaa=1 in the GSdx.ini. and use the PgUp key to toggle.

I'm putting the sharpening code into the gsdx texture filtering code atm, to see what it looks like. If it looks good, I'll let you know.

[refraction]

That seems to be doing something, it's very subtle close up but you can see the distant textures look crisper for sure (checking with the FFXII demo start area on the beach)

[Asmodean]

Nope. It only breaks the planet. That gsdx tfx shader is seriously mind boggling xD.

I had it looking decent at one stage, with a bicubic filter on the tc, but then I realised later on that, that also broke an ass load of games. I was testing it in FFXII, as that's my go-to test game. When I tried it in others then. The whole thing went to sh*t lol.

[dabore]

those ffx shots look good. pretty big. AA is good. it's just weird that it sharpens already sharp textures and edges. those are really crazy in there - already hd. but the shader could need a pass that really finds the unsharp regions (like that pillars and stairs on the to house to the left) and do some dithering with a adaptive sampling kernel rather then that internal bilinear look. it's synthesized detail, but maybe it could look better.

just posting my sh*t idea.

edit: had a look. the texture sharpen pass is really static. you should loop the sampling. if the luma difference is big it's already sharp and you can skip the pixel if that differences (sharpen bias) is under. or you accumulate a distance threshold and take another pattern of samples around the center (let's call it "unsharp detection range") to gather more for your luma and (how conveniant) you could combine it with a random dither sampling for the unsharp regions at the same time. and throw out some more scattered color value. that's beast.

[Asmodean]

Well, I've a headache, but I think I've improved the sharpening edge detection a bit (picky gits! xD jk ofc). I used a modified version of the already included bicubic interpolation, for the blurred sample, and passed it to a triangular function.

Screens of the detection before, and now, below;

Detection before
Detection now

It still needs more testing though, and I'm going to bed Zzzz lol.

--------
dabore: thanks for the suggestions, but I'm pretty limited with what I can do here for now. I've one bound texture object, and a single pass. If I tried to do all that in a single pass, I dunno if it would be worth the performance cost for a single effect, in a suite of effects. PCSX2's post processing is still young : )
Edit: the AA is the fxaa btw. I keep it on when I'm testing other effects, so I know if any of my changes have any cause & effect on it's output. Seeing as I'm doing all of my effects in a single pass, I've to be careful about the balance.