Q3 2021 Progress Report

[Bositman]

Written by RedDevilus and lightningterror

Core Improvements

[Enhancement] #4436 Core/GS: Merge last plugin GSdx into the Core. By GovanifY.

As we finally merge the last plugin namely the one responsible for the graphics settings (upscaling, blending, shaders) we can look back towards the past releases. Plugins were once a great source to decouple the core team and separate contributors so neither one wasn't blocked for doing improvements, however, these plugin creators diminished over the years until 2016 had not many advancements:

There was pretty much only ongoing work for USBqemu-wheel by jackun and CLR-DEV9 by TheLastRar, not only that the core team had better replacements over time and the last plugin creators also became de facto more a core contributor and residing in the discord group. While we do lose some other notable plugins that are highly requested like Nuvee for Lightguns, OnePad legacy which handled controller remapping but these are only temporary drawbacks as the pros outweigh the cons as you don't need to scourge our website or GitHub for plugins and fewer clicks in the GUI.

It's also one of the reasons why Q2 2021 was on the meager side, it blocked a lot of contributors potential work or even would invalidate other PRs.

[Enhancement] DEV9 improvements:

#4220 DEV9: Add Internal DNS server. By TheLastRar.

#4834 DEV9: Auto Gateway Fix for winPcap backend. By TheLastRar.

In this quarter TheLastRar has added a fix for the gateway which is the route outside your local network as it will now work manually or automatically. There is also the new internal DNS server if you know what the hosts file on your computer is, it's that. If you don't know you can compare it to a telephone book where it has your contacts with their names (URL/Websites) and their telephone number (IP addresses / Location) this is available in your ini folder.

[Enhancement] Counters improvements:

#4621 Counters: Reset Timer Count Regardless of Interrupt Enable. By F0bes.

#4647 Counters: Correct H-Blank for 1080i mode. By refractionpcsx2.

#4690 COP0 PCCR: Don't update counters if counting is disabled. By refractionpcsx2.

Gets rid of the spooky apparition in Grand Theft Auto 3 if you watch the intro.

This is kind of just enabling the old code which was kind of correct, but the old code didn't ignore the time passed, so this also updates the last cycles.

RedDevilus: "I've recently bought GTA 3 second-hand and did my regular testing phase when I heard talks about a cloned character, it was not triggered on my screen because I skipped the intro FMV. So if anyone says we only fix certain games or big bugs, you can look again. It was a minor bug that was actually a regression. Meanwhile, before the fix, I had my own methods of solving the issue but refraction wasn't happy with this brute-force approach."

6c76bd Counters: Fix scanline count for double strike modes. By refractionpcsx2.

[Enhancement] MTVU improvements:

#4503 microVU: Enable T-Bit to work with MTVU. By refractionpcsx2.

e256d1 VU: Fix address masking in MTVU. By refractionpcsx2.

[Enhancement] VU improvements:

#4539 VU: Improve sync with VU Kickstart, loosen without kickstart. By refractionpcsx2.

This tightens the VU Sync with the EE when Kickstart is enabled to fix some SPS in PAL Ratchet games, also loosened it slightly when Kickstart is disabled to improve the speed of games that don't require it.

#4694 VU: Optimise entering VU JITs. By refractionpcsx2.

This is a pretty huge performance uplift on multiple games,  it's one of the hugest gains (with the exception being the frametimes tolerance PR) of the dev build cycle.

Keep in mind the Speed percentage for the title bar on top of the viewport (also referred to as the emulation window) are the emulator frames called VPS, or in full Vsync per second in essence, how much the emulator refreshes but not the game itself which would be the Internal Frames or real FPS. It could be 60 VPS and 60 FPS (like Final Fantasy X pause screen) or 60 VPS with 20 FPS or other combinations like SOCOM II menus are 33-34 FPS even though it reports 60 VPS so you can feel the differences.

Another side note is that for normal PC games the internal resolution = viewport but in PCSX2 case, it's separated in Graphics Settings for upscaling and in General Settings where GS Window is, is the location of how big your viewport (window size) would be.

Here is an example of 3 tested games:

Ratchet: +12.36%

Tekken: +19.8%

My Street: +1.91%

2414ac VU: Mask start addresses. By refractionpcsx2.

cbb93f VU: Improve sync during interlock and Scratchpad VU mem writes. By refractionpcsx2.

404e2e VU: Adjust sync timing for VU Kickstart - Fixes Crash Twinsanity. By refractionpcsx2.

2392bf VU: Remove the need to pass VU struct to XGKick function. By refractionpcsx2.

[Enhancement] microVU improvements:

#4487 microVU: Consolidate I-bit hacks into one generic one. By refractionpcsx2.

#4703 microVU: Skip VU1 instructions on VU0. By F0bes.

#4719 microVU: Combine mVU0cacheReserve and mVU1cacheReserve. By lightningterror.

#4767 microVU: Replace XGKick hack with synced XGKick option. By refractionpcsx2.

e13e1b microVU: avoid half-completed program loading null block. By refractionpcsx2.

#4668 microVU: Don't emit add reg, 0 in a few instructions #4668. By stenzek.

[Enhancement] Gif improvements:

#4619 GIF: Reimplement GIF FIFO to enable only when it is needed. By refractionpcsx2.

[Enhancement] Vif improvements:

#4726 Vu int improve. By refractionpcsx2.

Our interpreters were frankly inaccurate and slow, this Pull Request has improved the accuracy in turn fixing a lot of visuals on uncountable games. Most people won't notice this change but it can help in comparing between the speedy recompilers if there is a bug hiding in plain sight.

4a370b VIF: Only enable reverse FIFO hack if VIF1 is still active. By refractionpcsx2

[Enhancement] SPU2 improvements:

#4530 SPU2: Delete broken/unused alsa backend. By Ziemas.

#4574 SPU2: Minor cleanup. By MrCK1.

#4581 SPU2: Remove NEVER_SKIP_VOICES. By Ziemas.

#4582 SPU2: remove unused variable. By Mrlinkwii.

#4591 SPU2: Remove user options for disabling reverb and de-alias filter. By MrCK1.

#4648 SPU2: Remove Waveout backend from Windows. By Mrlinkwii.

#4699 SPU2: Adjust DMA timings for IRQ's and small packets. By refractionpcsx2.

#4700 SPU2: Don't initialise sound buffer if it's not open. By refractionpcsx2.

#4774 SPU2: Disallow KeyOn within 2T of last KeyOn. By refractionpcsx2.

#4803 PortAudio: Remove DirectSound backend By CookiePLMonster.

Perhaps an oversight as we removed DirectSound module but it was still listed as a Host API for PortAudio, reason for removal is it tends to break audio samples to even refuse to work at all. In the future, it would be nice to have Cubeb as an audio module.

849ad1 SPU2: Add rogue BIOS loop point which Megaman X7 relies on. By refractionpcsx2 and Ziemas.

[Enhancement] USB improvements:

#4571 USB: Freeze mode enum for keyboardmania. By Ziemas.

[Enhancement] CDVD improvements:

#4578 CHD: Fix parent search on windows. By Ziemas.

#4645 CDVD: Fix CdlReadS for PS1 games with CDDA. By kozarovv.

#4646 CDVD: Implement SCMD 0x27. By kozarovv.

Implement SCMD 0x27 to allow PS1 games to boot on SCPH-50009 and DTL-H50009 bios.
According to krat0s, this command should return PS1 Disc Serial.
Disc Serial is later used by ps1drv itself to find matching game config.

Note: While DTL consoles are known to use region-free PS1 bios, SCPH-50009 uses Asia/Japan PS1 bios, and region lock applies here.

For the observant people if you have looked at our compatibility rating you may have noticed that we had 0 games for several months until we detected that the Chinese entries and BIOS were different in code and very few exist in our proximity. Now we are up to 2 entries (actually 1 but the forum compatibility needs an update).

#4702 CDVD: Fix up Pause and some Status behavior. By refractionpcsx2.

Correctly sets the CDVD Status after a Pause command and corrects how the CDVD Status is handled during N-Commands. Also adjusted game fixes as necessary for known fixed games (below).

Some games refused to boot as PCSX2 ignored the correct way to handle them.

[Enhancement] IPU improvements:

#4526 IPU: Fix SETTH threshold masks. By kozarovv.

According to available sources, it's 8:0 for transparency, and 24:16 for translucency.
Currently, PCSX2 is masking bits 8 and 24 making them always 0.

In essence, IPU handles FMV/cutscenes and this PR improves some issues with them.

Hype Time Quest: Before > After

 

#4613 IPU: Store thresholds for color conversions in u16, bump savestate version. By kozarovv.

[Enhancement] Debuggers improvements:

#4638 Debugger: Add option to display VU0F registers as floats. By F0bes.

#4644 Debugger: Fix vmulai and vmaddai disassembly. By JayFoxRox.

#4671 Debugger: Purge 64 / 32 bit register view options. By F0bes.

Remove the option to view > 32 bit registers as 32 bit or 64 bit and instead display the entire register value.

#4741 Debugger: CtrlRegisterList: Fix warning and screw up. By F0bes.

[Enhancement] Misc core improvements:

#4742 Emitter: Use 64-bit operations on x64. By TellowKrinkle.

#4636 HostFS: Add remove file functionality. By fjtrujy.

#4642 PGIF: Code refactoring/cleanup. By kozarovv.

Playing PS1 games has been a passion for some but we don't recommend it in its current state. However, we made some big improvements but it's better to use a dedicated PS1 emulator like DuckStation.

Here are some example comparisons of the improvements we witnessed:

Resident Evil: Before > After

 

Street Fighter EX Plus Alpha: Before > After

 

#4676 IOPBios: Fix OOB read when IRQ line is invalid. By F0bes.

#4739 EE JIT: Implement SDR/SDL, LDR/LDL instructions. By refractionpcsx2.

This has a very huge performance lift, though this is the best-case scenario if it uses these instructions. So don't expect games to reflect such large performance increases like these charts would initially suggest.

Above were refraction's results from the homebrew ELF, located here: SDL/SDR LDL/LDR Homebrew ELF

Below are the results of RedDevilus on the same homebrew which taxed the registers.

3680ce Core: Reset game CRC when rebooting. Avoids previous games patches being loaded. By refractionpcsx2.

An oddity was discovered where patches meant for one game were accidentally injected into another which effectively broke games. Oops!

#4667 iR5900: Use a signed multiply for MULT1 const prop. By stenzek.

#4748 R5900: Fix LWL did not sign extending in interpreter. By stenzek.

#4548 IOP Interpreter: Warn on branches to zero. By F0bes.

#4523 Misc: Replace ATL with WIL. By CookiePLMonster.

While this may seem an innocent library API change (how it interprets everything from instructions to clicks)

Definition of WIL: The Windows Implementation Libraries (WIL) is a header-only C++ library created to make life easier for developers on Windows through readable type-safe C++ interfaces for common Windows coding patterns.

Silent: 'I used their CoInitializeEx that explodes loudly on failure, so regressions = quietly ignored errors in the code started showing up spectacularly
ATL->WIL refactor itself isn't that interesting, if anything those regressions were
as they were issues that were always around but quietly ignored instead of manifesting themselves in an obvious way.'

So as you can refer from Silent's statement, it led to a domino effect in that lots of things seemed to break from audio to wherever, as seems consistent with PCSX2 how did this even work in the first place to then see old bugs float to the surface hiding in wait. Every day PCSX2 becomes less hacky and has a more sane codebase. You can imagine the chaos and the confusion it brought to many when trying to comprehend it all not only in terms of the PS2 but how PCSX2 has implemented it.

---

GS Improvements

[Enhancement] Misc GS improvements:

#4508 GS-hw: Enable pabe bit only when sw blending is enabled. By lightningterror.

#4615 GS: Manage draw rectangle in GS instead of wx. By stenzek.

#4632 GS-d3d11: Cleanup fxaa and external shader. By lightningterror.

#4633 GS-d3d11: Minor Blend changes By lightningterror.

#4664 GS-GUI: Add 7x Multiplier. By RedDevilus.

#4682 GS-GUI: Enable Software Edge Anti-Aliasing by default. By RedDevilus.

This option helps several games on Software renderer but it won't help on a Hardware Renderer as it's not implemented at this time.

Some examples where AA is useful to fix issues:

Final Fantasy X: Before > After

Relevant: #983

Look at the blue triangle below Tidus.

 

Final Fantasy X: Before > After

Relevant: #3341

Look at the weird line on the tree (bottom left) and the black triangle that is right of Tidus.

 

Doko Demo Issho - Toro to Ippai, Toro to Kyuujitsu: Before > After

Relevant: #4674

Look at the eyes of these weird cat characters.

 

Football 2002: Before > After

Look at the number 4 on the orange shirt.

 

Syphon Filter - Dark Mirror: Before > After

Look at the shadows of the character, a harder one to see than the other comparisons.

 

These are just a few examples, of course there are more.

#4706 GS: Update tooltip to remove reference to DirectX. By Mrlinkwii.

#4789 GS-GUI: Add tooltip for Dithering. By RedDevilus.

#4811 GS: Test for th for tex0 flush. By tadanokojin.

 

---

Misc Improvements

 [Enhancement] GUI improvements:

#4512 GUI: Re-order config settings. By RedDevilus.

It will look closer to the layout of 1.6 stable release, help with muscle memory and visual cues.

#4532 GUI: Set missing icons and update titles. By Florin9doi.

#4620 GUI: Rehaul BIOS / Folder Selector. By RedDevilus.

BIOS and folder tabs are now together in General Settings (Used to be called Emulation Settings)

This will make it easier to find settings with fewer clicks, although there is a height issue that forces you to scroll for the time being.

#4640 AppMain: Fix no$ symbol loading for elfs. By F0bes.

#4663 Misc: Fix lastpath updating when using the --elf cmdline arg By F0bes.

#4675 WX: Make position validation multi-monitor aware. By tadanokojin.

#4750 Config: Normalize limiter values before saving. By tadanokojin.

#4756 GUI: Stdio console on mac. By tellowkrinkle.

#4791 GUI: Select a bios by default. By F0bes.

This will automatically click on the first BIOS in the list if you haven't selected one, affecting First-time Wizard and in General Settings. It's nice for the User Experience and it hopefully grows on you when using dev builds.

#4798 GUI: Fix crash on assertion in newer wx. By tellowkrinkle.

#4838 GUI: Make 'Synchronized MTGS' UI settings exclusive to debug/devel. By Mrlinkwii.

The setting below has been quite the staple in releases and it has actually no business in here as its only purpose is debug and not the greatest at that.

#4844 Move Vu overflow flag checks to a Gamefix + reorganise panel. By refractionpcsx2.

[Enhancement] GitHub/CI improvements:

#4509 CI: Update PR labelers to match GS merge. By lightningterror.

#4522 CI: Split AVX2 and non-AVX2 windows builds By xTVaser.

At the start of this report was the last GS merge, it has produced one minor annoyance. We couldn't keep all 3 GS plugins, we removed SSE2 plugin (which frankly was bad anyway) then there was SSE4 and AVX2.

What is PCSX2-AVX2?
PCSX2 uses SIMD instructions to process graphics. The following are the SIMD instruction sets PCSX2 supports:
- AVX2: Newest and fastest. Recommended for CPUs released in the last 8-10 years.
- SSE4: Older, still fast. Recommended for older CPUs which do not support AVX2.
- SSE2: Oldest, and slowest. Recommended only if your CPU is too old for SSE4.

Which one do I pick?
- 1.6.0: PCSX2 will choose the appropriate plugin for you automatically.
- 1.7.0: Try pcsx2-avx2.exe. If the program fails to start, then use pcsx2.exe.

So in essence, if you have a 4th gen intel or higher with exception the Pentiums you are probably fine or AMD Ryzen series. In the future this will turn back into a single exe where it's default SSE4 with a checker to enable AVX2 if you support it.

#4529 CI: Remove dashes from build names. By tellowkrinkle.

#4840 CI: Bump macos version to 11. By lightningterror.

#4657 GitHub: Switch to new issue forms. By xTVaser.

[Enhancement] Other misc improvements:

#4501 macOS: Fix async file reader error handling. By tellowkrinkle.

#4527 KeyboardQueue: Remove a Windows-specific critsec. By CookiePLMonster.

#4554 3rdparty: add jpgd project. By tadanokojin.

#4566 Misc: Don't update homebrew when not necessary. By tellowkrinkle.

#4570 Dependencies: Update GHC 1.5.4 to 1.5.8. By RedDevilus.

#4592 Dependencies: Update OpenGL dependencies. By RedDevilus.

#4651 Misc: Add StartupWMClass to desktop file. By GiantEnemyCrab.

#4670 Add Linux distro information to program log. By Mrlinkwii.

#4701 App: Add Windows exe version information. By refractionpcsx2.

---

GameDB Improvements

As always there are many GameDB changes per quarter, since there are too many to even list we will just link the date range for merged pull requests, not counting directly pushed commits to master. List can be found at GameDB Q3 2021.

#4556 Noteworthy changes include patches for unplayable games which makes the games playable from this thread. RedDevilus took the patches and opened a pull request adding the patches to the GameDB.

---

Thank you all, see you in our next coverage, cheers