On PPSSPP and other emulators I often disable the fps limit to speed things up and when I enable the fps limit,the music speed is restored immediately but in pcsx2 it takes 2 seconds to restore the audio playback speed.
Using any of the "Synchronizing Modes" doesn't help
I think it's because the way our timestretcher works. It basically kind of blends the speed up and down to avoid any sudden jumps and crackling. Async audio should fix it though as... it's async.
The stretcher takes a little time readjusting. It's a small price to pay for correct emulated speed vs what you hear.
I don't know how other emulators do this but on the PS2, the SPU2 has to be in tight sync with the rest of the system.
TimeStretch - immediately increase the speed and there is a delay before restoring it
ASync - play normally for a while and then increase the speed...and there is a delay before restoring it
No Sync - either the same or almost the same as async