Its defenetly your GPU, I was able to run the game at full speed with no framedrop during gameplay with my system. CPU usage in the title bar was always high simetimes at 95-99% so even my GPU couldnt keep up at times.
Edit: Fixed a typo
My video card definitely isn't top of the line, it's only an eVGA GTS 250, but all the PS2 games I have, FFX, Grandia 3 and Xtreme, Tales of Rebirth, Breath of Fire: Dragon Quarter, and Star Ocean 3 run very well. In fact, most of these games run full speed. Some games, like Star Ocean 3, don't run too well in large areas, however.