CPU testing in pcsx2

[tohdom]

I want to discuss the idea, so DO NOT follow the instructions yet, it is just an idea, it could be changed or discarded completely.

So, AFAIK the CPU is the most important part of PC for ps2 emulation. And we cant compare CPU based on their performance in normal PC games.
So, the idea is to ask users on this forum with different CPU to benchmark it in pcsx2. I think the most popular game on ps2 is FFX NTSC, so what if every owner of this game will:
1) launch pcsx2 0.9.6 with default settings
2) set native resolution to lower GPU factor.
3) if intel, perform different test for each SSE
4) turn off fps limiter
5) benchmark FFX with FRAPS, for example, the intro scene.

This way we could gather data about CPU performance, analyze it. Use it to make our own CPU chart or find the popular benchmark (like EVEREST or something) which will perform similar to pcsx2.

What do you think? is it pointless? Somewhere flawed? or reasonable idea?

[yunsabit]

Yeh - play it on i7 in sw mode with 7 threads for graphic stuff and have 250 fps \o_.

Best thing for pcsx2 is still the i7, and it's easily oc'ed to 4 ghz on air, too.

intel>amd.

[Alfer]

I like the idea, and especially like the idea of having a PCSX2-specific benchmark tool. I'll volunteer time writing a php script that allows users to enter their score in a mysql database for us all to use if there winds up being a lot of interest. I would possibly be up to writing a benchmark tool if I could get some performance metrics that should be evaluated.

[Koji]

The problem is a lot of people don't own specific games, and the current FPS counter is not accurate (according to Bositman when I tried to use it for a few other comparison studies) so you'd have to use an external program like fraps to get an accurate estimate.

The ideal solution would be to either find a homebrew PS2 demo (something that can be freely distributed) that really pushes the PS2 hardware, or find someone who can make such a homebrew project and ask for his input on it.

[InfelPira]

I don't think this would end up being a very good idea in the end. It would give people false hope of their games working, when possibly, it will run slow due to compatibility reasons, etc, which in turn, would bring them here saying "OMG WHY CANTS I RUNZ THISZZZZZ!" There's also the fact that different configs work for different computers. What may work for my 2.0ghz c2d may not work for someone elses 2.5ghz c2d due to various differences within the processors (FSB, L1 Cache, L2 Cache, etc. processor speed isn't the only thing to look at. A 3ghz processor with a slowass FSB wont be any better than a 2.0ghz processor with a fast as hell FSB). So there are just so many variables that it would be hard to pull something like this off accurately.

[tohdom]

InfelPira, using this argument, all benchmarks for the regular PC games could be thrown away too? They always use high-end GPU, ram etc to exclude any bottleneck for CPU and end-user will never achieve such results on his home-PC.
But it is showing one useful thing - how much CPU A is faster than CPU B. So if i have A, i will see how much speed i gain(in % of course) if i update to B (only in this tested game, but hopefully, just hopefully, in other games too)

So. i have FFXII demodisk (http://ps2.ign.com/objects/785/785169.html)
and can get Silent Hill 2 Trial version from E3 2001. Is it legal to distribute them on this forum?

[Shadow Lady]

There's too many factors that affect speed on a computer, having accurate results from multiple users would be really difficult this way... I mean just check this tests from an user with very similar machines getting variable results: http://forums.pcsx2.net/thread-9356.html

You would have to get very big differences on tests to think of anything close to accurate and with differences in chip architecture this is not really easy, like the Core 2 wolfdale processors being a little faster than the older Core 2 conroe ones and not even talking about SSE4.1 tests, and Quad processors being faster than Dual Core on some machines because there's more room on them for threads... Not even mentioning people with different RAM timings/speed which not as important as CPU it does affect the final result just enough to make it even more inaccurate.

[Zeydlitz]

This is incorrect methodology. If you want to obtain real values, not depending's on users abilities, than you need homebrew testing suite, that would be running though elf-file-run menu and check FPS internally, also it should run different CPU instructions, so we could check them. That's correct way to do pcsx2 benchmarking.

[tohdom]

Zeydlitz
uh.. it is beyond my knowledge

[kidaware]

i wonder how hard it is to program a homebrew benchmark program for pcsx2? do you need a expensive sony ps2 dev kit to make a homebrew app/game?