[WORKAROUND] build with -fprofile-generate
#1
I've set up a 32bit chroot to build pcsx2 from svn. The basic build works, and the program runs and plays games.

I want to make a profiled build for better performance. This is a two step process: building once with cflag "-fprofile-generate" and ldflag "-lgcov" then rebuild with cflag "-fprofile-use" and ldflag "-lgcov".

No matter what one specifies for the cflags in build.sh, the build will fail. Something is going wrong parsing this string which causes cmake to fall apart and give up. The only error reported comes from a temporary file that exists only during complie time....
Code:
CMake Error: Error in cmake code at
/home/pcsx2-read-only/build/CMakeFiles/CMakeTmp/CMakeLists.txt:12:
Parse error.  Function missing ending ")".  Instead found unterminated string with text ")
TARGET_LINK_LIBRARIES(cmTryCompileExec ${LINK_LIBRARIES})
".
CMake Error: Internal CMake error, TryCompile configure of cmake failed


This is what I added to flags in build.sh
Code:
flags=" -DUSER_CMAKE_C_FLAGS:STRING=\"-fprofile-generate\" -DUSER_CMAKE_CXX_FLAGS:STRING=\"-fprofile-generate\" -DUSER_CMAKE_LD_FLAGS:STRING=\"-lgcov\" "
I have tried other flags; the error never changes. Although the error mentions "target link libraries", it will be exactly the same without any specified ldflags.
I have also tried without the escape characters, which results in all of the flags being discarded or ignored.

I am fully aware that special configurations, and in particular the use of special cflags, are unsupported. The problem, I belive, is not the flags but how the strings are being parsed for or by cmake. This can probably be fixed by carefully looking over all of the cmake files in the source directory for an out of place quotation mark, parenthesis, or missing escape character.
Reply

Sponsored links

#2
When specified as
Code:
flags=" -DUSER_CMAKE_C_FLAGS:STRING=\"-fprofile-generate\" -DUSER_CMAKE_CXX_FLAGS:STRING=\"-fprofile-generate\" -DUSER_CMAKE_LD_FLAGS:STRING=\"gcov\" "
(note the incorrectly specified ldflag) the compiler doesn't throw any siginificant errors, but the flags are being ignored: no profile data is generated while running pcsx2.
Reply
#3
The way it's probably meant to be done is from the command line like so:
Code:
DUSER_CMAKE_C_FLAGS="-fprofile-generate" DUSER_CMAKE_CXX_FLAGS="-fprofile-generate" DUSER_CMAKE_LD_FLAGS="-lgcov" ./build.sh --clean --release
but that doesn't work either.
Reply
#4
It is possible to sneak in some cflags by editing cmake/BuildParameters.cmake at line 97
Code:
set(DEFAULT_GCC_FLAG "-m32 -msse -msse2 -march=i686 -pthread ${DEFAULT_WARNINGS}")
Those flags can be replaced with whatever you want....

I'm waiting on a compile to finish before I can test if editing line 115
Code:
string(STRIP "${USER_CMAKE_LD_FLAGS} -s" USER_CMAKE_LD_FLAGS)
provides a place to specify -lgcov....
Reply
#5
(04-21-2013, 09:00 AM)quequotion Wrote: Those flags can be replaced with whatever you want....
Within limitations, most cflags will cause segmentation faults either in the compiler, pcsx2 itself, or its plugins.

Quote:I'm waiting on a compile to finish before I can test if editing line 115
Code:
string(STRIP "${USER_CMAKE_LD_FLAGS} -s" USER_CMAKE_LD_FLAGS)
provides a place to specify -lgcov....
Conclusion: UNKNOWN. While the program was able to compile and run, I was never able to prove that any profile data generation was actually taking place. There should be ".gcda" files created somewhere in the source tree, but there are not.

Has anyone else ever successfuly used PGO with pcsx2?

Was there ever a time when cflags or ldflags could be specified as intended?
Reply
#6
Hum, it is not mandatory to use the build script. You can call cmake and make directly. Here an exmple, adatpt it to your need.
Code:
cmake . -B/tmp/obj-i486-linux-gnu \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_BUILD_STRIP=FALSE \
        -DXDG_STD=TRUE \
        -DPACKAGE_MODE=TRUE \
        -DUSER_CMAKE_C_FLAGS:STRING="-g -O2 -D_FORTIFY_SOURCE=2 -Bsymbolic" \
        -DUSER_CMAKE_CXX_FLAGS:STRING="-g -O2 -D_FORTIFY_SOURCE=2 -Bsymbolic" \
        -DUSER_CMAKE_LD_FLAGS:STRING="-L/usr/lib/i386-linux-gnu" \
        -DCMAKE_C_COMPILER="/usr/lib/ccache/gcc-4.7" \
        -DCMAKE_CXX_COMPILER="/usr/lib/ccache/g++-4.7" \
        -DCMAKE_VERBOSE_MAKEFILE=FALSE \
        -DREBUILD_SHADER=TRUE
Then the build
Code:
make -C /tmp/obj-i486-linux-gnu

However there is a high probably that the compiler will crash or PCSX2 will segfault with those profiling data.
Reply
#7
(04-21-2013, 04:07 PM)gregory Wrote: Hum, it is not mandatory to use the build script. You can call cmake and make directly. Here an exmple, adatpt it to your need.
Code:
cmake . -B/tmp/obj-i486-linux-gnu \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_BUILD_STRIP=FALSE \
        -DXDG_STD=TRUE \
        -DPACKAGE_MODE=TRUE \
        -DUSER_CMAKE_C_FLAGS:STRING="-g -O2 -D_FORTIFY_SOURCE=2 -Bsymbolic" \
        -DUSER_CMAKE_CXX_FLAGS:STRING="-g -O2 -D_FORTIFY_SOURCE=2 -Bsymbolic" \
        -DUSER_CMAKE_LD_FLAGS:STRING="-L/usr/lib/i386-linux-gnu" \
        -DCMAKE_C_COMPILER="/usr/lib/ccache/gcc-4.7" \
        -DCMAKE_CXX_COMPILER="/usr/lib/ccache/g++-4.7" \
        -DCMAKE_VERBOSE_MAKEFILE=FALSE \
        -DREBUILD_SHADER=TRUE
Then the build
Code:
make -C /tmp/obj-i486-linux-gnu

However there is a high probably that the compiler will crash or PCSX2 will segfault with those profiling data.
Thanks for the reply! I didn't know there were flags to specify the compiler (I've been working with 4.6/4.6.4/4.7/4.8 by changing alternatives; gcc 4.8.0 is working for pcsx2 btw....).

It's taking longer to build than usual, which is perhaps a good sign. I'll get back to you with results. I put in this
Code:
cmake .. -DUSER_CMAKE_C_FLAGS:STRING="-fprofile-generate" \
-DUSER_CMAKE_CXX_FLAGS:STRING="-fprofile-generate" \
-DUSER_CMAKE_LD_FLAGS:STRING="-lgcov" \
-DREBUILD_SHADER=TRUE

Actually, I have another challege to overcome before I can do much profiling. Running an OpenGL client from a 32bit chroot as a client for a 64bit X11 server isn't working well. Ordinary xclients are fine; the pcsx2 program window works well--but only the GSnull plugin can open a "video" window. ZZoGL and GSdx crash with x11 warnings.
Reply
#8
(04-21-2013, 09:22 PM)quequotion Wrote: I'll get back to you with results.
It worked, sort of. -DREBUILD_SHADER=true caused build failure by failing to find an opengl library (it's there, it just fails to find it).

So, PGO Building PCSX2:

Step 1: Generate profile data
0. start in (create) pcsx2-read-only/build
1. Configure cmake build
Code:
cmake .. -DUSER_CMAKE_C_FLAGS:STRING="-fprofile-generate" \
-DUSER_CMAKE_CXX_FLAGS:STRING="-fprofile-generate" \
-DUSER_CMAKE_LD_FLAGS:STRING="-lgcov" \
2. Build
Code:
make -j`getconf _NPROCESSORS_ONLN` -C ./
(getconf _NPROCESSORS_ONLN should return number of cpu cores)
3. Install (necessary to have plugins in one directory)
Code:
make install

Step 2: Play with PCSX2
0. For amd64, set up xorg to handle clients from the chroot:
a. in amd64 side terminal
Code:
xhost +
b. in 32bit chroot side terminal
Code:
export DISPLAY=:0.0
1. Run pcsx2
Code:
./pcsx2
2. Try everything--every plugin, every action--try to load a game with GSnull at least.
3. Exit (hopefully without any segfaults, have to start over if that happens).

Step 3: Rebuild and apply profile
0. The profile data was saved in pcsx2-read-only/build/pcsx2/CMakeFiles/pcsx2.dir/ and similar dirs for each plugin.
1. Configure cmake build
Code:
cmake .. -DUSER_CMAKE_C_FLAGS:STRING="-fprofile-use -fprofile-correction" \
-DUSER_CMAKE_CXX_FLAGS:STRING="-fprofile-use -fprofile-correction" \
-DUSER_CMAKE_LD_FLAGS:STRING="-lgcov" \
(-fprofile-correction is a cludge for profile corruption during multi-core compile)
2. Install
Code:
make install

Quote:Ordinary xclients are fine; the pcsx2 program window works well--but only the GSnull plugin can open a "video" window. ZZoGL and GSdx crash with x11 warnings.
This is still a problem, and not the only one. Some of the plugins were not recognized by pcsx2 during the profiling run. Results were inconsistent across two identical builds: sometimes the .so files would be declared invalid or ignored. This significantly limits the performance benefits PGO could yeild, particularly for the GSdx software renderer.
Reply
#9
Sorry to bring up a thread that's over a year old, but I never did work out why the plugins won't load when using -fprofile-generate.

This certainly causes the libraries to be much larger than usual, but I'm not sure if it has any other side effects that pcsx2 would notice.

Does pcsx2 run some checks on the libraries before loading them (size, hash, bytecodes, etc)?

Are libraries that generate profiles somehow invalid in and of themselves?

::EDIT:: Most of them will load and profile with "-ftls-model=global-dynamic"; but this is inconsistently ignored across identical builds....
Reply
#10
Alright, no matter what cflags either wxwidgets or pcsx or compiled with I keep slamming my head against this error:
Code:
[wx] dlopen: cannot load any more object with static TLS
.

This is with -ftls-model=global-dynamic in cflags for both compiles. The problem seems to be dlopen.

I read a little in the pre-fedora thread that pcsx2 does not feature static linking; is that still the case?

If so, where in the code could I start working on that?

::EDIT:: In the mean time, I'm going to attempt another work-around by increasing DTV_SURPLUS in lib32-glibc
Reply




Users browsing this thread: 1 Guest(s)