Issue when building (with Full PGO) #418

Closed
opened 7 months ago by infrandomness · 13 comments

Hi, I am trying to build Foot with Full PGO;

but I'm having some troubles with rebuilding the codebase after having executed the command "meson configure -Db_pgo=use"

FAILED: pgo.p/pgo_pgo.c.o
clang -Ipgo.p -I. -I../.. -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/libxml2 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -flto -Xclang -fcolor-diagnostics -fprofile-use -fprofile-correction -DNDEBUG -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -std=c11 -O3 -DMEMFD_CREATE -D_GNU_SOURCE=200809L -fno-asynchronous-unwind-tables -DFOOT_IME_ENABLED=1 -pedantic -fstrict-aliasing -Wstrict-aliasing -fmacro-prefix-map=../../= -DHAVE_TERMINFO -O3 -Wno-ignored-optimization-argument -Wno-profile-instr-out-of-date -Wno-profile-instr-unprofiled -pthread -MD -MQ pgo.p/pgo_pgo.c.o -MF pgo.p/pgo_pgo.c.o.d -o pgo.p/pgo_pgo.c.o -c ../../pgo/pgo.c
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) async_write Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) fdm_add Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) fdm_event_add Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) fdm_event_del Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) render_refresh_csd Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) render_xcursor_set Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) wayl_win_init Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) wayl_win_destroy Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) spawn Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) slave_spawn Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) render_worker_thread Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) extract_begin Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) extract_one Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) extract_finish Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) cmd_scrollback_up Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) cmd_scrollback_down Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) ime_enable Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) ime_disable Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) ime_reset_preedit Hash = 12884901887 [-Werror,-Wbackend-plugin]
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

I get a bunch of errors from clang and then the building process stops;
(For information, I was able to build foot with the "Performance optimized, non-PGO" steps)
I am pretty sure I haven't missed any steps from the guide and have done it right.

Informations about my environment :
OS/Distro : Linux (5.11.8-200.fc33.x86_64)

VERSION="33 (Thirty Three)"
ID=fedora
VERSION_ID=33
VERSION_CODENAME=""
PLATFORM_ID="platform:f33"
PRETTY_NAME="Fedora 33 (Thirty Three)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:33"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=33
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=33
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"

LLVM/clang/lld version : 11.0.0.
I tried to build against both master (649820eeba) at this time, and agains the 1.7.0 release.

Hi, I am trying to build Foot with Full PGO; but I'm having some troubles with rebuilding the codebase after having executed the command "meson configure -Db_pgo=use" ``` FAILED: pgo.p/pgo_pgo.c.o clang -Ipgo.p -I. -I../.. -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/libxml2 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -flto -Xclang -fcolor-diagnostics -fprofile-use -fprofile-correction -DNDEBUG -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -std=c11 -O3 -DMEMFD_CREATE -D_GNU_SOURCE=200809L -fno-asynchronous-unwind-tables -DFOOT_IME_ENABLED=1 -pedantic -fstrict-aliasing -Wstrict-aliasing -fmacro-prefix-map=../../= -DHAVE_TERMINFO -O3 -Wno-ignored-optimization-argument -Wno-profile-instr-out-of-date -Wno-profile-instr-unprofiled -pthread -MD -MQ pgo.p/pgo_pgo.c.o -MF pgo.p/pgo_pgo.c.o.d -o pgo.p/pgo_pgo.c.o -c ../../pgo/pgo.c error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) async_write Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) fdm_add Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) fdm_event_add Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) fdm_event_del Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) render_refresh_csd Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) render_xcursor_set Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) wayl_win_init Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) wayl_win_destroy Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) spawn Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) slave_spawn Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) render_worker_thread Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) extract_begin Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) extract_one Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) extract_finish Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) cmd_scrollback_up Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) cmd_scrollback_down Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) ime_enable Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) ime_disable Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../pgo/pgo.c: Function control flow change detected (hash mismatch) ime_reset_preedit Hash = 12884901887 [-Werror,-Wbackend-plugin] fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. ``` I get a bunch of errors from clang and then the building process stops; (For information, I was able to build foot with the "Performance optimized, non-PGO" steps) I am pretty sure I haven't missed any steps from the guide and have done it right. Informations about my environment : OS/Distro : Linux (5.11.8-200.fc33.x86_64) ``` VERSION="33 (Thirty Three)" ID=fedora VERSION_ID=33 VERSION_CODENAME="" PLATFORM_ID="platform:f33" PRETTY_NAME="Fedora 33 (Thirty Three)" ANSI_COLOR="0;38;2;60;110;180" LOGO=fedora-logo-icon CPE_NAME="cpe:/o:fedoraproject:fedora:33" HOME_URL="https://fedoraproject.org/" DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/" SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=33 REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=33 PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy" ``` LLVM/clang/lld version : 11.0.0. I tried to build against both master (649820eeba) at this time, and agains the 1.7.0 release.
Owner

Hi, and thanks for the bug report!

This appears to be something new in LLVM/Clang 11.x.

The root cause is that the 'pgo' binary hasn't been run after the PGO generation phase.

I was able to get a build through after manually executing both ./pgo and ./footclient --version, after running foot with the script generated random input, but before meson configure -Db_pgo=use.

However, I'd like to keep this issue open until I've looked at this a bit more; there might be a better, and more automated way, to solve this.

But hopefully the above can get you going for now.

Hi, and thanks for the bug report! This appears to be something new in LLVM/Clang 11.x. The root cause is that the 'pgo' binary hasn't been run after the PGO generation phase. I was able to get a build through after manually executing both `./pgo` and `./footclient --version`, after running foot with the script generated random input, but before `meson configure -Db_pgo=use`. However, I'd like to keep this issue open until I've looked at this a bit more; there might be a better, and more automated way, to solve this. But hopefully the above can get you going for now.
Poster

Hi, the workaround worked perfectly for me, thanks.

Hi, the workaround worked perfectly for me, thanks.
dnkl referenced this issue from a commit 7 months ago
Owner

In the end, the easiest solution was to update the build instructions; in the generate phase we now execute all binaries that are going to be built in the use phase.

This also allowed us to drop several -Wno-* flags.

As a bonus, the pgo binary, which is only used in the generate phase (of partial PGO builds) are no longer built at all in no-PGO builds, or in the use phase of PGO builds. Resulting in slightly faster build times for those build types.

In the end, the easiest solution was to update the build instructions; in the _generate_ phase we now execute all binaries that are going to be built in the _use_ phase. This also allowed us to drop several `-Wno-*` flags. As a bonus, the `pgo` binary, which is only used in the _generate_ phase (of partial PGO builds) are no longer built at all in no-PGO builds, or in the _use_ phase of PGO builds. Resulting in slightly faster build times for those build types.
dnkl added the
bug
doc
labels 7 months ago
dnkl referenced this issue from a commit 7 months ago
dnkl closed this issue 7 months ago
Poster

Hi, I'm re commenting on this to leave a notice :

I retried to build foot on archlinux just now;

Founded out that if no fcft is installed on the system (which causes the meson to pull and build fcft as a subproject), the same issue seems to occure.

However after installing fcft using the AUR package and reconfiguring foot to use the newly installed fcft, everything goes fine

Informations :

FAILED: subprojects/fcft/libfcft.a.p/log.c.o
clang -Isubprojects/fcft/libfcft.a.p -Isubprojects/fcft -I../../subprojects/fcft -Isubprojects/tllist -I../../subprojects/tllist -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/pixman-1 -flto -Xclang -fcolor-diagnostics -fprofile-use -fprofile-correction -DNDEBUG -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -std=c11 -O3 -D_GNU_SOURCE=200809L -DMEMFD_CREATE -fmacro-prefix-map=../../subprojects/fcft/= -DFCFT_HAVE_HARFBUZZ -O3 -Wno-ignored-optimization-argument -fPIC -pthread -MD -MQ subprojects/fcft/libfcft.a.p/log.c.o -MF subprojects/fcft/libfcft.a.p/log.c.o.d -o subprojects/fcft/libfcft.a.p/log.c.o -c ../../subprojects/fcft/log.c
error: ../../subprojects/fcft/log.c: Function control flow change detected (hash mismatch) log_init Hash = 12884901887 [-Werror,-Wbackend-plugin]
error: ../../subprojects/fcft/log.c: Function control flow change detected (hash mismatch) log_deinit Hash = 12884901887 [-Werror,-Wbackend-plugin]

LLVM version : 11.1.0
foot version : 4cb8172373
fcft version : 873bdc81b5

Hi, I'm re commenting on this to leave a notice : I retried to build foot on archlinux just now; Founded out that if no fcft is installed on the system (which causes the meson to pull and build fcft as a subproject), the same issue seems to occure. However after installing fcft using the AUR package and reconfiguring foot to use the newly installed fcft, everything goes fine Informations : ``` FAILED: subprojects/fcft/libfcft.a.p/log.c.o clang -Isubprojects/fcft/libfcft.a.p -Isubprojects/fcft -I../../subprojects/fcft -Isubprojects/tllist -I../../subprojects/tllist -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/pixman-1 -flto -Xclang -fcolor-diagnostics -fprofile-use -fprofile-correction -DNDEBUG -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -std=c11 -O3 -D_GNU_SOURCE=200809L -DMEMFD_CREATE -fmacro-prefix-map=../../subprojects/fcft/= -DFCFT_HAVE_HARFBUZZ -O3 -Wno-ignored-optimization-argument -fPIC -pthread -MD -MQ subprojects/fcft/libfcft.a.p/log.c.o -MF subprojects/fcft/libfcft.a.p/log.c.o.d -o subprojects/fcft/libfcft.a.p/log.c.o -c ../../subprojects/fcft/log.c error: ../../subprojects/fcft/log.c: Function control flow change detected (hash mismatch) log_init Hash = 12884901887 [-Werror,-Wbackend-plugin] error: ../../subprojects/fcft/log.c: Function control flow change detected (hash mismatch) log_deinit Hash = 12884901887 [-Werror,-Wbackend-plugin] ``` LLVM version : 11.1.0 foot version : 4cb8172373 fcft version : 873bdc81b5
Owner

A while after #420 was merged, the instructions were updated again, adding the step ninja test to the generation phase. See 4cb8172373/INSTALL.md (performance-optimized-pgo).

Can you please try that?

A while after https://codeberg.org/dnkl/foot/pulls/420 was merged, the instructions were updated again, adding the step `ninja test` to the _generation_ phase. See https://codeberg.org/dnkl/foot/src/commit/4cb817237319b2bb9fc988c5440051b3468c3090/INSTALL.md#performance-optimized-pgo. Can you please try that?
Owner
https://codeberg.org/dnkl/foot/commit/a88a86f7
Poster

Those tests passed successfully across two attemps (I deleted the whole build folder between each attemps), they did not prevent the error from happening

Those tests passed successfully across two attemps (I deleted the whole build folder between each attemps), they did not prevent the error from happening
Owner

Hmm, ok. Let me test this again.

Hmm, ok. Let me test this again.
Owner

I tried following the instructions in INSTALL.md in a clean build directory, using clang/llvm 11.1.0, and it builds just fine for me.

That said, the functions log_init() and log_deinit() are defined in two places; subprojects/fcft/log.c, and log.c.

Long term, we should consider removing logging from fcft, and instead make it possible for users of the library to retrieve and log the information that currently only fcft has access to.

Short term, can you try applying the attached patch to fcft?

I tried following the instructions in INSTALL.md in a clean build directory, using clang/llvm 11.1.0, and it builds just fine for me. That said, the functions `log_init()` and `log_deinit()` are defined in **two** places; `subprojects/fcft/log.c`, and `log.c`. Long term, we should consider removing logging from fcft, and instead make it possible for users of the library to retrieve and log the information that currently only fcft has access to. Short term, can you try applying the attached [patch](https://codeberg.org/attachments/efa6d692-4406-4491-a658-c7c0cfaee38b) to **fcft**?
dnkl reopened this issue 6 months ago
Owner

Closing; with

This should be fixed.

Closing; with * https://codeberg.org/dnkl/fcft/pulls/18 * https://codeberg.org/dnkl/fcft/pulls/19 * https://codeberg.org/dnkl/fcft/pulls/20 This should be fixed.
dnkl closed this issue 6 months ago
Poster

Short term, can you try applying the attached patch to fcft?

Well, I installed it as a package using the AUR but I can try

> Short term, can you try applying the attached patch to fcft? Well, I installed it as a package using the AUR but I can try
Poster

This works perfectly 👍

This works perfectly :thumbsup:
Owner

Great! Thanks!

Great! Thanks!
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.