Modern C porting

From Gentoo Wiki
Jump to:navigation Jump to:search

Set of notes for Modern C porting.

This has two phases:

  1. Preparation for Clang 16 and GCC 14 (now)
  2. Preparation for C23 becoming default (in the future, don't worry about this for now)

TODO: mention GNU_SOURCE and other FTMs

What changed?

All of these were either invalid in C99, invalid even in C89, or extremely dubious. Compilers just tolerated them as quasi-extensions for a while (until now) to avoid disruption.

  • Clang 15 makes the following errors by default:
    • -Werror=int-conversion
  • Clang 16 (released March 2023) makes the following errors by default:
    • -Werror=implicit-function-declaration
    • -Werror=implicit-int
    • -Werror=incompatible-function-pointer-types (GCC does not have a specific equivalent error, use -Werror=incompatible-pointer-types instead when testing)
  • GCC 14 (to be released appx. May 2024) is likely to do the same.

What will change in a few years?

Please do not worry about C23 for now.

  • C2x (likely C23) makes additional changes like removing certain deprecated prototypes. This is not the main priority for now.

Why does it matter?

  • Lots of packages fail to build with these settings.
  • Sometimes packages build successfully but their ./configure scripts have misdetected features or otherwise made the wrong conclusion about the system because they expect a test to succeed when it now fails.

Examples

FAQ

Where can I find a list of Gentoo bugs to hack on?

See bug #870412 and the list here.

Additionally, for C23 preparedness (see above), see bug #880545. This is lower priority for now.

How do I reproduce these bugs?

In general:

  1. Use Clang 16 and set CC=clang-16, or
  2. Use Clang 15 and set CC=clang-15 and =sys-devel/clang-common-15* stricter in /etc/portage/package.use/clang, or
  3. Use GCC and set -Werror=implicit-function-declaration -Werror=implicit-int -Werror=int-conversion -Werror=incompatible-pointer-types

configure or build system bugs

Developers may need to follow the above to setup their environment, run ./configure, then:

  • grep config.log, or
  • inspect ./configure, or
  • check other build system-generated files if the problem does not appear in build.log.

A /etc/portage/bashrc hook is available to save logs in /var/tmp/clang to help capture issues from homebrew configure scripts which do not log. In order to use this without root rights with the ebuild command, make sure that users have writing privileges for /var/tmp/clang.

How do I install Clang 16?

Tip
Remember that it's fine to use Clang 15 or GCC with specific settings described above, so there's no specific need to use Clang 16.

Clang 16 has been released, so simply:

root #emerge sys-devel/clang:16


Is this cosmetic?

No!

Implicit function declarations can affect code generation. They've been a long-standing cause of runtime failures like crashes. They are particularly a problem if the calling convention for an architecture is sufficiently "different", e.g. Apple's ARM64 ABI.

Even on amd64, it can cause problems: if a function returns a _Bool in reality but the prototype is missing, the compiler will assume int. On amd64, this causes messy corruption because there's no obligation for a _Bool to have filled the remaining bits correctly.

A new revision of the ebuild is required for fixing these bugs because of the possible runtime effects.

How do I fix these bugs?

All of these fixes require a new revision ("revbump") for the reasons described above. Also, developers want to know quickly if the fix is somehow insufficient, and a new revision helps to weed out any problems.

-Wimplicit-function-declaration

  • Do not pass -Wno-error=...
  • GCC will usually helpfully emit a 'fixit' (an annotation to the warning/error with the missing header).
  • Add the relevant #include - determine this possibly by looking at man pages for the missing functions, or grepping in the codebase
  • Internal functions
    • grep the codebase for uses of the function to determine the correct return type.
    • Sometimes packages are just missing includes for their own internal functions
    • Sometimes adding a prototype into an internal header is needed

-Wimplicit-int

  • Do not pass -Wno-error=...
  • grep the codebase for uses of the function to determine the correct return type.
  • Do not assume it is supposed to be an int.

-Wincompatible-function-pointer-types

  • grep the codebase for uses of the function to determine the correct return type.
  • This can often be somewhat convoluted and may require filling in various prototypes both to head off possible C23 issues but also to make the compiler give better errors
    • Sometimes it's useful to temporarily put in the _wrong_ type just to get a better error, rather than _no_ type
  • These bugs are the hardest to solve and often require understanding the intent of the software's author. It's okay to feel stuck with these.
  • Many of these end up being last-rite candidates because they're abandoned upstream and have other code smells.
  • It's not always possible (or at least practical) to determine the correct types if the codebase is particularly old because they relied on ambiguity.
    • In some extreme cases, it may be okay to pass -Wno-error=incompatible-function-pointer-types, but please avoid it.
    • If doing this, make sure there's an upstream report, or if upstream is gone, that there's truly no alternative to this software available (so we can last-rite).

-Wdeprecated-non-prototypes

  • It'd be swell if you fix these but you don't have to for Clang 16 / GCC 14. But they're easy to do usually.
  • Add -std=gnu89 instead in CFLAGS in the ebuild if you don't fix them.

-Wstrict-prototypes

  • This is future proofing and can help with C23 onwards but is not strictly required right now.
  • Add -std=gnu89 instead in CFLAGS in the ebuild if you don't fix them.

Do I have to send patches upstream?

  • If upstream still exists, yes, please do. We need other distributions to do the same as well. This is a huge task and we can't be needlessly duplicating work. It's also just part of being a good FOSS citizen, of course.
  • If upstream is completely gone, of course, you need not feel guilt.

Tips & Tricks

Fixing K&R C declarations

Often errors are caused by old K&R style function definitions. So this:

CODE 
int
 REmatch(pattern, start, end)
 char *pattern;
 int start,end;
 {
    ...
 }

needs to be reworked into this:

CODE 
int
 REmatch(char *pattern, int start, int end)
 {
    ...
 }

This is not a very hard task, but it becomes exhausting when doing this for a larger project.

dev-util/cproto can automate this. For a given file, myCfile, cproto will convert (and return the prototypes of all functions it can find) with

user $cproto -a myCfile.c

Or for all the .c-files in a project:

user $find ./ -name "*.c*" | xargs cproto -a

Using Clang on a package-basis

Clang can be used only for specific packages by leveraging Portage's package.env mechanism. Files similar to the following should be created.

FILE /etc/portage/env/clang_fixes/use_clang.confClang overrides
# Build packages with clang instead of gcc
CC="clang"
CXX="clang++"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"

LDFLAGS="${LDFLAGS} -fuse-ld=lld -Wl,--as-needed"
FILE /etc/portage/package.env/clang_fixesTell Portage to use Clang for some package
# Bug 000000
category/package clang_fixes/use_clang.conf

Including a link to the relevant bug as a comment in the package.env entry makes it easier to keep track of the context for that package.

Using Portage to find build system bugs

Portage (as of version 3.0.45.1) will scan the standard configure logs (config.log, CMakeError.log, meson-log.txt) for configure-time implicit function declarations as part of a post-install QA check. Any results that found are given as a QA message as well as logged into qa.log in the package build tree in a script-friendly format.

If the message is a false positive (e.g. BSD-only functions), mark them as such in QA_CONFIG_IMPL_DECL_SKIP in the ebuild.

If the message is from tests built in to autoconf (not from the package's own configure.ac or m4 macros), then try eautoreconf.

FILE /var/tmp/portage/dev-lang/python-3.10.9-r1/temp/build.logExample QA message
... snip ...
>>> Completed installing dev-lang/python-3.10.9-r1 into /var/tmp/portage/dev-lang/python-3.10.9-r1/image

 * Final size of build directory: 130296 KiB (127.2 MiB)
 * Final size of installed tree:  127600 KiB (124.6 MiB)

 * Verifying compiled files for python3.10
 * QA Notice: Found the following implicit function declarations in configure logs:
 *   /var/tmp/portage/dev-lang/python-3.10.9-r1/work/Python-3.10.9/config.log:10419 - chflags
 *   /var/tmp/portage/dev-lang/python-3.10.9-r1/work/Python-3.10.9/config.log:10766 - lchflags
 * Check that no features were accidentally disabled.
strip: x86_64-pc-linux-gnu-strip --strip-unneeded -N __gentoo_check_ldflags__ -R .comment -R .GCC.command.line -R .note.gnu.gold-version
... snip ...
FILE /var/tmp/portage/dev-lang/python-3.10.9-r1/temp/qa.logExample qa.log
- tag: config.log-impl-decl
  data:
    line: "10419"
    func: "chflags"
  files:
    - "/var/tmp/portage/dev-lang/python-3.10.9-r1/work/Python-3.10.9/config.log"
- tag: config.log-impl-decl
  data:
    line: "10766"
    func: "lchflags"
  files:
    - "/var/tmp/portage/dev-lang/python-3.10.9-r1/work/Python-3.10.9/config.log"

See also

Resources

Retrieved from "https://wiki.gentoo.org/index.php?title=Modern_C_porting&oldid=1233877"