User:Pietinger/Tutorials/Kernel Hardening with KSPP

From Gentoo Wiki
Jump to:navigation Jump to:search
Note
Even though this page is in the user namespace, corrections and additions are much appreciated! This is simply wiki policy, this page can be moved to the main wiki as soon as it achieves critical mass more.

No, KSPP does not mean Kerbal Space Program Project !

Tutorial: Kernel Hardening with KSPP

This tutorial shows an easy way to harden your kernel. Only our Gentoo default sources will be used.

Note
If you think you dont need it because you have Hardened-Sources you are wrong. Hardened-Sources has no hardened kernel; you must harden your kernel by yourself !

Maybe this article will become obsolete when kernel version 6.7 is available, because then we will have this:

https://lore.kernel.org/lkml/202310300946.C0E11C5@keescook/

Also described here: https://www.phoronix.com/news/Linux-6.7-Hardening

Update: I said that it could become obsolete with kernel version 6.7. Since I have now installed 6.7.0, I immediately looked at the file /usr/src/linux-6.7.0-gentoo/kernel/configs/hardening.config and the file /usr/src/linux-6.7.0-gentoo/arch/x86/configs/hardening.config and realized that only the most primitive options are activated.

This means that the KSPP page is much more comprehensive and should still be our target.

Link to KSPP

In every case you need this: https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project/Recommended_Settings

Print the whole page or copy it into a text-file; you will need it later.

The following patch has not (yet) been included in the kernel, but contains more detailed explanations of some options (for reading only): https://lore.kernel.org/kernel-hardening/1536516257-30871-1-git-send-email-s.mesoraca16@gmail.com/

In this video from the "Linux Security Summit 2021" Kees Cook (who also updates this KSPP Page) explains why it is so important to harden your kernel and also explains the progress made from Linux version 5.3 until 5.15: https://www.youtube.com/watch?v=-Binxid8t_8

(Maybe interesting: Kees Cook & Gustavo A.R. Silva explain the "Progress On Bounds Checking in C and the Linux Kernel" (Linux Security Summit North America 2023): https://www.youtube.com/watch?v=V2kzptQG5_A )

Transfer KSPP settings into your kernel

Of course you can set all these settings manually into your kernel. This you have to do if you dont use Gentoo-Sources. With Gentoo-Sources there is an easy way ... Maybe you have have already seen this kernel setting:

KERNEL
Gentoo Linux  --->
    [ ] Kernel Self Protection Project

After enabling this option maybe you will see this (the second option only appears when you have enabled the first):

KERNEL
--- Kernel Self Protection Project
[*]   Enable Kernel Self Protection Project Recommendations
[ ]     X86_64 KSPP Settings

Maybe you dont have these two options. Why ?

If you read all recommended settings in KSPP Homepage you will see there are many kernel options which must be disabled. Disabling a kernel option with a Kconfig file is not possible. The only way to proof if something is disbled is a check in Kconfig with "... !options". ! means: NOT. Now look into /usr/src/linux/distro/Kconfig. You will see in the second half of this file a line like this:

FILE /usr/src/linux/distro/Kconfig of Kernel Version 6.6.1
depends on GENTOO_LINUX && !ACPI_CUSTOM_METHOD && !COMPAT_BRK && !PROC_KCORE && !COMPAT_VDSO && !KEXEC && !HIBERNATION && !LEGACY_PTYS && !X86_X32_ABI && !MODIFY_LDT_SYSCALL && GCC_PLUGINS && !IOMMU_DEFAULT_DMA_LAZY && !IOMMU_DEFAULT_PASSTHROUGH && IOMMU_DEFAULT_DMA_STRICT && SECURITY && !ARCH_EPHEMERAL_INODES  && RANDSTRUCT_PERFORMANCE

If one of these options are enabled this line cause a hiding of this option. For example if you have enabled Hibernation you will not see the kernel option "Enable Kernel Self Protection Project Recommendations". This leads to my recommendation for a ...

Course of action

1. First of all disable all kernel options with the description in the link you have now. Yes, also "DEVMEM". If you search for "DEVKMEM" you will not find it if you have kernel 5.15 (or newer); kernel developer have revoked it. Do not disable IA32 if you have a Multilib-System ! Do not disable Module-Support if you have kernel modules enabled with {M}. You can disable Module-Support only if you have a monolithic kernel.

2. If you have problems to find all necessary options do all settings described in the next chapter.

3. Now you are able to enable MANY options with these two options in our Gentoo section. Cancel all these many options (best with content of usr/src/linux/distro/Kconfig; but you can use also <HELP>) on your print output (or in your text file).

4. Check if there are some options you dont have enabled automatically. A diff between KSPP version 2023-10-20 and our Gentoo settings for kernel version 6.6.1 will show these missing setings - further updates to the KSPP version from 2024-04-26 can be found in the chapter below:

1)
CONFIG_DEBUG_VIRTUAL=y
2)
# Enable "lockdown" LSM for bright line between the root user and kernel memory.
CONFIG_SECURITY_LOCKDOWN_LSM=y
CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y
CONFIG_LOCK_DOWN_KERNEL_FORCE_CONFIDENTIALITY=y
3) (automatically enabled in .../distro/Kconfig of kernel version 6.9)
# Enable trapping bounds checking of array indexes (since v5.11). All the other UBSAN checks should be disabled.
CONFIG_UBSAN=y
CONFIG_UBSAN_TRAP=y
CONFIG_UBSAN_BOUNDS=y
CONFIG_UBSAN_SANITIZE_ALL=y
# CONFIG_UBSAN_SHIFT is not set
# CONFIG_UBSAN_DIV_ZERO is not set
# CONFIG_UBSAN_UNREACHABLE is not set
# CONFIG_UBSAN_BOOL is not set
# CONFIG_UBSAN_ENUM is not set
# CONFIG_UBSAN_ALIGNMENT is not set
# This is only available on Clang builds, and is likely already enabled if CONFIG_UBSAN_BOUNDS=y is set:
CONFIG_UBSAN_LOCAL_BOUNDS=y
4)
# Do not ignore compile-time warnings (since v5.15)
CONFIG_WERROR=y
5)
# Disable DMA between EFI hand-off and the kernel's IOMMU setup.
CONFIG_EFI_DISABLE_PCI_DMA=y
6)
# Enable feeding RNG entropy from TPM, if available.
CONFIG_HW_RANDOM_TPM=y
7)
# Wipe RAM at reboot via EFI.
CONFIG_RESET_ATTACK_MITIGATION=y
8)
# This needs userspace support, and will break "regular" distros. See: https://github.com/tych0/huldufolk
CONFIG_STATIC_USERMODEHELPER=y
9)
# Block TTY stuffing attacks (this will break screen readers, see "dev.tty.legacy_tiocsti" sysctl below).
# CONFIG_LEGACY_TIOCSTI is not set
10)
# Keep root from altering kernel memory via loadable modules.
# CONFIG_MODULES is not set

# But if CONFIG_MODULE=y is needed, at least they must be signed with a per-build key.
CONFIG_DEBUG_SET_MODULE_RONX=y (prior to v4.11)
CONFIG_STRICT_MODULE_RWX=y (since v4.11)
CONFIG_MODULE_SIG=y
CONFIG_MODULE_SIG_FORCE=y
CONFIG_MODULE_SIG_ALL=y
CONFIG_MODULE_SIG_SHA512=y
CONFIG_MODULE_SIG_HASH="sha512"
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"

x86_64

11)
# Remove additional (32-bit) attack surface, unless you really need them.
# CONFIG_COMPAT is not set
# CONFIG_IA32_EMULATION is not set
# CONFIG_X86_X32 is not set
# CONFIG_X86_X32_ABI is not set
12)
# Disallow allocating the first 64k of memory.
CONFIG_DEFAULT_MMAP_MIN_ADDR=65536
13)
# Straight-Line-Speculation
CONFIG_SLS=y (MY NOTE: this option was renamed with 6.9. to: CONFIG_MITIGATION_SLS)

My comments:

1. This costs some performance. Decide yourself.

2. See the big warning !

3. Enable it.

4. This could make some problems when doing cross compile (I dont do and therefore I have enabled it).

5. See the big warning !

6. Only senseful if you have such a hardware module (I dont have; so its disabled).

7. See the big warning !

8. Enable it and make sure that the PATH is empty. But test it if everything still works afterwards, because I got a report that iptables does not work properly anymore if it has to load netfilter modules (not verified because I have all netfilter modules statically configured in the kernel and therefore have no problem with iptables AND this option).

9. Read the <Help> and test it ... ;-)

10. I have disabled the module support and have a monolithic kernel ... after I found out which modules I need for my firewall ;-) If you need module support (e.g. for nvidia) sign your kernel modules.

11. You can do this only if you have a No-Multilib system. See also: User:Pietinger/Tutorials/Manual_Configuring_Kernel_Version_6.6#32-bit_support

12. This is very senseful and we have done it already.

13. If security is more important than performance enable it (I have it enabled) - All mitigitations cost some performance; dont use it for a gaming system:

KERNEL
[*] Mitigations for speculative execution vulnerabilities  --->
    [*]   Mitigate Straight-Line-Speculation
Warning
Since 2023-10-20 we have new recommendations in KSPP. One of them is enabling LOCKDOWN. Do this only if you have signed your Kernel modules (see here: Signed_kernel_module_support ), OR you have a monolithic kernel (see more here: Kernel_Modules#Going_completely_.22module-less.22 ) !
Warning
If you boot a stub-kernel directly from UEFI, these settings could harm your boot sequence (BIOS hangs): CONFIG_EFI_DISABLE_PCI_DMA and CONFIG_RESET_ATTACK_MITIGATION. Best is to try only one of them and (if ok) then enable the second option.

5. Dont forget some settings in SYSCTL. You can copy it from KSPP directly in (just add them at the end):

root #nano -w /etc/sysctl.conf

If you have problems when emerging some packages (e.g. dev-lang/go) because of this:

FILE /etc/sysctl.conf
kernel.yama.ptrace_scope = 3

Then set it to 2 (or even 1). Please see more in: /usr/src/linux/Documentation/admin-guide/LSM/Yama.rst

6. In KSPP there also some settings via kernel command line parameter, like e.g.: "nosmt" which would disable SMT (Intel call it "Hyperthreading"). In my point of view this is only necessary if it is a server and you have virtual machines. A desktop doesnt need it. But it is your decision ! Another kernel command line parameter: "slab_nomerge # Disable slab merging (makes many heap overflow attacks more difficult)" can be set also directly in the kernel configuration; see in chapter KSPP 2024-04-26.

Minimum kernel configuration to get both KSPP options

... for a X86_64 system. You must enable "expert users" to be able to reach all necessary options. After enabling "expert users" check if "Framebuffer Console support" still is set. If not enable it again. Now you have to DISABLE and ENABLE all these:

Kernel 5.15

KERNEL
General setup  --->
    [*] Configure standard kernel features (expert users)  --->

Processor type and features  --->
    [ ] Enable vsyscall emulation
    [ ] /dev/cpu/*/msr - Model-specific register support
    [ ] kexec system call
    [ ] Enable the LDT (local descriptor table)

Power management and ACPI options  --->
    [ ] Hibernation (aka 'suspend to disk')

General architecture-dependent options  --->
    [*] GCC plugins  --->

Memory Management options  --->
    [ ] Disable heap randomization

Device Drivers  --->
    Character devices  --->
        [ ]   Legacy (BSD) PTY support
    Graphics support  --->
        Console display driver support  --->
            [*] Framebuffer Console support
    [*] IOMMU Hardware Support  --->
        IOMMU default domain type (Translated - Strict)  --->

File systems  --->
    Pseudo filesystems  --->
        [ ]   /proc/kcore support

Security options  --->
    [*] Enable different security models

Kernel 6.1

Do all settings from 5.15 and ADDITIONALLY:

KERNEL
Security options  --->
    Kernel hardening options  --->
        Randomize layout of sensitive kernel structures (Limit randomization of structure layout to cache-lines)  --->

Kernel 6.6

Do all settings from 5.15 and 6.1. With Kernel 6.6 the option "KEXEC" was moved from "Processor type and features" to "General setup":

KERNEL
General setup  --->
    Kexec and crash features  --->
        [ ] Enable kexec system call
        [ ] Enable kexec file based system call
        [ ] kernel crash dumps

Bad option

If you have enabled the really "bad" option "DEBUG_FS" it could be possible you must disable another option, because "ACPI_CUSTOM_METHOD" depends on "DEBUG_FS"; but it is much more better to disable "DEBUG_FS" (then you dont have "ACPI_CUSTOM_METHOD"). If you dont have enabled "DEBUG_FS" you will find this option only by pressing z and it is disabled by default (and you have nothing to do here):

KERNEL
Power management and ACPI options  --->
    [*] ACPI (Advanced Configuration and Power Interface) Support  --->
        - -   Allow ACPI methods to be inserted/replaced at run time

ACPI_CUSTOM_METHOD was completely removed with 6.9 and can therefore no longer cause problems. :-)

Slow System start ?

If you boot the first time with these new settings you will notice that your kernel needs longer to start. Before it took 1.2 seconds (until "Run /sbin/init as init process") and now kernel needs 2.5 seconds. If you think this is crazy; these settings have halved my CPU performance ... then don't worry ... and look carefully into your dmesg output and examine the timestamps. You will find something like this:

root #dmesg
[...]
[    0.030070] mem auto-init: stack:all(zero), heap alloc:on, heap free:on
[    0.030071] mem auto-init: clearing system memory may take some time...
[    0.030076] software IO TLB: area num 8.
[    1.358628] Memory: 16087256K/16653428K available (14343K kernel code, 1960K rwdata, 2456K rodata, 1564K init, 2888K bss, 565912K reserved, 0K cma-reserved)
[...]

(As you can see I have 16 GB RAM; machine is an Intel i7; but it doesn't matter how many cores your CPU has).

Yes, for security reasons your kernel will clear now your whole memory (with ONE Core of your CPU). If you have a 64 GB RAM machine you can expect it took aprox. 5 seconds.

KSPP 2024-04-26

All of the following options - except the first one, because disabling you have to do yourself - are automatically enabled in .../distro/Kconfig of kernel version 6.9. If you have stable 6.6 you really should enable these also.

KERNEL Kernels 5.15+
Memory Management options  --->
    SLAB allocator options  --->
        [ ] Allow slab caches to be merged
KERNEL Kernels 6.1+
Kernel hacking  --->
    Memory Debugging  --->
        [*] Check for invalid mappings in user page tables
        [*]   Enforce the page table checking by default
KERNEL Kernels 6.6+
# If a query with "cpuid" gives you: "CET_SS: CET shadow stack = true" then enable this:
Processor type and features  --->
    [*] X86 userspace shadow stack

Memory Management options  --->
    SLAB allocator options  --->
        [*] Randomize slab caches for normal kmalloc

This option I have already described in User:Pietinger/Tutorials/Manual_Configuring_Kernel_Version_6.6:

KERNEL Kernels 6.6+
# If a query with "cpuid" gives you: "IBRS/IBPB: indirect branch restrictions  = true" then enable this:
Processor type and features  --->
    [*] Indirect Branch Tracking

Two options are only impportant if you use SysReq (I have it disabled) and module support (I am using a monolithic kernel):

CONFIG_MAGIC_SYSREQ_DEFAULT_ENABLE=176

CONFIG_MODULE_FORCE_LOAD is not set

More Hardening Options

When Google disables IO_URING then I do the same ;-) see more here: [[1]]

KERNEL Kernels 5.15+
General setup  --->
    [*] Configure standard kernel features (expert users)  --->
        [ ]   Enable IO uring support

With kernel version 6.9 we have a new option for UBSAN: CONFIG_UBSAN_SIGNED_WRAP. It's there for development testing, but I have enabled it nevertheless. Together with the old options it looks like this:

KERNEL Kernels 6.9+
Kernel hacking  --->
    Generic Kernel Debugging Instruments  --->
        [*] Undefined behaviour sanity checker  --->

--- Undefined behaviour sanity checker
[*]   Abort on Sanitizer warnings (smaller kernel but less verbose)
[*]   Perform array index bounds checking
[ ]   Perform checking for bit-shift overflows
[ ]   Perform checking for integer divide-by-zero
[*]   Perform checking for signed arithmetic wrap-around
[ ]   Perform checking for non-boolean values used as boolean
[ ]   Perform checking for out of bounds enum values

Check it with kernel-hardening-checker

At the moment it is not stable, so you must add this package to your /etc/portage/package.accept_keywords

root #emerge -pv kernel-hardening-checker

Start it with:

OLD Version->

root #kconfig-hardened-check -c /usr/src/linux/.config

Version 0.6.6->

root #kernel-hardening-checker -c /usr/src/linux/.config

Do not worry if some points are displayed as "Fail". You can only have AMD or Intel MCE ... or IOMMU ... ;-) See my result:

[+] Kconfig file to check: /usr/src/linux/.config
[+] Detected architecture: X86_64
[+] Detected kernel version: 6.1
[+] Detected compiler: GCC 130201

              option name               | type  |desired val | decision |      reason      | check result

CONFIG_BUG                              |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_SLUB_DEBUG                       |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_THREAD_INFO_IN_TASK              |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_GCC_PLUGINS                      |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_IOMMU_SUPPORT                    |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_STACKPROTECTOR                   |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_STACKPROTECTOR_STRONG            |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_STRICT_KERNEL_RWX                |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_STRICT_MODULE_RWX                |kconfig|     y      |defconfig | self_protection  | OK: CONFIG_MODULES is "is not set"
CONFIG_REFCOUNT_FULL                    |kconfig|     y      |defconfig | self_protection  | OK: version >= 5.5
CONFIG_RANDOMIZE_BASE                   |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_VMAP_STACK                       |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_DEBUG_WX                         |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_WERROR                           |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_X86_MCE                          |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_X86_MCE_INTEL                    |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_X86_MCE_AMD                      |kconfig|     y      |defconfig | self_protection  | FAIL: is not found
CONFIG_MICROCODE                        |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_RETPOLINE                        |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_SYN_COOKIES                      |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_X86_SMAP                         |kconfig|     y      |defconfig | self_protection  | OK: version >= 5.19
CONFIG_X86_UMIP                         |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_PAGE_TABLE_ISOLATION             |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_RANDOMIZE_MEMORY                 |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_INTEL_IOMMU                      |kconfig|     y      |defconfig | self_protection  | OK
CONFIG_AMD_IOMMU                        |kconfig|     y      |defconfig | self_protection  | FAIL: "is not set"
CONFIG_BUG_ON_DATA_CORRUPTION           |kconfig|     y      |   kspp   | self_protection  | OK
[...]

Dont worry if CONFIG_DEBUG_CREDENTIALS fails. It was removed in kernel version 6.6: [[2]] and there is already an issue reported: [[3]]

Note
This is my third try for creating a wiki article. Maybe some formatting is wrong. Because it is a tutorial and not a reference guide I ignored the standard of not writing in first or second person. Yes, my english is very poor.