Sunday, October 05, 2025

Boot a custom (MacOS) kernel on MacOS Apple Silicon

The past several months I've been attempting to debug some (Java related) issues on macos. Most of these issues relate to the syscalls or native function calls that the JDK does on macos in order to provide implementation for some of the Java SE APIs. Naturally, debugging those issues means looking at macos specific system/kernel logs and sometimes even the macos kernel code. In some cases even that isn't enough.

Several parts (but not all) of macos kernel source is available as opensource code at https://opensource.apple.com/releases/. For each macos release, Apple updates that page with the relevant kernel versions which allows you to download the source or check out relevant tag from the linked GitHub repo. For example, here's the XNU repo https://github.com/apple-oss-distributions/xnu which represents parts of the kernel code that runs macos.

Given this, for debugging some of the issues, I've been looking to build the XNU kernel code and then have macos boot with that custom built kernel. That's anyway the end goal. It isn't easy though, because although there are build instructions on how to do it, those aren't always up to date. Plus, I'm told that it requires the exact set of tools and exact versions of those tools to be able to locally build a kernel and then have macos succesfully boot using that built kernel. I haven't yet managed to accomplish that.

For now though, I'll focus on a slightly different but related topic. For each release of macos, Apple provides a Kernel Debug Kit (KDK). Among other things, each KDK consists of pre-built kernels for different models of the macos hardware. Furthermore, for each of these macos models, the KDK has different variants of the kernel - "release" variant, "development" variant and "debug" variant. I haven't found an official document which states what exactly is different between each variant (or how exactly they are built), but from what I have read so far, the "release" variant is the one that matches the kernel that runs "production" macos. "development" and "debug" on the other hand, I'm guessing, are built using different flags to allow kernel developers and kernel extension developers to experiment with the kernel or kernel extensions.

Given that the KDK ships these pre-built kernels, it then means that I should be able to skip the part where I was considering building the XNU project and just jump to the step where I somehow point macos to use the kernel binary that's present in the KDK instead of the one it uses by default to boot macos. Unfortunately, there's no official documentation which explains how to do it. I have been looking for some hints and articles to find a way to boot a custom kernel binary and some of the articles I read had instructions that no longer worked. So I finally asked in the macos developer forum https://developer.apple.com/forums/thread/801605. A big thank you to Kevin for not just responding but also providing additional guidance on how to get this working.

In the rest of this article, I'll summarize the steps that I followed to have macos boot a kernel binary that's shipped in the Apple provided KDK. Do note that these steps are only applicable for Apple Silicon based macos systems (i.e. macos ARM64 architecture) and aren't applicable for Intel based macos systems.

Also note that these steps have been influenced heavily by the inputs provided in that developer forum thread as well as the following 2 articles:

https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html

and

https://khronokernel.com/macos/2023/08/08/AS-VM.html

The steps I note here are essentially a working and trimmed down version of what's explained in those articles.

It's very important to first know which KDK is relevant for use. Each release of XNU kernel has a corresponding KDK version. First, we will see what macos version we are currently running on. Note that macos OS version and XNU kernel version are not the same. Each has a different value.

In order to find out the macos version, run the "sw_vers" command:

sw_vers

ProductName:        macOS
ProductVersion:        15.6
BuildVersion:        24G84

So we are on 15.6 version of macos. Next we go to https://developer.apple.com/download/all and sign in with an Apple developer account. This page will list various products including the Kernel Debug Kit (KDK) for various versions of macos. We will locate the KDK for 15.6 of macos here. You will find it listed as "Kernel Debug Kit 15.6 build 24G84" (the build matches with the output we saw for sw_vers). Download the linked ".dmg" file from there and once downloaded, install it (like you would install any other dmg file). Upon installation, this version of the KDK will end up under the "/Library/Developer/KDKs/" directory on the filesystem. This specific KDK installation will be at "/Library/Developer/KDKs/KDK_15.6_24G84.kdk".

Let's set an environment variable named "KDK" to point to this location and for the rest of this article we will refer to that environment variable.

export KDK=/Library/Developer/KDKs/KDK_15.6_24G84.kdk

Next, let's list the kernel binaries that are shipped in this KDK. They will be present under the "$KDK/System/Library/Kernels/" directory. So:

ls $KDK/System/Library/Kernels/

This will print out several files, like:

kernel                kernel.kasan.t8112.dSYM
kernel.development        kernel.kasan.t8122
kernel.development.t6000    kernel.kasan.t8122.dSYM
kernel.development.t6020    kernel.kasan.t8132
kernel.development.t6030    kernel.kasan.t8132.dSYM
kernel.development.t6031    kernel.kasan.vmapple
kernel.development.t6041    kernel.kasan.vmapple.dSYM
kernel.development.t8103    kernel.release.t6000
kernel.development.t8112    kernel.release.t6000.dSYM
kernel.development.t8122    kernel.release.t6020
kernel.development.t8132    kernel.release.t6020.dSYM
kernel.development.vmapple    kernel.release.t6030
...


Each of these are either kernel binaries or debug symbols for the kernel binaries. And each of these binaries are for a different model of macos - the text "t8122", "t8132" and such is the macos model name.

So at this point, we have identified the macos operating system version we are on, then downloaded the corresponding KDK version for that macos version and installed it. The next step is to identify the macos model of our system. To do that we use the "uname -v" command:

uname -v

This prints:

Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:30 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6020

In the above output you will notice "RELEASE_ARM64_T6020". The T6020 is the macos model of this system and that's the one we are interested in. Note that the output will be different depending on which macos model you are running on and it is very important to make note of the exact model from the output.

Now that we know the model of our macos, let's again list the contents of our KDK installation and this time "grep" only for this T6020 model:

ls $KDK/System/Library/Kernels/ | grep -i T6020

This prints:

kernel.development.t6020
kernel.kasan.t6020
kernel.kasan.t6020.dSYM
kernel.release.t6020
kernel.release.t6020.dSYM

You will notice that there are 3 variants of the kernel available for this macos model in this KDK installation - "development", "kasan" and "release". Furthermore, the "release" and "kasan" variants each have a separate debug symbols file. For the "development" variant, there isn't a debug symbols file and I'm guessing it's because the "development" variant is compiled with debug symbols enabled within the binary. But that's merely a guess, like I noted previously, I haven't yet found the exact details of what each of these variants represent or how they are compiled. Curiously, there isn't a "debug" variant. There is a "kasan" variant, but I hadn't heard about that one before and I don't know what it is. But that's not important for now since our experiment is merely to boot a custom kernel binary instead of the default one. For this experiment, we will choose the "development" variant as the kernel binary we want to boot, so the "$KDK/System/Library/Kernels/kernel.development.t6020" file.

At this point, we have everything available locally to start the steps that are necessary to initiate the boot. First thing we need to do is create a "kext collection" using this kernel binary. To do that we use the "kmutil create" command ("kmutil" is a tool that's by default shipped in macos). Before doing that, I will create a directory on my filesystem where I would like to have this "kext collection" generated. This will be the file which will eventually be used by macos to boot the system. You can specify a location of your choice, and I chose "/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/" as the directory:

mkdir -p /Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/

Next we will use the "kmutil create" command to create the kext collection in this directory. The following command is borrowed from the articles I linked previously:

kmutil create \
        --arch arm64e \
        --no-authorization \
        --variant-suffix development \
        --new boot \
        --boot-path /Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/macos-15.6.kc \
        --kernel $KDK/System/Library/Kernels/kernel.development.t6020 \
        --repository $KDK/System/Library/Extensions \
        --repository /System/Library/Extensions \
        --repository /System/Library/DriverExtensions \
        --explicit-only $(kmutil inspect -V release --no-header | grep -v "SEPHiber" | awk '{print " -b "$1; }')



Except for the "--variant-suffix", "--boot-path" and the "--kernel" options, the rest of them will usually remain the same if you had to run it on your Apple Silicon based macos system.

We are choosing the "development" variant of the kernel, so I use "development" as the "--variant-suffix" option's value.

For the "--boot-path", I provide "/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/macos-15.6.kc" as the value. Notice that "/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/" is the directory we just created a few lines above to use it as a destination directory for this generated "kext collection". Also notice that for this "--boot-path" option value, I use "macos-15.6.kc" as the file name. It can be anything, I chose this value to be clear what version of macos it corresponds to. When "kmutil create" successfully completes and creates the "kext collection", you will find a file named "macos-15.6.kc.development" where "development", in that file name, is the value you provided for the "--variant-suffix" option.

For the "--kernel" option, we point to the "$KDK/System/Library/Kernels/kernel.development.t6020" kernel file that we previously listed and identified for the current macos model, under the KDK installation directory.

The rest of the options I left as-is.

When you run this command and after it completes successfully, you should find the generated "kext collection" file at "/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/macos-15.6.kc.development" (or whatever option values you provided for "--boot-path" and "--variant-suffix").

At this point we now have a bootable file for macos boot process to use. Our next step is to configure macos to use this file. To do this we will first have to boot macos in recovery mode. So we first shutdown macos and then start macos in recovery mode by pressing down and holding the "power" button. When you do that, you will see a message which says macos is loading options. Continue to hold down the "power" button. This is a standard way to boot into recovery mode in macos, so I won't go into this detail. Once it boots in recovery mode, you will see a screen which has a "Options" icon. Click on it and then it will show a "Continue" button. Click on the "Continue" button. That will then show a screen with various menu options (at the top). Choose the "Utilities" -> "Terminal" menu option. It will open the terminal window.

On the terminal window, we will first disable System Integrity Protection (SIP) using the "csrutil" command:

csrutil disable

Confirm the action when prompted and provide the user/password if prompted.

Then we will run a command which will relax the restriction which prevents macos from booting arbitrary kernel binaries. We use the "bputil" command as follows:

bputil --disable-boot-args-restriction

Confirm the action when prompted and provide the user/password if prompted.

And now the last step where we configure macos to boot the kext collection file that we generated previously at "/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/macos-15.6.kc.development". We use the "kmutil configure-boot" command for that:

kmutil configure-boot --volume "/Volumes/Macintosh HD" --custom-boot-object "/Volumes/Macintosh HD/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/macos-15.6.kc.development"

The "--volume" options points to the target volume which is being configured. Typically, there's only one volume on the system and usually it's named "Macintosh HD", so the path is almost always "/Volumes/Macintosh HD". But if you have something different or if you have more than one volume on your system, then choose the appropriate value for this option.

The "--custom-boot-object" is the full path to the kext collection file that we created using the kernel binary shipped in the KDK. The full path includes the volume on which that kext collection file resides. So "/Volumes/Macintosh HD/Users/me/macos-kernel-experiments/macos-15.6/kext-collection/development/macos-15.6.kc.development".

It's important to note that the volume name and the path name may have space characters in their name. So remember to enclose these values in double quotes (like in the above example) or use appropriate escaping using the backslash (\) character.

Also note that the terminal window on which you type these commands has a very small font (at least on my instance) so it isn't easy to type in this lengthy command/text there. This became a bit of hassle when I was experimenting with these commands. So before booting into recovery mode, I created a ".sh" file beforehand with the complete "kmutil configure-boot" command and saved it at the "/Users/me/configure-boot.sh". Then instead of typing that "kmutil configure-boot" command manually, I ran:

bash "/Volumes/Macintosh HD/Users/me/configure-boot.sh"

from the terminal in the recovery mode. To be clear, creating this script file and using it is completely optionally. If you prefer typing the "kmutil configure-boot" command, that's fine too.

At this point, after the "kmutil configure-boot" completes successfully, all the steps needed to boot this custom built kernel are now done. Next, we shutdown and restart the system as usual (there's the Shutdown/restart option in the top menu).

When the system starts next, it will use the "macos-15.6.kc.development" binary that we built. To verify that it indeed is using this custom kernel binary, we can run the following command from the terminal once we have logged in to the system:

sysctl kern.osbuildconfig

This should output:

kern.osbuildconfig: development

which is the variant we built and configured to boot. This implies that we have successfully booted a custom variant of the kernel that is shipped in the KDK.

Do note that, sometimes, booting of this custom kernel binary may not be successful. When that happens, due to issues with the binary or some other issue that causes the boot process to fail, then during the boot you will see the Apple icon and it will stay up for a while before the system reboots and this will keep happening in a (never ending) loop. That's a sign that something has gone wrong and you won't be able to use this custom kernel binary. So to restore back to your normal kernel binary, you will have to reboot into recovery mode, like you did the previous time, by holding down the power button and waiting for the screen to show the "Options" screen and then clicking "Continue" button. Once you are into recovery mode, this time you will choose the "Utilities" -> "Startup Security Utility" menu option. The screen that then shows up will have 3 options and would have selected the "Permissive" mode selected (because we ran the csrutil and the bputil commands previously). Since the custom kernel binary that we have configured for the boot is not functioning, we will have to select the "Restricted" mode in that screen options. Choosing "Restricted" mode will prevent the use of the custom kernel binary and instead will use the one shipped by macos by default, when we boot next (and that's what we want). Save/Submit/Continue with the "Restricted" mode and, if prompted, provide the user/password to apply the change. After that's done, Shutdown/restart from the top menu option and let the system boot in normal mode. This will boot you in the release version of the kernel that macos ships by default and you can verify that by running the following command from the terminal once you have logged in to the system:

sysctl kern.osbuildconfig

That will output:

kern.osbuildconfig: release

That completes the steps needed to boot a custom kernel variant on macos and the step to recover if that process fails. Do note that we did not build the XNU project from source and instead reused a previously built kernel variant shipped in the KDK. Building the XNU from source (with trivial code modifications for experiment/debugging) is still on my TODO list but I am told that it isn't straightforward nor guaranteed to work. So that's for some other time.

No comments: