Guidelines/BuildingPatchingAndMaintainingTheApertisKernel

From Apertis
Jump to: navigation, search

This guide will explore a number of workflows for retrieving, patching, building and packaging the Apertis kernel. This guide will suggest workflows both utilising the Debian infrastructure on which Apertis is built and a workflow that eschews the Debian infrastructure and thus is usable by third party integrators who may need access to a patched kernel tree for integration, but who's workflow would make using the Apertis infrastructure difficult.

This document is targeted towards development of Apertis and where possible, we would strongly advise using the Apertis SDK image and the Debian tooling provided for it, as this workflow will be more efficient ("option 1" where present). We provide a guide to get to a built kernel (though not packaged) for those not able to use the provided infrastructure ("option 2").

Required knowledge

Prior experience with the following topics will help with understanding the workflows presented below:

  • Familiarity with using git
  • Building and developing the Linux kernel
  • Debian packaging
  • Familiarity with basic Linux commands

Building locally patched kernel

Generating patched kernel source

Apertis provides a version of the Linux kernel, which is based on the upstream kernel with a number of patches applied to it and packaged using Debian packaging. We want to use this version of the kernel as a starting point for our modifications. As with most Debian packages, the upstream source is stored separately from any distribution specific modifications, scripting and metadata. In the case of the Apertis kernel, all the required components are stored in a single git repository, however on different branches. To retrieve the git repository run the following git command:

$ git clone https://gitlab.apertis.org/pkg/target/linux
$ cd linux

Ensure that you have the branch you're interested in checked out (in this case, we're interested in the v2019: Apertis release):

$ git checkout apertis/v2019

Before continuing, we will create a fresh branch into which we will create any changes - that way we have the original branch available which will help with updating later. We will assume we have a project called "foo" for the sake of this guide.

$ git branch apertis/v2019-foo
$ git switch apertis/v2019-foo

Option 1: Apertis tools

We are going to use Debian's git-buildpackage (gbp) to generate a patched source tree in a git branch, with the existing Apertis patches applied as commits.

First ensure that you have gbp installed:

$ sudo apt update
$ sudo apt install git-buildpackage

We can then use the following command to generate the source tree:

$ gbp pq import

Option 2: Manually constructing kernel tree

If git-buildpackage is not available, it is possible to use git-quiltimport to generate a patched kernel source tree (abet loosing out on a lot of the automation that the git-buildpackage approach provides. To get a patched kernel setup in a branch, run the following commands:

$ git quiltimport --patches debian/patches --author "Unknown <unknown@example.com>"

Note: We provide a default author to git-quiltimport via the `--author "Unknown <unknown@example.com>"` command line argument. We do this because git-quiltimport is unable to parse the author from some of the patches and providing a default on the command line avoids the patching process from being blocked with a request for the author to be specified.

Adding modifications

The extracted and patched kernel source can be used as a baseline for porting or developing additional features. Existing patches can be imported using git am, which will import each patch as a separate commit. If developing additional drivers/board support it is advisable to break down this work into well explained atomic commits.

As a developer you will need to build and test your modifications during development prior to packaging. There is no single recommended way to achieve this as the most efficient way will depend on the development setup available to you. The suggested approach is to simply build the kernel image in the correct format and copy it to a location where the firmware can be made to boot the kernel.

In the kernel source, which was cloned to a directory called linux, add the file drivers/misc/trivial.c containing the following:

 1 // SPDX-License-Identifier: GPL-2.0
 2 
 3 #include <linux/init.h>
 4 #include <linux/module.h>
 5 
 6 static int trivial_init(void)
 7 {
 8 	pr_info("Trivial module init\n");
 9 
10 	return 0;
11 }
12 module_init(trivial_init);
13 
14 static void trivial_exit(void)
15 {
16 	pr_info("Trivial module exit\n");
17 }
18 module_exit(trivial_exit);
19 
20 MODULE_AUTHOR("Martyn Welch <martyn.welch@collabora.co.uk");
21 MODULE_DESCRIPTION("Trivial Driver");
22 MODULE_LICENSE("GPL");

The following section needs to be patched into the configuration system in drivers/misc/Kconfig (This needs to be added somewhere between the menu ... and endmenu lines):

1 config TRIVIAL
2 	tristate "Trivial driver"
3 	---help---
4            Trivial example driver to show how to integrate code into the kernel.

Finally we need to add the following to the makefile drivers/misc/Makefile:

1 obj-$(CONFIG_TRIVIAL)		+= trivial.o

Add and commit these changes to git:

$ git add -f drivers/misc/trivial.c drivers/misc/Kconfig drivers/misc/Makefile
$ git commit -m "Add trivial driver"

Note: Depending on the entries in your .gitignore file (such as some of those present in the packaging repository) may require the use of the -f option when adding files such as trivial.c above.

Building

Now that we have made changes to the kernel, it would be advisable to build the kernel to ensure that the changes which have been made don't introduce build failures.

We can build the configs used by Apertis with the following command:

$ make -f debian/rules setup

This command generates the configuration files that can be used for various platforms under `debian/config`. We will us the standard 64-bit x86 configuration as an example here, tweaking it to build our additional driver and building in a separate directory to keep the source tree clean:

$ mkdir ../test-build
$ cp debian/config/amd64/config ../test-build/.config
$ echo "CONFIG_TRIVIAL=m" >> ../test-build/.config
$ yes "" | make O=../test-build oldconfig
$ make -j $(nproc) O=../test-build

Note: Depending on the changes made, modifying the configuration may not be necessary and/or using the default x86 config may not be appropriate. It may require a specific configuration to be utilised and to compile for a specific architecture. This will need to be considered on a case-by-case basis and your preferred config substituted for debian/config/amd64/config.

If cross-compilation is needed for your tests, ensure the required cross-compiler is installed and pass both the desired architecture and cross-compiler prefix to make as required. For example, for 32-bit ARM:

$ yes "" | make ARCH=arm O=../test-build oldconfig
$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- -j $(nproc) O=../test-build

Using with NFS root file systems

Now that we have a binary kernel, we can boot the board. Assuming the config is setup correctly (you'll need the relevant network and other drivers built in to the kernel, along with NFS root support) you can boot with a pre-built NFS root. The Apertis provides rootfs for various architectures, for example the x86_64 (amd64) Apertis v2019.0rc3 nfs root can be found here:

https://images.apertis.org/release/v2019/v2019.0rc3/amd64/nfs/

Packaging

Once local changes have been made, they can be added to the Apertis packaging so that the modifications can be provided in a packaged kernel.

All further discussion covers packaging that requires the Apertis/Debian tooling. If you do not have a suitable environment and/or haven't been using the Apertis tooling, stop here.

Updating patch series

So far we have a git repository holding a number of branches, including:

  1. A packaging branch with the contents of the debian/ directory.
  2. A pristine source branch, holding the broadly unaltered upstream source.
  3. A local branch, which has the patches held in the packaging branch applied to the pristine source as git commits.

We've made one or more changes that have been applied to the patched source. We now want these changes to be added back into the packaging branch so that we can push these changes into the packaged kernel. We can do this with the following command:

$ gbp pq export --no-patch-numbers

This will switch us back to the packaging branch, with the changes that we've committed previously applied as patches under debian/patches/.

Depending on how "clean" the existing patch series is, at this point gbp may make a surprising number of changes to the existing patch series. Most of these changes will be to tweak metadata and formatting to match that preferred by the current version of git. However reordering of the files patches has been seen and this makes it hard to determine whether unintended changes have occurred. In addition, comments in the series file (debian/patches/series) will have been removed. One or more additional patches will also be present in debian/patches/ (the local changes) and will have been added to the series file.

Before committing the new changes we will revert the extraneous changes. Whilst these may make the series look cleaner, applying these changes makes it hard to follow what modifications were actually made. These changes also increase the delta between the modified Apertis kernel and the upstream Apertis kernel (and for that matter the upstream from which the Apertis kernel is derived) making it harder to update at a later date. These changes can be reverted with the following command:

$ git checkout HEAD debian/patches/

This will leave us just with the additional patch that was added in debian/patches/:

$ ls debian/patches/*.patch
debian/patches/Add-trivial-driver.patch

The debian/patches/ directory contains patches that get applied to the original source. Typically Debian packages provide a single set of patches and a single series file that informs the patch management tool (typically quilt) the order in which to apply the patches. The kernel is a little different in this regard. Where as most packages are built for a handful of fairly generic architectures, the kernel is built multiple times with differing configurations and even patches applied. In order to accomodate this and to make it easier to track which patches are applicable to which platforms/features the kernel uses configuration fragments, split into generic and more specific fragments that are combined depending on the architecture and even specific board for which we are building (this is the reason we needed the make -f debian/rules setup command in the building section).

For our custom kernel we are going to build a "standard", non-realtime, kernel. Create a directory debian/patches/apertis/<project_name> (where <project_name> is the name of the project. The patch names and paths (relative to the debian/patches/ directory) should be appended to the debian/patches/series file in the order in which they need to be applied.

For the above example, make the directory debian/patches/apertis/example and move the generated patch:

$ mkdir -p debian/patches/apertis/example
$ mv debian/patches/Add-trivial-driver.patch debian/patches/apertis/example/

Add the following to the end of debian/patches/series:

# Trivial driver example
apertis/example/Add-trivial-driver.patch

These changes need to be committed into the git repository:

$ git add -f debian/patches/apertis/ debian/patches/series
$ git commit -m "Add trivial driver kernel patch"
[apertis/v2019 1c66c033e] Add trivial driver kernel patch
 2 files changed, 65 insertions(+)
 create mode 100644 debian/patches/apertis/example/Add-trivial-driver.patch

Updating kernel package configuration

Adding the driver into the kernel is not enough to make it build, the config option that we have added (CONFIG_TRIVIAL) needs to be enabled. In this instance this can be done to build the driver into the kernel, or a a kernel module that can be loaded into the kernel at runtime.

The kernel is built for a large number of platform variants, whilst some of these have significant similarities to each other, and thus can use the same binary kernel, many different builds need to be performed to support boards which have different architectures. Still, there are certain kernel configuration options, typically those that aren't linked to specific hardware, that we want to build for all platforms and those that are only required for specific hardware. In order to accommodate this without needless repetition of the kernel configuration, the configuration is built from a number of configuration fragments found in the packaging repository under debian/config/.

The main fragment for all platforms is debian/config/config, under the debian/config/ directory there are a number of directories for the individual architectures for which the kernel can be built. Under each of these directories there is another config file that is for the architecture specific options. There may also be one or more config.<sub_architecture> config files for configuration options specific to a sub-architecture build.

We will treat the trivial driver as architecture independent and build it as a module by running the following to add CONFIG_TRIVIAL=m to the end of debian/config/config:

echo "CONFIG_TRIVIAL=m" >> debian/config/config

This change will need committing:

$ git add debian/config/config 
$ git commit -m "Enable trivial driver for all builds"
[apertis/v2019 47ad258fe] Enable trivial driver for all builds
 1 file changed, 1 insertion(+)

Update changelog

As we have now modified the Debian metadata, we should update the changelog. This is important as the version specified in the changelog determines that which will be used when creating the binary packages and thus whether the newly built kernel is considered as an upgrade from that already installed should you attempt an upgrade with apt.

We can use git-buildpackage to automate much of this process. First though we want to ensure that git is configured with the correct user name and email:

$ git config --local user.name="User Name"
$ git config --local user.email="user@example.com"

Once these are set we can run the following:

$ gbp dch --git-author

The changelog (debian/changelog) will be updated with an entries based on the subject of the added commits:

$ git diff
diff --git a/debian/changelog b/debian/changelog
index ae9563fe8..0ba50cc86 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+linux (4.19.67-2co4) UNRELEASED; urgency=medium
+
+  * Add trivial driver kernel patch
+  * Enable trivial driver for all builds
+
+ -- User Name <user@example.com>  Fri, 20 Sep 2019 17:18:33 +0100
+

Unfortunately we can see that gbp dch has incrememted the co portion of the version, from 4.19.67-2co3 to 4.19.67-2co4. The co4 portion of the version number signifies the revisions of the Apertis additions (as done by Collabora). Incrementing this value would show that the Apertis package has been updated, which is not the case here (and if we leave this as it is, it will collide with the version numbering used by the next Apertis change. To avoid this, we add another element to the version number, for example here, well use foo, so the version number should be updated to 4.19.67-2co3foo1 to indicate the first foo change on top of 4.19.67-2co3.

If this is the last of the changes that we plan to make before releasing the kernel, we should change the UNRELEASED tag to apertis.

This should now be committed to git:

$ git add debian/changelog
$ git commit -s -m "Release 4.19.67-2co3foo1"

Building package

First we need to ensure that the packages required for the build installed:

$ sudo apt update
$ sudo apt install kernel-wedge quilt bc

The following script can be used to perform the build (save as build-debian-kernel one directory above the kernel source):

#!/bin/sh
set -x

PARALLEL=$(/usr/bin/nproc)

export DEB_BUILD_PROFILES="pkg.linux.notools nodoc noudeb cross nopython"
export DEB_BUILD_OPTIONS=parallel=$PARALLEL

if [ -n "$1" ] ; then
  ARCH="-a$1"
else
  ARCH=""
fi

gbp buildpackage \
  --git-debian-branch=apertis/v2019-foo \
  --git-ignore-new \
  --git-prebuild='debian/rules debian/control || true' \
  --git-builder='debuild -eDEBIAN_KERNEL_USE_CCACHE=1 -i -I' \
  --git-export-dir='~/build' \
  ${ARCH} \
  -B -d -uc -us

Make the script executable with:

$ chmod +x ../build-debian-kernel

It can be run with the required architecture as a command line option. This will limit the build to the supplied architecture, rather than building for all available architectures, so using this option is recommended.

The script uses git build package (gbp) to generate the binary kernel packages. As mentioned before gbp needs to be able to find a original source archive. We use --git-debian-branch=apertis/v2019-foo to tell gbp to use the branch apertis/v2019-foo to generate an upstream tarball.

We also specify a build area via the --git-export-dir option. As we have specified this as ~/build, which causes the build and the build processes artifacts to be extracted/stored in a directory called build in the users home directory.

Note: Such options can also be specified in a file ~/.gbp.conf. We are not using that here for simplicity, but this is commonly used by developers.

Let's build the example for amd64 (substitute amd64 for the desired architecture if your requirements differ):

$ sudo apt update
$ sudo apt install libelf-dev libssl-dev
$ mkdir ~/build
$ ../build-debian-kernel amd64

The built kernel packages will be available in ~/build.

Installing package and testing

To test we are going to boot the Apertis 2019 Minimal image on the Minnowboard. Download the image and write to an SD Card using bmaptool:

$ wget https://images.apertis.org/release/v2019/v2019.0rc4/amd64/minimal/apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.bmap
--2019-09-26 18:14:45--  https://images.apertis.org/release/v2019/v2019.0rc4/amd64/minimal/apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.bmap
Resolving images.apertis.org (images.apertis.org)... 2a00:1098:0:82:1000:25:2eeb:e3bc, 46.235.227.188
Connecting to images.apertis.org (images.apertis.org)|2a00:1098:0:82:1000:25:2eeb:e3bc|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15513 (15K) [application/octet-stream]
Saving to: ‘apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.bmap’

apertis_v2019-minim 100%[===================>]  15.15K  --.-KB/s    in 0s      

2019-09-26 18:14:45 (238 MB/s) - ‘apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.bmap’ saved [15513/15513]

$ wget https://images.apertis.org/release/v2019/v2019.0rc4/amd64/minimal/apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.gz
--2019-09-26 18:14:56--  https://images.apertis.org/release/v2019/v2019.0rc4/amd64/minimal/apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.gz
Resolving images.apertis.org (images.apertis.org)... 2a00:1098:0:82:1000:25:2eeb:e3bc, 46.235.227.188
Connecting to images.apertis.org (images.apertis.org)|2a00:1098:0:82:1000:25:2eeb:e3bc|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 398529744 (380M) [application/octet-stream]
Saving to: ‘apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.gz’

apertis_v2019-minim 100%[===================>] 380.07M  5.70MB/s    in 67s     

2019-09-26 18:16:03 (5.65 MB/s) - ‘apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.gz’ saved [398529744/398529744]

$ sudo bmaptool copy apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.gz /dev/sdi
bmaptool: info: discovered bmap file 'apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 1708985 blocks of size 4096 (6.5 GiB), mapped 229302 blocks (895.7 MiB or 13.4%)
bmaptool: info: copying image 'apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.gz' to block device '/dev/sdi' using bmap file 'apertis_v2019-minimal-amd64-uefi_v2019.0rc4.img.bmap'
bmaptool: info: 100% copied
bmaptool: info: synchronizing '/dev/sdi'
bmaptool: info: copying time: 1m 22.2s, copying speed 10.9 MiB/sec
$ 

Insert the SD card into the minnowboard and boot. We can see that it's running a 4.19 kernel:

$ uname -a
Linux apertis 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2co3 (2019-09-24) x86_64 GNU/Linux

Copy the new kernel deb file to the minnowboard (this can be done via ssh) and install it:

$ sudo mount -o remount,rw /
$ sudo dpkg -i linux-image-4.19.0-6-amd64_4.19.67-2co3foo1_amd64.deb 
(Reading database ... 19817 files and directories currently installed.)
Preparing to unpack linux-image-4.19.0-6-amd64_4.19.67-2co3foo1_amd64.deb ...
Unpacking linux-image-4.19.0-6-amd64 (4.19.67-2co3foo1) over (4.19.67-2co3bv2019.0b1) ...
Setting up linux-image-4.19.0-6-amd64 (4.19.67-2co3foo1) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-4.19.0-6-amd64
W: Possible missing firmware /lib/firmware/i915/bxt_dmc_ver1_07.bin for module i915
W: Possible missing firmware /lib/firmware/i915/skl_dmc_ver1_27.bin for module i915
...

Reboot and we boot with the new kernel:

$ uname -a
Linux apertis 4.19.0-6-amd64 #1 SMP Apertis 4.19.67-2co3foo1 (2019-09-26) x86_64 GNU/Linux

We now have the trivial driver available:

$ sudo modinfo trivial
filename:       /lib/modules/4.19.0-6-amd64/kernel/drivers/misc/trivial.ko
license:        GPL
description:    Trivial Driver
author:         Martyn Welch <martyn.welch@collabora.co.uk
depends:        
retpoline:      Y
intree:         Y
name:           trivial
vermagic:       4.19.0-6-amd64 SMP mod_unload modversions 

Which we can load and unload, receiving the log messages when we do (we need to increase the log level seen on the console to get the messages):

$ sudo dmesg -n 7 
$ sudo modprobe trivial
[  379.639027] Trivial module init
$ sudo rmmod trivial
[  386.541440] Trivial module exit

Maintenance

It is important to consider the effort required to effectively maintain a modified kernel within the Apertis ecosystem. There are a number of approaches that can be taken and depending on the modifications made to the Apertis kernel, one or more are likely to be suitable.

Upstreaming changes

It may be viable for some or all of the changes made to be upstreamed to the Apertis kernel. For any change to be considered for this approach it's likely that (at least) the following criteria would need to be met:

  • Changes must be signed off
  • Changes follow the kernel coding style
  • Changes need to be provided as a set of well formatted patches
  • The changes are either:
    • A generic bugfix
    • Controlled via kernel configuration options so as to be easily disabled. That is to say, they must not impact the usability of the Apertis kernel for other Apertis users. A good way to achieve this is to ensure the changes are self contained.

The advantage to upstreaming the changes is that such changes will no longer require to be actively maintained as this will be carried out by the Apertis team.

Local maintenance of modified Apertis kernel

It is likely that some Apertis users will make changes to the kernel that are either inappropriate to be pushed into the upstream Apertis kernel or for which they are unable or willing to make available in the upstream kernel. Such changes will need to be managed by the team responsible for them. The Apertis kernel, as with other Apertis packages, will be updated over time both to fix bugs and to stay up-to-date with changes to the Linux kernel. In order for those who are using a locally modified kernel will need to port their changes over to the updated kernel. It is expected that these modifications will be stored in a fork of the Apertis kernel packaging repository.

The following guidelines will help minimise the effort required to successfully achieve this:

  • As with upstreaming, ensure patches do one thing, are as self contained as possible and have a good description of the intent of the patch in the commit message. Well described patches are easier to follow and understand at a later date. If patches are broken down to achieve a single change it is easier to test each patch as they are ported.
  • Ensure that commits don't break the kernel build. Ensuring patches build provides another data point to ensure correct porting after each patch is applied to the updated tree. It also retains the ability to bisected the kernel which can be invaluable when tracking down bugs.

We will consider 2 cases, a minor Apertis update (for example 4.19.37-5co4 to 4.19.37-5co5) and a major update (for example 4.19.37-5co4 to 4.19.52-1co1).

Updating after minor upstream changes

It is expected that minor updates represent no change in the upstream sources. Unless the local patches happen to collide with the additional upstream patches, then usually the patches will just apply cleanly. Generally for minor changes, adding the local patches into debian/patches, updating debian/patches/series and the changelog is all that is required.

Updating after major upstream changes

Major updates represent a change in the upstream source. There is a greater chance that such changes will require changes to the patches in the series. So changes may also be applied upstream (should they have been back-ports or previously submitted upstream for inclusion). These need to be left out and the remainder ported to the new kernel.

An example of how to achieve this:

  • Generate patched kernel source of locally modified version using gbp pq import.
  • Fetch and checkout latest packaging branch from Apertis.
  • Generate patched kernel source of new version using gbp pq import.
  • Cherry pick additional patches from original locally modified tree, performing any modifications necessary to have each patch apply and build.
  • Use gbp pq export to add local changes into packaging patch series.
  • Use gbp dch to update changelog, tweak as necessary.

Version numbering

The updated nature of the package is reflected in the version number given to the package and this will have an impact on the version number that should be assigned to the newly rebased tree (new upstream, plus or or more local changes). As an example will assume 3 local revisions have been made since the last rebase on 4.19.37-5co4, following our previous example the version would expected to be 4.19.37-5co4foo3, should a new Apertis release be made (4.19.37-5co5) and assuming that the local changes are still needed, then the local version with these inclusions with constitute the first local release on top of the new upstream, thus becoming 4.19.37-5co5foo1.