OpenBMC Kernel Development Process

[Joel Stanley]

Hello OpenBMCers,

Over the past few days I've been chatting with some of the people
involved in kernel development for this project. We've looked at ways
to make this a better experience for new developers, those overseeing
development, and folks wanting to ship products on deadlines.

With Jeremy's help we've drafted a proposal that hopes to take
everyone's needs into account. Please have a read and reply  with your
feedback. The draft of the process is also on the OpenBMC wiki at
https://github.com/openbmc/linux/wiki/DevelopmentProcess

## Upstream is key

In order to leverage the benefits of using the Linux kernel, OpenBMC
maintains a policy that all kernel code will be merged upstream.

However, there are often cases where product requirements mean that
synchronising with an upstream schedule is not always possible. This
process aims to allow both goals (upstream inclusion and a useful
OpenBMC build) to occur in parallel.

## Pre-upstream inclusion plans

The project has a process for including patches before they are merged
upstream in order to allow development and testing before the upstream
work is complete.

Patches for pre-upstream inclusion in the OpenBMC kernel should at
least be of a level approximately ready to submit upstream. Although
this is somewhat subjective, this will almost always be indicated by:

 - passing the 'checkpatch' utility without warnings

 - the patch should not break the build for the kernel configurations
that are currently in use, nor should it introduce compiler warnings

 - a kernel including the patch should be boot-tested in the qemu environment

Non-upstreamed patches will be carried in the OpenBMC kernel until the
upstream stable release that occurs 30 days after original patch
inclusion. With a rough maximum time of 14 days between stable
updates, this means that the changes will be carried for 30-44 days.

However, work doesn't end here: during this time, the patch submitter
should keep interacting with the community to get the patch included
in the upstream kernel.

After the 30-day cycle, the patch will typically be on its way through
the upstream process. It is the patch owner's responsibility to ensure
that this proceeds. Once the patch has been included in an upstream
tree, we will switch over to the upstreamed patch. The OpenBMC kernel
maintainers will take responsibility for carrying that patch until a
subsequent update moves to a base version that includes the upstream
change.

However, if a patch is still not included by that time, it *may* be
resubmitted for inclusion in the next 30-day cycle. This is
conditional on at least one new revision being sent upstream for
further review, demonstrating that the owner is making progress on
upstream inclusion.  In order to indicate that the patch should be
carried over, the patch owner should re-submit it to the mailing list,
with details of upstream progression so far.

At this point, the patch owner should be in contact with the OpenBMC
kernel maintainers via either the mailing list, or #openbmc IRC
channel, for assistance with the upstreaming process.

In situations where the upstream acceptance is taking longer than
expected, we can repeat the cycle above, for a maximum of three
iterations (ie, three 30-day periods).

After those three periods, the patch will be dropped from the tree.

Note that this process is not a "free ticket" to 90 days of inclusion
in the tree. The kernel maintainer has the responsibility to ensure
that the code is of sufficient quality for the multiple consumers of
the OpenBMC kernel, and may reject submissions on serious grounds if
necessary (eg, security issues, major code review issues, copyright
concerns, adversely affecting other platforms). The intention here is
to facilitate collaboration on the kernel and scale development across
OpenBMC contributors.

## Platform scaling

A consequence of the process above is that we will be including
patches that affect platforms that the OpenBMC kernel team does not
necessarily have access to, without outside code review. This means we
have no way to test those patches.

To prevent failures from these carried patches, owners of the patches
will be added as code reviewers on the OpenBMC yocto updates.

We expect these owners to test that the updated kernel still carries
the required functionality, and provide code review feedback if not.
This will prevent the OpenBMC system from being updated to a kernel
which breaks pre-upstream support for vendor-specific functions.

If no feedback is provided with one week, is it assumed that the
update is OK to proceed. Regardless, a positive code review is still
useful, to provide quicker feedback.

# Processes

## Patch submitters (wishing to have a change included through this process)

 - ensure that patches conform to base-level quality guidelines
   - no warnings from checkpatch
   - does not introduce errors or warnings, for specific defconfigs
      - aspeed_g4_defconfig
      - aspeed_g5_defconfig
   - boot test the kernel using the qemu simulation environment
      - At a minimum, the system should boot to a userspace prompt of
the OpenBMC userspace
 - submit upstream first
 - submit proposed patches to openbmc at lists.ozlabs.org
    - This version of hte patch should contain "[PATCH linux
<branch>]" in the subject, where <branch> is the name of the current
OpenBMC development tree
 - continue with upstream discusions, by promptly following-up with
code review feedback, and post new versions of code regularly
 - if the 30-day expiry date is approaching, re-send to
openbmc at lists.ozlabs.org, requesting re-inclusion in the tree, with
details of upstream work so far  (just a short sentence in the patch
email should be sufficient for this)
 - monitor gerrit review queue for test requests from OpenBMC kernel maintainer

## OpenBMC kernel maintainer

 - Review patches submitted to the openbmc list
 - Merge patches into the OpenBMC tree
 - Rebase the OpenBMC tree at the agreed interval (30 days)
   - Announce the merge window on the mailing list
   - Update to the latest stable upstream release
   - Apply patches from the list
   - Update the status documentation for pre-upstream patches
- Test kernel on available platforms
   - Palmetto, Witherspoon
   - Qemu Palmetto, Qemu Romulus
- Add relevant reviewers for gerrit openbmc yocto "kernel bump" changes

## All developers

 - Review patches on the openbmc list
 - Exceed RDI of vegemite

## Project management

 - regularly check the status page, to ensure that:
     - patches required for your project are not at risk of exipiring
     - upstreaming efforts are not falling behind schedule