Upcoming enhancements to `run-unit-test-docker`

[Patrick Williams]

Hello,

Inside openbmc-build-scripts is a tool which is used by Jenkins to run
repository-level CI and I think some developers also use it:
`run-unit-test-docker`.  I've rewritten a component of this, which is
now at `scripts/build-unit-test-docker`, in Python and done some
enhancements to it.  The [currently] last commit in this sequence is at
[1].

Important notices:

    - Location change: build-unit-test-docker.sh ->
      scripts/build-unit-test-docker
        + I suspect few people run this directly, but instead use
          `run-unit-test-docker` which hides the rename.

    - Python3.6 minimum.
        + Bitbake already requires this so it really shouldn't be a
          problem for anyone.

    - Python 'sh' module needed.
        + You can install this with the 'python3-sh' package on many
          distros or 'pip install sh'.

    - Docker stages are now tagged with different names / tag revs.
        + I wrote a `scripts/clean-unit-test-docker` which will clean
          out old Docker tags and can be ran periodically.

Major changes:

    - Rewrite in Python.
    - Significant speed improvements:
        - Docker stages all build in parallel to formulate the final image.
        - All Docker stages are tagged and reused from run to run.
        - CMake packages now use the Ninja backend.  This is the same as
          bitbake and faster than make.
    - Included packages are defined in a dictionary rather than as raw
      Dockerfiles.

Upcoming changes:

    - Shortly, I will add support to incorporate base package changes
      into the image itself.  We have had cases where a change to
      a common package like `sdbusplus` broke CI because a package that
      depended on it failed to compile with the changes.  I'll add code
      that uses some of the GERRIT_* environment variables to determine
      if the commit-under-test is part of one of the base packages and
      include it in the Docker image for the test run.

    - Topic-based testing.  We often run into co-dependencies between
      phosphor-dbus-interfaces and another repository.  Within the next
      month I'll enhance the script so that if you change PDI and your
      repository and tag the commit with the same topic, the PDI commit
      from that topic will be pulled into the Docker image so your
      repository commit can pass CI.

    - Some refactoring.  I did a lot of incremental changes from the
      original shell script to the Python version I currently have and
      didn't always rewrite in the most Pythonic way.  There will be
      some more, mostly non-functional, changes to fix this.

Feel free to comment on the review series if you have any concerns with
these changes.

1. https://gerrit.openbmc-project.xyz/c/openbmc/openbmc-build-scripts/+/40366

-- 
Patrick Williams
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20210211/97463b33/attachment.sig>