Upcoming enhancements to `run-unit-test-docker`

[Johnathan Mantey]

On 2/11/21 4:28 PM, Patrick Williams wrote:
> Hello,
>
> Inside openbmc-build-scripts is a tool which is used by Jenkins to run
> repository-level CI and I think some developers also use it:
> `run-unit-test-docker`.  I've rewritten a component of this, which is
> now at `scripts/build-unit-test-docker`, in Python and done some
> enhancements to it.  The [currently] last commit in this sequence is at
> [1].
>
> Important notices:
>
>      - Location change: build-unit-test-docker.sh ->
>        scripts/build-unit-test-docker
>          + I suspect few people run this directly, but instead use
>            `run-unit-test-docker` which hides the rename.
>
>      - Python3.6 minimum.
>          + Bitbake already requires this so it really shouldn't be a
>            problem for anyone.
>
>      - Python 'sh' module needed.
>          + You can install this with the 'python3-sh' package on many
>            distros or 'pip install sh'.
>
>      - Docker stages are now tagged with different names / tag revs.
>          + I wrote a `scripts/clean-unit-test-docker` which will clean
>            out old Docker tags and can be ran periodically.
>
> Major changes:
>
>      - Rewrite in Python.
>      - Significant speed improvements:
>          - Docker stages all build in parallel to formulate the final image.
>          - All Docker stages are tagged and reused from run to run.
>          - CMake packages now use the Ninja backend.  This is the same as
>            bitbake and faster than make.
>      - Included packages are defined in a dictionary rather than as raw
>        Dockerfiles.
>
> Upcoming changes:
>
>      - Shortly, I will add support to incorporate base package changes
>        into the image itself.  We have had cases where a change to
>        a common package like `sdbusplus` broke CI because a package that
>        depended on it failed to compile with the changes.  I'll add code
>        that uses some of the GERRIT_* environment variables to determine
>        if the commit-under-test is part of one of the base packages and
>        include it in the Docker image for the test run.
>
>      - Topic-based testing.  We often run into co-dependencies between
>        phosphor-dbus-interfaces and another repository.  Within the next
>        month I'll enhance the script so that if you change PDI and your
>        repository and tag the commit with the same topic, the PDI commit
>        from that topic will be pulled into the Docker image so your
>        repository commit can pass CI.
>
>      - Some refactoring.  I did a lot of incremental changes from the
>        original shell script to the Python version I currently have and
>        didn't always rewrite in the most Pythonic way.  There will be
>        some more, mostly non-functional, changes to fix this.
>
> Feel free to comment on the review series if you have any concerns with
> these changes.
>
> 1. https://gerrit.openbmc-project.xyz/c/openbmc/openbmc-build-scripts/+/40366
>
I tried using this script a day or two ago. I needed to fix some unit 
tests for a commit I pushed earlier this month. I had used the script in 
the past and couldn't remember how to use it.

A year ago I had to fix some unit tests, and I had to get Docker, and 
this test script configured. I have long since forgotten how I got the 
system setup, so I went back to try and run the unit tests I had fully 
configured and running from a year ago. In spite of having a fully 
configured environment I could not figure out how to run the tests any 
longer.

I decided to find the run-unit-test-docker.sh contents on the github 
repos, to see if I could read prepared content for how to configure, and 
run the unit test script. The readme.md is one line. The comments in the 
shell script assume you know what each settable parameter does, and how 
it is to be assigned.

I'm pretty sure I had 1:1 assistance from someone familiar with the test 
configuration to get this to work. The effort involved is more than "get 
docker, run script".

I request that someone familiar with the configuration and operation of 
this script please provide information about how to get the environment 
configured, and some example use cases for running the system. One place 
in particular that I found difficult/impossible was to run a test on 
code that required changes in other repositories. For example, if 
phosphor-network needed changes that are provided by 
phosphor-dbus-interfaces how would you configure the test run to be 
successful?

-- 
Johnathan Mantey
Senior Software Engineer
*azad te**chnology partners*
Contributing to Technology Innovation since 1992
Phone: (503) 712-6764
Email: johnathanx.mantey at intel.com <mailto:johnathanx.mantey at intel.com>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20210212/1abb424b/attachment.sig>