jenkins.openbmc.org server infrastructure

[Andrew Geissler]

On Mon, Oct 29, 2018 at 9:43 AM Joseph Reynolds
<joseph-reynolds at charter.net> wrote:
>
>
> Message: 1
> Date: Thu, 25 Oct 2018 14:49:20 -0500
> From: Andrew Geissler <geissonator at gmail.com>
> To: OpenBMC Maillist <openbmc at lists.ozlabs.org>
> Subject: jenkins.openbmc.org ?
> Message-ID:
> <CALLMt=pqTesc1qD3wCsv6GTZVRgh2CM1_zF+PM3VVGiDyKf6Qg at mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> I'd had on my list for a while to move us over to a jenkins instance
> running on the same server as our gerrit server. The reasoning was:
>
> 1. OpenBMC has grown to the point where it deserves it's own jenkins
> instance, and tying it to openpower.xyz doesn't really fit anymore.
> 2. The gerrit server (and the new jenkins server) are running within
> Softlayer which has 24x7 support (openpower.xyz is running on donated
> hardware from Rackspace, which is awesome, but support is mainly 2nd
> hand via emails to contacts we have at Rackspace)
> 3. There have been a few instances where the openpower builds have
> affected openbmc (and vice versa).
>
> I went ahead and created a new jenkins instance at the above domain,
> but haven't done anything with it. I've been kind of wishy-washy on
> whether the extra work is really worth it. It's been quite nice having
> Joel maintain our openpower.xyz server with all the security updates
> and such :)
>
> Thoughts from the community?
>
>

Hey Joseph, I think you need to make sure your email client is in
plain text. The formatting
looks a bit off on your responses.

> Andrew, your email reads like a plea for help with OpenBMC project infrastructure (systems that host OpenBMC servers for Gerrit, Jenkins, and related CI).  If so, it may help to understand the current configuration.  Here is my understanding:

Not as much a plea for help as much as "is it worth us doing this
extra work". I've maintained
a lot of different jenkins servers in my time so it's not a ton of
work, but if no one cares
then I don't see a reason to do it.

>
>  - GitHub (source code mirror), hosted by GitHub at https://github.com/openbmc (various repos).
>  - Gerrit (code review tool), hosted by https://gerrit.openbmc-project.xyz (and redirected from gerrit.openbmc.org).
>  - Jenkins (CI server), hosted by https://openpower.xyz, proposal to move to a new server at https://jenkins.openbmc.org.
>  - Various CI build and test servers (run jobs on different hardware architectures).
>  - See history in https://lists.ozlabs.org/pipermail/openbmc/2016-July/004280.html

Yep, looks about right. We also have an internal jenkins which
coordinates the HW CI once a
job passes the basic build and QEMU CI.

>
> Then reading questions into the email:
>
> Are community members comfortable with these Gerrit and Jenkins servers in terms of ownership and administration privileges?
> Should we have a larger variety of CI servers?

More CI servers for building are always appreciated :)

> Do we need help administering these servers (workload: reboot servers, update Jenkins version, restart CI jobs, etc.)?
> What are the project ideals?

Yeah, maintaining the jenkins nodes is something I haven't been doing real well.
I think I'm going to have to start doing a maintenance window every
quarter or so and ensuring all of the
most recent Ubuntu updates are picked up. Major upgrades to new Ubuntu
LTS releases would
need a larger window (but hopefully this is a once ever other year
type thing). I'm curious how other projects
handle this. Is there a dedicated person/team to Infrastructure?
Currently this isn't a ton of work for me
but it does seem like something that's going to keep growing.

>
>
> ps: gerrit.openbmc.org will also redirect to our gerrit in case anyone
> was interested.
>
> Andrew
>
>