Security Working Group - Wednesday July 22 - results

Tue Aug 4 19:36:43 AEST 2020

On Thu, Jul 23, 2020 at 8:46 PM Ed Tanous <ed at tanous.net> wrote:

> First off, if you are finding certificates are expiring in a
> production environment, you need to rethink your certificate
> replacement strategy.  The BMC cannot be the primary entity in charge
> of certificate replacement and revocation, and the fact that this is
> happening points to a greater organizational problem than the bmc can
> cover.
>
> One thing to note;  At one point, I had talked through how to
> prototype ACME protocol replacement of certificates automatically, so,
> given an ACME server on the network, the BMC could essentially
> automatically provision itself and keep its certs up to date.  If
> someone wanted to run with that, it might reduce some of the pain here
> (and be extremely cool).
>
> On Thu, Jul 23, 2020 at 7:24 AM Joseph Reynolds <jrey at linux.ibm.com>
> wrote:
> >
> >
> > 4 Question: If BMCWeb finds an unusable HTTPS site identity certificate,
> > it DELETES it and self-generates one.  This has caused problems for
> > certificates that are not valid until a future date. In general, what
> > certificate management support should we have for BMCWeb?  What is
> needed?
>
> This is not how bmcweb was intended to work, and I had explicitly
> tested that it doesn't have this behavior (prior to it being moved
> over to certificate manager).  The only time it previously would
> replace a certificate was if the cert file didn't exist, or was
> completely unreadable.  I suspect the right thing here is to bisect
> the commit that broke this behavior (I'll bet it's easy to find) and
> get a bug filed with the author so they can either fix it or revert
> the patch.
>
> This one smells awfully suspicious:
> 082f28fd2588fd9fcd9452ad38234ce875319163.
>

  Ed,
        https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/22938
 introduced to fix issue https://github.com/openbmc/bmcweb/issues/91.
     We should also add check to ignore “X509_V_ERR_CERT_NOT_YET_VALID” to
allow user to upload “NOT_YET_VALID” certificate similar to certificate
manager verification function.

>
> > ANSWER:
> > There were two discussion threads: The BMC’s notion of time of day
> > (TOD), and how BMCWeb should handle certificates.
> >
> > Does the BMC have a battery backed TOD clock?  Depends on BMC model.
> > Can it validate if it has access to its NTP server (when configured)?
> > Does the BMC know if its time was set correctly?
> >
> > How does the BMC know if the BMC has the correct time?  Have a BMC flag
> > that says, “Look like the BMC TOD clock is not working.”  Does the BMC
> > know if we got a good time from an  NTP server?  Can we read the GPS
> > signal?  What is the industry solution?
> > Should the BMC store its idea of what date it is?  So it can report if
> > the time changes significantly.  Or will this lead to a bigger problem?
> > Is it better/simpler to check for TOD = beginning-of-era-1/1/1970?  →
> > start an email thread
>
> The above is all asking the wrong question: "Can we determine if the
> certificate is valid?"  This is irrelevant, the question is: "Should
> we ever be replacing a user provided certificate with one generated on
> the BMC."  The answer previously has been no.  In almost all cases the
> user provided certificate, even an expired one, will still be better
> than one the BMC self signs.  Between having an invalid certificate
> chain, and an invalid date, I'll take the invalid date every time.
>
> It should be noted, most browsers (in my testing) seem to ignore the
> HTTP date header entirely, so the BMC doesn't even need the correct
> time to set up a proper encryption channel.
>
> >
> > BMCWeb configuration?  Configure option: delete cert and generate
> > self-signed -vs- use defective certificate.  What is the purpose of
> > deleting the unusable cert?
>
> This question needs answered.  I suspect this was unintentional, and
> someone simply used some bad openssl code to attempt to verify the
> cert against the (possibly non existent) chain without realizing this
> requirement.
>
> >  Should “out of date” not be part of the
> > “unusable” definition? ⇒ Ideas: 1. If bmcweb finds a usable cert but is
> > out of date, that cert can still be used.  2. Leave the defective
> > certificate (do not delete it) and log an error.
>
> A lot of BMCs don't have a dedicated RTC, and rely on other systems
> (like the PCH or NTP) to get the correct time.  bmcweb needs to come
> up long before the PCH or NTP (both of which are also optional) so as
> a general rule, using these for valid time is a non-starter.  I could
> see logging an error _if_ you know time is valid, but I'm not sure how
> a bmc could know that.
>
> >
> > The group consensus was that BMCWeb should treat its HTTPS site identity
> > certificate like this:
> > 1 certificate is perfectly good - Use the certificate
> > 2 certificate is good but expired or not yet valid - Use the certificate
> > and log a warning
>
> Add "and we know time has been set appropriately".  Also, be careful
> with not yet valid.  I don't know how openssl chains handle clock skew
> between certificate generator and client.  If the BMC has a time
> that's 1 minute fast, is the certificate "not yet valid"?
>
> > 3 certificate is missing or bad format or algorithm too old - Use
> > another certificate or self-generate a certificate (and log that action)
>
> We shouldn't be replacing certificates unless it's completely missing
> (ie, we're on a first boot or a factory reset) and wouldn't be
> switching certificates on the fly.  If the algorithm is too old, the
> user is free to replace it with their own, which is the right
> procedure anyway.  We also have no idea if the user has added this
> public certificate to their local root store, so replacing it out of
> the blue might look like an attack.  Also, we have no idea if the
> users' client systems support the new crypto format.  We could be
> unintentionally DOSing them if, say, we turn on a new keytype or size
> and their clients don't support that key.
>
> > There are no cases where BMCWeb should delete any certificate.
>
> I think I know what you mean, but to be clear, we should delete the
> old certificates when they're replaced.  We should delete root trust
> store certificates when requested, ect.  I don't think we can say "no
> cases" here.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20200804/442a9219/attachment.htm>