YASL Request

[Patrick Venture]

Hi Brad,

On Tue, Apr 10, 2018 at 7:34 PM, Brad Bishop
<bradleyb at fuzziesquirrel.com> wrote:
>
>> On Apr 9, 2018, at 6:27 PM, Patrick Venture <venture at google.com> wrote:
>>
>> Everyone,
>>
>> I'm working on unit-testing in openbmc, and have cracked most of
>> sdbusplus into mockable pieces and verified I can in fact test against
>> those mocks with a downstream daemon.  I'll be grabbing an upstream
>> daemon (or providing a piece of one) to demonstrate how to leverage
>> the mocks to test OpenBMC.  If one designs with testing in mind, the
>> designs come out very differently if not, and so getting unit-tests
>> throughout OpenBMC will be a lot of breaking things apart into
>> testable pieces.  Anyways, where I'm going with this email is that
>> everything we do within a daemon needs to be something that can be
>> mocked -- basically.
>>
>> ***
>> What do I mean specifically?  Consider, file access.  If a daemon
>> routes all file accesses throug a common object implementation
>> provided by a shared library, that shared library can easily also
>> provide a mock interface for those accesses, such that one can easily
>> verify behaviors based on file contents without implementing local
>> files or trying to inject errors.  With a mock's file system
>> interface, you can simply say that a file was unable to be read, or
>> written, or opened, etc.  Another example is mocking ctime.  If you
>> want to test whether something happens after some sleep or period, if
>> your code can receive a mock version of that library, one can
>> deliberately control the results of 'time' or 'difftime', etc.  I have
>> to build these interfaces for some of our downstream daemons and
>> likely for other parts of OpenBMC, and to avoid code duplication it'll
>> help to have them in some library.
>>
>> YASL (yet-another-shared-library) Request.
>
> Can you talk more about what goes in this?
>
> Do you envision something like:
>
> openbmc repo -> upstream project being wrapped
> libcmock -> glibc
> libstdc++mock -> libstdc++
> libsystemdmock -> libsystemd (or sdbusplusmock)
> libfoomock -> libfoo
> libbarmock -> libbarmock
>
> Or is there a super-repo/super-library that has all the wrappers?
> Do applications link a single shared library that provides wrappers
> or do they link multiple libraries that provide wrappers?

I really envision using shared libraries for mocks, and I don't see
value in mocking glibc itself, there are certainly overkill things,
for instance, there's no value IMHO to mocking memcmp from cstring.

So far, from a quick audit of our code, I see uses of the
experimental/fs library, direct file access, some fstream access, some
time, and some socket accesses, and of course, sdbusplus tie-ins.  And
soon sdevent stuff, which really -- you'd be testing the callback
method itself not that it gets called -- strictly speaking, you
could... but it's wonky.

For a program downstream, I wrote something simple

class TimeInterface {
  public:
      virtual std::time_t time(std::time_t *t) = 0;
};
class TimeImpl {
  public:
      std::time_t time(std::time_t *t) override { return ::time(t); }
};
class TimeMock {
};

The idea being to control the value from time.  However, if you didn't
care about that, you could focus on difftime(), which was actually
doing the computation, to make it return whatever you needed to
trigger the cases you needed.


>
> I’m just trying to figure out what to call the repo, and get a feel
> for what the structure would look like over time.  Is the repo for
> non-openbmc wrappers (like standard library stuff), and wrappers for
> openbmc hosted libraries like sdbusplus would go in the sdbusplus
> repo?

If you wanted a repo for each library, that'd be your call.  I think
it's fine to group them.

>
>>
>> Previous conversations along these lines lead to the idea that we need
>> multiple new libraries for various things.  So, this is yet another
>> use case.  The library itself should be written in such a way that it
>> can be tested via unit-tests, but depending on how thin of a shim it
>> is, that isn't always practical.  See:
>>
>> class FileInterface {
>>  public:
>>     virtual int open(const char *filename, int flags) = 0;
>> };
>>
>> class FileImplementation : public FileInterface {
>>  public:
>>    int open(const char *filename, int flags) override {
>>        return ::open(filename, flags);
>>    }
>> };
>>
>> class FileMock : public FileInterface {
>>   public:
>>     MOCK_METHOD2(open, int(const char *, int));
>> };
>
> Doesn’t this style of programming slow things down (when you look at
> it at scale)?  If you have a software stack, and you turn N shared
> library calls from simple branches into this, multiplied by N processes…
> aren’t we going to waste a lot of cycles?  Is this how people want
> to use their BMC cycles or do they want it running their business
> logic?  I know IBM is already struggling with bigger-than-ideal load
> averages / runqueue depths.
>
> Is it your vision the the project _requires_ that all applications
> be written in this manner and use GMock as the unit test framework as
> a condition for inclusion in OpenBMC, or are you just looking for the
> capability for applications that elect to use it?  There are other ways
> to write unit tests.  I think only allowing one framework (considering
> the impact it has on how we write code) would drive people away from
> the project.  Assuming sdbusplus is made to be mockable, we can probably
> operate in the latter mode since we just don’t have very many shared
> libraries in the first place (a metric I’d like to maintain as we grow).
>
>>
>> .... then one just uses the FileInterface for their normally direct
>> posix-style file access.  This can easily wrap iostream, or fstream,
>> or anything.  And then if we provide some libraries for use by
>> daemons, they can transition over to them over time, and then they get
>> mocks for free :D  For a daemon downstream, I've written a ctime
>> wrapper, I'll submit it for consideration later tonight along with a
>> few other things, and then I'll reply to this email with links.
>>
>> ***Not meant as a unit-test primer, just trying to provide some real examples.
>>
>> Patrick
>
>