[OpenBMC]: Unit test coverage analysis on openbmc repositories

Sat Feb 29 05:20:56 AEDT 2020

On Fri, Feb 28, 2020 at 7:36 AM Lakshminarayana R Kammath
<lkammath at in.ibm.com> wrote:
>
> Hi All,
>
> Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories
>
> https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318
>
> This script does following
>
> Clone the repo
> Use the CI build environment to build code
> Publish the result in the following format
>
> Latest report:
> *************************UNIT TEST COVERAGE REPORT*************************
> https://github.com/openbmc/bmcweb.git NO
> https://github.com/openbmc/boost-dbus.git ERROR
> https://github.com/openbmc/btbridge.git NO
> https://github.com/openbmc/dbus-sensors.git NO
> https://github.com/openbmc/entity-manager.git NO
> https://github.com/openbmc/fb-ipmi-oem.git NO
> https://github.com/openbmc/google-ipmi-i2c.git ERROR
> https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST
> https://github.com/openbmc/gpioplus.git YES, COVERAGE
> https://github.com/openbmc/hiomapd.git YES, UNIT TEST
> https://github.com/openbmc/ibm-dbus-interfaces.git NO
> https://github.com/openbmc/ibm-logging.git YES, UNIT TEST
> https://github.com/openbmc/ibm-pldm-oem.git ERROR
> https://github.com/openbmc/inarp.git ERROR
> https://github.com/openbmc/intel-dbus-interfaces.git NO
> https://github.com/openbmc/intel-ipmi-oem.git NO
> https://github.com/openbmc/ipmbbridge.git NO
> https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST
> https://github.com/openbmc/ipmi-fru-parser.git NO
> https://github.com/openbmc/ipmitool.git ERROR
> https://github.com/openbmc/jsnbd.git ERROR
> https://github.com/openbmc/kcsbridge.git NO
> https://github.com/openbmc/lenovo-ipmi-oem.git NO
> https://github.com/openbmc/libmctp.git YES, UNIT TEST
> https://github.com/openbmc/librrdplus.git NO
> https://github.com/openbmc/obmc-console.git YES, UNIT TEST
> https://github.com/openbmc/obmc-ikvm.git NO
> https://github.com/openbmc/openbmc.git NO
> https://github.com/openbmc/openbmc-tools.git NO
> https://github.com/openbmc/openbmc.github.io.git NO
> https://github.com/openbmc/openpower-dbus-interfaces.git NO
> https://github.com/openbmc/openpower-debug-collector.git NO
> https://github.com/openbmc/openpower-host-ipmi-flash.git NO
> https://github.com/openbmc/openpower-host-ipmi-oem.git NO
> https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE
> https://github.com/openbmc/openpower-inventory-upload.git NO
> https://github.com/openbmc/openpower-libhei.git YES, COVERAGE
> https://github.com/openbmc/openpower-logging.git NO
> https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST
> https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST
> https://github.com/openbmc/openpower-power-control.git NO
> https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST
> https://github.com/openbmc/openpower-sbe-interface.git NO
> https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST
> https://github.com/openbmc/pam-ipmi.git NO
> https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-buttons.git NO
> https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-dbus-interfaces.git NO
> https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-debug-collector.git NO
> https://github.com/openbmc/phosphor-ecc.git NO
> https://github.com/openbmc/phosphor-event.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE
> https://github.com/openbmc/phosphor-host-ipmid.git ERROR
> https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-hostlogger.git NO
> https://github.com/openbmc/phosphor-hwmon.git NO

^---- This one should be YES, UNIT TEST.

Thanks for working on identifying missing coverage.  I wanted to point
this out so you can take another swing at this, -- Thanks!

> https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-mboxd.git ERROR
> https://github.com/openbmc/phosphor-misc.git NO
> https://github.com/openbmc/phosphor-mrw-tools.git NO
> https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-nvme.git NO
> https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-pcie-presence.git NO
> https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-post-code-manager.git NO
> https://github.com/openbmc/phosphor-power.git YES, COVERAGE
> https://github.com/openbmc/phosphor-power-control.git NO
> https://github.com/openbmc/phosphor-power-monitor.git NO
> https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE
> https://github.com/openbmc/phosphor-rest-server.git NO
> https://github.com/openbmc/phosphor-sel-logger.git NO
> https://github.com/openbmc/phosphor-settingsd.git NO
> https://github.com/openbmc/phosphor-snmp.git ERROR
> https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE
> https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO
> https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE
> https://github.com/openbmc/phosphor-webui.git NO
> https://github.com/openbmc/pldm.git YES, COVERAGE
> https://github.com/openbmc/pyphosphor.git NO
> https://github.com/openbmc/qualcomm-ipmi-oem.git NO
> https://github.com/openbmc/qualcomm-misc.git NO
> https://github.com/openbmc/quanta-ipmi-oem.git NO
> https://github.com/openbmc/rest-dbus.git NO
> https://github.com/openbmc/rrd-ipmi-blobs.git NO
> https://github.com/openbmc/s2600wf-misc.git NO
> https://github.com/openbmc/sdbusplus.git YES, UNIT TEST
> https://github.com/openbmc/sdeventplus.git YES, COVERAGE
> https://github.com/openbmc/skeleton.git ERROR
> https://github.com/openbmc/slpd-lite.git NO
> https://github.com/openbmc/smbios-mdr.git ERROR
> https://github.com/openbmc/stdplus.git YES, COVERAGE
> https://github.com/openbmc/telemetry.git NO
> https://github.com/openbmc/u-boot NO
> https://github.com/openbmc/webui-vue.git NO
> https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE
> https://github.com/openbmc/x86-power-control.git NO
> *************************UNIT TEST COVERAGE REPORT*************************
> SUMMARY:
> TOTAL REPOSITORIES : 110
> TESTED REPOSITORIES : 43
> ERROR : 11
> COVERAGE REPORT : 12
> UNIT TEST REPORT : 31
> NO REPORT : 56
> ***************************************************************************
>
>
> Following actions, we can draw from above
>
> There are 12 repo's that have GCOV/LCOV plugged in - We need to maintain which has good coverage and try to improve those which has less coverage
> There are 31 repo's that have unit test cases added but no coverage analysis because they aren't plugged in yet to GCOV/LCOV  - Is there a plan for plugging in GCOV
> There are 11 repo's which fail to compile and errors out - Fix compilation issue. Some of these maybe not used currently?
> There are 56 repo's for which there is no UT tests available  - This is a major issue we need to tackle and improve upon
>
> Feedback's , views highly appreciated
>
> Thanks & Regards,
> Lakshminarayana Kamath
> --------------------------------------------------------------------------------------------------
> Senior Engineer,
> IBM India Software Labs
> Bangalore
> github:lkammath
> ---------------------------------------------------------------------------------------------------
>
>