[OpenBMC]: Unit test coverage analysis on openbmc repositories

Sat Feb 29 03:08:23 AEDT 2020

Hi All,

Somehow my earlier mail did not get displayed properly

Recently we worked on pulling the UT coverage analysis from all the
available OpenBMC repositories

https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318

This script does following

   1. Clone the repo
   2. Use the CI build environment to build code
   3. Publish the result in the following format

Latest report:
*************************UNIT TEST COVERAGE REPORT*************************
https://github.com/openbmc/bmcweb.git NO
https://github.com/openbmc/boost-dbus.git ERROR
https://github.com/openbmc/btbridge.git NO
https://github.com/openbmc/dbus-sensors.git NO
https://github.com/openbmc/entity-manager.git NO
https://github.com/openbmc/fb-ipmi-oem.git NO
https://github.com/openbmc/google-ipmi-i2c.git ERROR
https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST
https://github.com/openbmc/gpioplus.git YES, COVERAGE
https://github.com/openbmc/hiomapd.git YES, UNIT TEST
https://github.com/openbmc/ibm-dbus-interfaces.git NO
https://github.com/openbmc/ibm-logging.git YES, UNIT TEST
https://github.com/openbmc/ibm-pldm-oem.git ERROR
https://github.com/openbmc/inarp.git ERROR
https://github.com/openbmc/intel-dbus-interfaces.git NO
https://github.com/openbmc/intel-ipmi-oem.git NO
https://github.com/openbmc/ipmbbridge.git NO
https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST
https://github.com/openbmc/ipmi-fru-parser.git NO
https://github.com/openbmc/ipmitool.git ERROR
https://github.com/openbmc/jsnbd.git ERROR
https://github.com/openbmc/kcsbridge.git NO
https://github.com/openbmc/lenovo-ipmi-oem.git NO
https://github.com/openbmc/libmctp.git YES, UNIT TEST
https://github.com/openbmc/librrdplus.git NO
https://github.com/openbmc/obmc-console.git YES, UNIT TEST
https://github.com/openbmc/obmc-ikvm.git NO
https://github.com/openbmc/openbmc.git NO
https://github.com/openbmc/openbmc-tools.git NO
https://github.com/openbmc/openbmc.github.io.git NO
https://github.com/openbmc/openpower-dbus-interfaces.git NO
https://github.com/openbmc/openpower-debug-collector.git NO
https://github.com/openbmc/openpower-host-ipmi-flash.git NO
https://github.com/openbmc/openpower-host-ipmi-oem.git NO
https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE
https://github.com/openbmc/openpower-inventory-upload.git NO
https://github.com/openbmc/openpower-libhei.git YES, COVERAGE
https://github.com/openbmc/openpower-logging.git NO
https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST
https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST
https://github.com/openbmc/openpower-power-control.git NO
https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST
https://github.com/openbmc/openpower-sbe-interface.git NO
https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST
https://github.com/openbmc/pam-ipmi.git NO
https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST
https://github.com/openbmc/phosphor-buttons.git NO
https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST
https://github.com/openbmc/phosphor-dbus-interfaces.git NO
https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST
https://github.com/openbmc/phosphor-debug-collector.git NO
https://github.com/openbmc/phosphor-ecc.git NO
https://github.com/openbmc/phosphor-event.git YES, UNIT TEST
https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST
https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE
https://github.com/openbmc/phosphor-host-ipmid.git ERROR
https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST
https://github.com/openbmc/phosphor-hostlogger.git NO
https://github.com/openbmc/phosphor-hwmon.git NO
https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST
https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST
https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT
TEST
https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST
https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST
https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST
https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST
https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST
https://github.com/openbmc/phosphor-mboxd.git ERROR
https://github.com/openbmc/phosphor-misc.git NO
https://github.com/openbmc/phosphor-mrw-tools.git NO
https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST
https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST
https://github.com/openbmc/phosphor-nvme.git NO
https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST
https://github.com/openbmc/phosphor-pcie-presence.git NO
https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST
https://github.com/openbmc/phosphor-post-code-manager.git NO
https://github.com/openbmc/phosphor-power.git YES, COVERAGE
https://github.com/openbmc/phosphor-power-control.git NO
https://github.com/openbmc/phosphor-power-monitor.git NO
https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE
https://github.com/openbmc/phosphor-rest-server.git NO
https://github.com/openbmc/phosphor-sel-logger.git NO
https://github.com/openbmc/phosphor-settingsd.git NO
https://github.com/openbmc/phosphor-snmp.git ERROR
https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE
https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST
https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO
https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST
https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE
https://github.com/openbmc/phosphor-webui.git NO
https://github.com/openbmc/pldm.git YES, COVERAGE
https://github.com/openbmc/pyphosphor.git NO
https://github.com/openbmc/qualcomm-ipmi-oem.git NO
https://github.com/openbmc/qualcomm-misc.git NO
https://github.com/openbmc/quanta-ipmi-oem.git NO
https://github.com/openbmc/rest-dbus.git NO
https://github.com/openbmc/rrd-ipmi-blobs.git NO
https://github.com/openbmc/s2600wf-misc.git NO
https://github.com/openbmc/sdbusplus.git YES, UNIT TEST
https://github.com/openbmc/sdeventplus.git YES, COVERAGE
https://github.com/openbmc/skeleton.git ERROR
https://github.com/openbmc/slpd-lite.git NO
https://github.com/openbmc/smbios-mdr.git ERROR
https://github.com/openbmc/stdplus.git YES, COVERAGE
https://github.com/openbmc/telemetry.git NO
https://github.com/openbmc/u-boot NO
https://github.com/openbmc/webui-vue.git NO
https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE
https://github.com/openbmc/x86-power-control.git NO
*************************UNIT TEST COVERAGE REPORT*************************
SUMMARY:
TOTAL REPOSITORIES : 110
TESTED REPOSITORIES : 43
ERROR : 11
COVERAGE REPORT : 12
UNIT TEST REPORT : 31
NO REPORT : 56
***************************************************************************

Following actions, we can draw from above

   1. *There are 12 repo's that have GCOV/LCOV plugged in* - We need to
   maintain which has good coverage and try to improve those which has less
   coverage
   2. *There are 31 repo's that have unit test cases added but no coverage
   analysis because they aren't plugged in yet to GCOV/LCOV*  - Is there a
   plan for plugging in GCOV
   3. *There are 11 repo's which fail to compile and errors out* - Fix
   compilation issue. Some of these maybe not used currently?
   4. *There are 56 repo's for which there is no UT tests availabl*e  -
   This is a major issue we need to tackle and improve upon

Feedback's, views highly appreciated

Thanks & Regards,
Lakshminarayana Kamath
--------------------------------------------------------------------------------------------------
Senior Engineer,
IBM India Software Labs
Bangalore
github:lkammath
---------------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20200228/2bc4bb67/attachment-0001.htm>