[OpenBMC]: Unit test coverage analysis on openbmc repositories

Thu Mar 5 12:25:56 AEDT 2020

On 2/28/20 10:20 AM, Patrick Venture wrote:
> On Fri, Feb 28, 2020 at 7:36 AM Lakshminarayana R Kammath
> <lkammath at in.ibm.com> wrote:
>>
>> Hi All,
>>
>> Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories
>>
>> https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318
>>
>> This script does following
>>
>> Clone the repo
>> Use the CI build environment to build code
>> Publish the result in the following format
>>
>> Latest report:
>> *************************UNIT TEST COVERAGE REPORT*************************
>> https://github.com/openbmc/bmcweb.git NO
>> https://github.com/openbmc/boost-dbus.git ERROR
>> https://github.com/openbmc/btbridge.git NO
>> https://github.com/openbmc/dbus-sensors.git NO
>> https://github.com/openbmc/entity-manager.git NO
This has some tests:

https://github.com/openbmc/entity-manager/blob/master/test/test_entity-manager.cpp

>> https://github.com/openbmc/fb-ipmi-oem.git NO
>> https://github.com/openbmc/google-ipmi-i2c.git ERROR
>> https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST
>> https://github.com/openbmc/gpioplus.git YES, COVERAGE
>> https://github.com/openbmc/hiomapd.git YES, UNIT TEST
>> https://github.com/openbmc/ibm-dbus-interfaces.git NO
>> https://github.com/openbmc/ibm-logging.git YES, UNIT TEST
>> https://github.com/openbmc/ibm-pldm-oem.git ERROR
>> https://github.com/openbmc/inarp.git ERROR
>> https://github.com/openbmc/intel-dbus-interfaces.git NO
>> https://github.com/openbmc/intel-ipmi-oem.git NO
So does this:
https://github.com/openbmc/intel-ipmi-oem/blob/master/tests/test_sensorcommands.cpp

>> https://github.com/openbmc/ipmbbridge.git NO
>> https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST
>> https://github.com/openbmc/ipmi-fru-parser.git NO
>> https://github.com/openbmc/ipmitool.git ERROR
>> https://github.com/openbmc/jsnbd.git ERROR
>> https://github.com/openbmc/kcsbridge.git NO
>> https://github.com/openbmc/lenovo-ipmi-oem.git NO
>> https://github.com/openbmc/libmctp.git YES, UNIT TEST
>> https://github.com/openbmc/librrdplus.git NO
>> https://github.com/openbmc/obmc-console.git YES, UNIT TEST
>> https://github.com/openbmc/obmc-ikvm.git NO
>> https://github.com/openbmc/openbmc.git NO
>> https://github.com/openbmc/openbmc-tools.git NO
>> https://github.com/openbmc/openbmc.github.io.git NO
>> https://github.com/openbmc/openpower-dbus-interfaces.git NO
>> https://github.com/openbmc/openpower-debug-collector.git NO
>> https://github.com/openbmc/openpower-host-ipmi-flash.git NO
>> https://github.com/openbmc/openpower-host-ipmi-oem.git NO
>> https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE
>> https://github.com/openbmc/openpower-inventory-upload.git NO
>> https://github.com/openbmc/openpower-libhei.git YES, COVERAGE
>> https://github.com/openbmc/openpower-logging.git NO
>> https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST
>> https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST
>> https://github.com/openbmc/openpower-power-control.git NO
>> https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST
>> https://github.com/openbmc/openpower-sbe-interface.git NO
>> https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST
>> https://github.com/openbmc/pam-ipmi.git NO
>> https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-buttons.git NO
>> https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-dbus-interfaces.git NO
>> https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-debug-collector.git NO
>> https://github.com/openbmc/phosphor-ecc.git NO
>> https://github.com/openbmc/phosphor-event.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE
>> https://github.com/openbmc/phosphor-host-ipmid.git ERROR
>> https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-hostlogger.git NO
>> https://github.com/openbmc/phosphor-hwmon.git NO
> 
> ^---- This one should be YES, UNIT TEST.
> 
> Thanks for working on identifying missing coverage.  I wanted to point
> this out so you can take another swing at this, -- Thanks!
> 
>> https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-mboxd.git ERROR
>> https://github.com/openbmc/phosphor-misc.git NO
>> https://github.com/openbmc/phosphor-mrw-tools.git NO
>> https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-nvme.git NO
>> https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-pcie-presence.git NO
>> https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-post-code-manager.git NO
>> https://github.com/openbmc/phosphor-power.git YES, COVERAGE
>> https://github.com/openbmc/phosphor-power-control.git NO
>> https://github.com/openbmc/phosphor-power-monitor.git NO
>> https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE
>> https://github.com/openbmc/phosphor-rest-server.git NO
>> https://github.com/openbmc/phosphor-sel-logger.git NO
>> https://github.com/openbmc/phosphor-settingsd.git NO
>> https://github.com/openbmc/phosphor-snmp.git ERROR
>> https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE
>> https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO
>> https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST
>> https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE
>> https://github.com/openbmc/phosphor-webui.git NO
>> https://github.com/openbmc/pldm.git YES, COVERAGE
>> https://github.com/openbmc/pyphosphor.git NO
>> https://github.com/openbmc/qualcomm-ipmi-oem.git NO
>> https://github.com/openbmc/qualcomm-misc.git NO
>> https://github.com/openbmc/quanta-ipmi-oem.git NO
>> https://github.com/openbmc/rest-dbus.git NO
>> https://github.com/openbmc/rrd-ipmi-blobs.git NO
>> https://github.com/openbmc/s2600wf-misc.git NO
>> https://github.com/openbmc/sdbusplus.git YES, UNIT TEST
>> https://github.com/openbmc/sdeventplus.git YES, COVERAGE
>> https://github.com/openbmc/skeleton.git ERROR
>> https://github.com/openbmc/slpd-lite.git NO
>> https://github.com/openbmc/smbios-mdr.git ERROR
>> https://github.com/openbmc/stdplus.git YES, COVERAGE
>> https://github.com/openbmc/telemetry.git NO
>> https://github.com/openbmc/u-boot NO
>> https://github.com/openbmc/webui-vue.git NO
>> https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE
>> https://github.com/openbmc/x86-power-control.git NO
>> *************************UNIT TEST COVERAGE REPORT*************************
>> SUMMARY:
>> TOTAL REPOSITORIES : 110
>> TESTED REPOSITORIES : 43
>> ERROR : 11
>> COVERAGE REPORT : 12
>> UNIT TEST REPORT : 31
>> NO REPORT : 56
>> ***************************************************************************
>>
>>
>> Following actions, we can draw from above
>>
>> There are 12 repo's that have GCOV/LCOV plugged in - We need to maintain which has good coverage and try to improve those which has less coverage
>> There are 31 repo's that have unit test cases added but no coverage analysis because they aren't plugged in yet to GCOV/LCOV  - Is there a plan for plugging in GCOV
>> There are 11 repo's which fail to compile and errors out - Fix compilation issue. Some of these maybe not used currently?
>> There are 56 repo's for which there is no UT tests available  - This is a major issue we need to tackle and improve upon
>>
>> Feedback's , views highly appreciated
>>
>> Thanks & Regards,
>> Lakshminarayana Kamath
>> --------------------------------------------------------------------------------------------------
>> Senior Engineer,
>> IBM India Software Labs
>> Bangalore
>> github:lkammath
>> ---------------------------------------------------------------------------------------------------
>>
>>