[Skiboot] GCOV for skiboot

[Stewart Smith]

Hi all,

As you know, we build and run gcov enabled unit tests and have done for
quite a while - and we've been improving unit test code coverage quite
decently over the past several months.

Results here: http://open-power.github.io/skiboot/coverage-report/
(make your own with "make coverage-report")


I've been working on getting gcov support going in skiboot so that we
could boot skiboot in either a simulator (mambo, or other) or on actual
hardware and then produce the same kind of output for tests on actual
hardware. This could be quite useful for determining if our testing is
adequate, especially around error cases.

After a bunch of poking around, I have some preliminary results for
booting on one of our IBM POWER8 systems:
http://open-power.github.io/skiboot/boot-coverage-tul169/

This is "boot to petitboot" run, which shows we hit 8585/23482 lines of
code, 812/1807 functions and 3153/11809 branches.

For those who are interested, below is an explanation of how it works:

(code is in https://github.com/stewart-ibm/skiboot/tree/gcov - not yet
ready for merge or primetime)

When you compile with gcov, GCC changes how it emits code. Basically,
for each branch it generates code to increment a counter on how many
times each code path was taken.

As you can imagine, this bloats the binary a fair bit. In the case of
skiboot, it approximately doubles the size: from ~700k to ~1.5MB. This
created some fun in skiboot due to a number of places where we assume
skiboot is always <1MB (fun time spent in mambo debugging why relocate()
wasn't working)

There's also some symbols the generated code assumes exist, so I had to
add those (core/gcov-profiling.c - profiling is only built if
SKIBOOT_GCOV=1 is set). I also had to make skiboot run GCC generated
constructors, as these are used to link together the linked list of gcov
info data structures.

All of the gcov data structures and counters are in the BSS section.

The downside is that the gcov data structures are rather GCC version
dependent, 3.4 does it differently than 4.7+, and even though we only
support 4.8+ for skiboot, there was a change in 4.9.

Once I had skiboot with gcov booting, the next step is to extract the
gcov data. Ordinarily, this is done at program exit by gcov code writing
out .gcda files. For a gcov enabled Linux kernel, there's a debugfs that
you can read them out from.

The .gcda files are what /usr/bin/gcov and lcov read to produce their
output. Since we don't have anything remotely like a filesystem in
skiboot (and I didn't want to implement one), I took the approach of
"dump out skiboot memory and then parse it with external tool".

This means you can trivially do this from:
mambo:
  mysim memory fwrite 0x30000000 0x240000 skiboot.dump
FSP:
  getmemproc 30000000 3407872 -fb skiboot-vpl3-pkvm-boot.dump
linux (e.g. petitboot environment):
  dd if=/proc/kcore skip=1572864 count=6656  of=skiboot-vpl3-pkvm-oot.dump

and then use my extract-gcov program to extract out the gcda files.
e.g.
./extract-gcov ~/skiboot-vpl3.dump 0x3011a300

which then writes all the gcda files to wherever the embedded filename
paths in the skiboot binary are.

You can then run lcov:
lcov -b . -q -c -d . -o skiboot-tul169-boot.info \
--gcov-tool /opt/cross/gcc-4.8.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcov

and then genhtml to get a HTML report like the one I linked to above.

I think this approach is fairly safe... it means that as long as you can
dump out host memory you can get gcov data, which can actually be useful
in debugging some strange problems. The endian thunking in
extract-gcov.c is kind of interesting when running on x86 and with the
weird formatting of 64 bit ints in gcda.

Thoughts/questions/constructive abuse are more than welcome!