Enabling OpenBMC Debug

[Andrew Geissler]

> On Feb 12, 2020, at 5:45 AM, Wiktor Gołgowski <wiktor.golgowski at linux.intel.com> wrote:
> 
> 
> 
> On 2/11/20 6:58 PM, Joseph Reynolds wrote:
>> On 2/11/20 9:51 AM, Andrew Geissler wrote:
>>> I find myself running a lot of different commands to enable a variety of
>>> debug data on OpenBMC when debugging different issues (usually BIOS
>>> communication issues). I also end up giving these commands to a lot of people
>>> when recreating issues for me. Stuff like this:
>>> 
>>> # enable debug logs in journal
>>> sed -i 's/info/debug/' /lib/systemd/journald.conf.d/journald-maxlevel-policy.conf
>>> systemctl restart systemd-journald.service
>>> 
>>> # Enable BIOS communication service debug
>>> sed -i 's/mboxd/mboxd -vv/' /lib/systemd/system/mboxd.service
>>> sed -i 's/btbridged/btbridged --vv/' /lib/systemd/system/org.openbmc.HostIpmi.service
>>> sed -i 's/ipmid/ipmid -d 0xff/'  /lib/systemd/system/phosphor-ipmi-host.service
>>> sed -i 's/0/1/' /etc/default/pldmd
>>> systemctl daemon-reload
>>> systemctl restart mboxd.service
>>> systemctl restart org.openbmc.HostIpmi.service
>>> systemctl restart phosphor-ipmi-host.service
>>> systemctl restart pldmd.service
>>> 
>>> I was contemplating wrapping the above stuff in a obmcutil command so instead
>>> of telling people to do all of the above (and dealing with situations when
>>> those commands change), I could say something like run "obmcutil debugon".
>>> 
>>> Any thoughts out there? Would finer granularity of the debug be useful?
>>> Are there other forms of debug people would like enabled?
>> 
>> Thank you for the information.  I keep learning all the time.
>> 
>> These settings factor into service scenarios and also deployment readiness.  For example:
>> - I want to turn off debug.  For example, I want a debug-off command.
>> - I would want a way to validate (or at least show) these settings when testing firmware image release candidates.  For example, another obmcutil command "debug-show" which uses "grep -H" instead of "sed".
>> - I would want some documentation added the BMC administrator's guide to make them aware of OpenBMC debug support.  Draft: The obmcutil command can control debug settings for a variety of the components running on the BMC.  For more information, see https://github.com/openbmc/phosphor-state-manager/blob/master/obmcutil
>> 
>> The obmcutil tool is a shell script, so folks can take just the pieces they need.  That is, don't be more granular at this time.
> 
> Could we consider some sort of debug hooks directory (Like /etc/obmc-debug-hooks.d/) where applications could drop in their relevant scripts? 
> Or is this going too far?

You can never go too far with debug :) I’m trying to think of what this would look
like though? Can you offer an example? For most applications they just have a
global “if verbose trace, log this extra data” type thing.

> 
> - Wiktor
> 
>> 
>> In summary, it seems like a good idea.  It seems like we should ask our service architects to weight in.  I'll go ask mine....
>> 
>> - Joseph
>> 
>>> 
>>> Andrew