<font size=2 face="sans-serif">Proposal : Method to trigger
data collection of state logs from applications, BMC and other information
on failure.</font><br><font size=2 face="sans-serif"> </font><br><font size=2 face="sans-serif">Background : Open BMC test is
written in ROBOT framework which generates XML/HTML logs but not sufficient
for debugging failures in general.</font><br><br><font size=2 face="sans-serif">
- Robot Test framework HTML/XML logs gives good info but generic</font><br><font size=2 face="sans-serif">
- FFDC supplements with more Open BMC state logs for analysis</font><br><font size=2 face="sans-serif">
- Asserts Test failure from Dev and Test perspective by</font><br><font size=2 face="sans-serif">
-- capturing generic logs </font><br><font size=2 face="sans-serif">
-- providing custom user define/preference data</font><br><font size=2 face="sans-serif">
-- point of failure data</font><br><br><font size=2 face="sans-serif">Layout : Capture data
with a default name openbmc_ffdc_report.txt under current execution directory
as logs/testSuite/testcaseName/ on failure.</font><br><br><font size=2 face="sans-serif">
$ tree
logs</font><br><font size=2 face="sans-serif">
logs</font><br><font size=2 face="sans-serif">
└── TestSample</font><br><font size=2 face="sans-serif">
└── VerifyConnection</font><br><font size=2 face="sans-serif">
└── openbmc_ffdc_report.txt</font><br><br><font size=2 face="sans-serif">
2 directories,
1 file</font><br><br><br><font size=2 face="sans-serif">Method : Robot framework
provides granular Test cases Setup and Teardown capability</font><br><br><font size=2 face="sans-serif">
Framework
methods :</font><br><font size=2 face="sans-serif">
</font><br><font size=2 face="sans-serif">
Calls
the Test case failure FFDC collection keyword as part of test case exit</font><br><font size=2 face="sans-serif">
[Teardown]
Log FFDC If Test Case Failed</font><br><font size=2 face="sans-serif">
</font><br><font size=2 face="sans-serif">
Ability to use Resource/Library modules to support functionality</font><br><font size=2 face="sans-serif">
Provides in-built Automatic variables updated by framework for checking
if test case PASSED or FAILED and getting test suite and case name and
paths</font><br><br><font size=2 face="sans-serif">
Auto Variable : ${TEST_STATUS} ${TEST_NAME} ${SUITE_NAME}
${EXECDIR} </font><br><font size=2 face="sans-serif">
</font><br><font size=2 face="sans-serif">
# Return if the test case is a PASS</font><br><font size=2 face="sans-serif">
Return From Keyword If '${TEST_STATUS}' != 'FAIL'
</font><br><font size=2 face="sans-serif">
... ELSE do the log collection</font><br><br><font size=2 face="sans-serif">
Connection
methods:</font><br><font size=2 face="sans-serif">
</font><br><font size=2 face="sans-serif">
Current
base support for SSH and SCP.</font><br><font size=2 face="sans-serif">
</font><br><font size=2 face="sans-serif">Example :</font><br><br><font size=2 face="sans-serif">
*** Settings
***</font><br><font size=2 face="sans-serif">
Resource
openbmc_ffdc.robot</font><br><br><font size=2 face="sans-serif">
*** Test
Cases ***</font><br><font size=2 face="sans-serif">
Verify
Connection</font><br><font size=2 face="sans-serif">
[Teardown]
Log FFDC If Test Case Failed</font><br><font size=2 face="sans-serif">
Should
Be Equal testing OOPS</font><br><br><font size=2 face="sans-serif">
Current code on Gerrit for review
: </font><a href="https://gerrit.openbmc-project.xyz/#/c/49/"><font size=2 color=blue face="sans-serif">https://gerrit.openbmc-project.xyz/#/c/49/</font></a><br><br><font size=2 face="sans-serif">output : </font><br><font size=2 face="sans-serif">
==============================================================================</font><br><font size=2 face="sans-serif">
Test Sample</font><br><font size=2 face="sans-serif">
==============================================================================</font><br><font size=2 face="sans-serif">
Verify
Connection
F </font><br><font size=2 face="sans-serif">
FFDC Collection
Started : 2016-07-21 01:41:14,252163</font><br><font size=2 face="sans-serif">
FFDC Collection
Completed : 2016-07-21 01:41:24,062357</font><br><font size=2 face="sans-serif">
Verify
Connection
| FAIL |</font><br><font size=2 face="sans-serif">
testing
!= OOPS</font><br><font size=2 face="sans-serif">
------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">
Test Sample
| FAIL |</font><br><font size=2 face="sans-serif">
1 critical
test, 0 passed, 1 failed</font><br><font size=2 face="sans-serif">
1 test
total, 0 passed, 1 failed</font><br><font size=2 face="sans-serif">
==============================================================================</font><br><br><font size=2 face="sans-serif">default ffdc list construct :</font><br><br><font size=2 face="sans-serif">#-----------------------------------------------------------------</font><br><font size=2 face="sans-serif">#Dict Name { Index string : {
Key String : Command string} }</font><br><font size=2 face="sans-serif">#-----------------------------------------------------------------</font><br><font size=2 face="sans-serif">FFDC_CMD = {</font><br><font size=2 face="sans-serif">
'DRIVER INFO' :</font><br><font size=2 face="sans-serif">
{</font><br><font size=2 face="sans-serif">
'FW Level' : 'cat /etc/os-release',</font><br><font size=2 face="sans-serif">
'OS Details' : 'uname -a',</font><br><font size=2 face="sans-serif">
'Build Info' : 'cat /etc/version',</font><br><font size=2 face="sans-serif">
},</font><br><font size=2 face="sans-serif">
'BMC DATA' :</font><br><font size=2 face="sans-serif">
{</font><br><font size=2 face="sans-serif">
'System journal log' : 'journalctl --no-pager',</font><br><font size=2 face="sans-serif">
'Displays processor activity' : 'top -n 1 -b',</font><br><font size=2 face="sans-serif">
},</font><br><font size=2 face="sans-serif">
'APPLICATION DATA' :</font><br><font size=2 face="sans-serif">
{</font><br><font size=2 face="sans-serif">
'BMC state' : '/usr/sbin/obmcutil state',</font><br><font size=2 face="sans-serif">
},</font><br><font size=2 face="sans-serif"> }</font><br><br><br><font size=2 face="sans-serif">FFDC o/p :</font><br><br><font size=2 face="sans-serif">$ cat logs/TestSample/VerifyConnection/openbmc_ffdc_report.txt</font><br><br><font size=2 face="sans-serif">----------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">
OPEN BMC TEST FAILURE DATA CAPTURE</font><br><font size=2 face="sans-serif">
----------------------------------------------------------------------</font><br><br><font size=2 face="sans-serif">TEST SUITE FILE
: Test Sample</font><br><font size=2 face="sans-serif">TEST CASE NAME
: Verify Connection</font><br><font size=2 face="sans-serif">FAILURE TIME STAMP :
2016-07-21 01:41:14,275235</font><br><br><font size=2 face="sans-serif">This document contains the following
information:</font><br><font size=2 face="sans-serif">* APPLICATION DATA</font><br><font size=2 face="sans-serif">* DRIVER INFO</font><br><font size=2 face="sans-serif">* BMC DATA</font><br><font size=2 face="sans-serif">-----------------------------------------------------------------------------------------------------</font><br><br><font size=2 face="sans-serif">
[ Detailed Logs Captured Section ]</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">APPLICATION DATA : BMC state
Executed : /usr/sbin/obmcutil state</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif"> = BMC_STARTING</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">DRIVER INFO : Build Info
Executed : cat /etc/version</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">20160716161753</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">DRIVER INFO : OS Details
Executed : uname -a</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">Linux barreleye 4.6.4-openbmc-20160713-1
#1 Wed Jul 13 16:38:10 UTC 2016 armv5tejl GNU/Linux</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">DRIVER INFO : FW Level Executed
: cat /etc/os-release</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">ID="openbmc-phosphor"</font><br><font size=2 face="sans-serif">NAME="Phosphor OpenBMC (Phosphor
OpenBMC Project Reference Distro)"</font><br><font size=2 face="sans-serif">VERSION="0.1.0 (krogoth)"</font><br><font size=2 face="sans-serif">VERSION_ID="v1.0-89-g5bc571e-dirty"</font><br><font size=2 face="sans-serif">PRETTY_NAME="Phosphor OpenBMC (Phosphor
OpenBMC Project Reference Distro) 0.1.0 (krogoth)"</font><br><font size=2 face="sans-serif">BUILD_ID="v1.0"</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">BMC DATA : System journal log
Executed : journalctl --no-pager</font><br><font size=2 face="sans-serif">--------------------------------------------------------------------------------------------------------</font><br><font size=2 face="sans-serif">-- Logs begin at Mon 2016-07-11 22:01:17
UTC, end at Tue 2016-07-12 10:45:00 UTC. --</font><br><font size=2 face="sans-serif">Jul 11 22:01:17 barreleye systemd-journald[395]:
Runtime journal (/run/log/journal/) is 3.1M, max 25.1M, 22.0M free.</font><br><font size=2 face="sans-serif">Jul 11 22:01:17 barreleye kernel: Booting
Linux on physical CPU 0x0</font><br><font size=2 face="sans-serif">Jul 11 22:01:17 barreleye kernel: Linux
version 4.6.4-openbmc-20160713-1 (jenkins@hudson) (gcc version 4.9.3 (GCC)
) #1 Wed Jul 13 16:38:10 UTC 2016</font><br><font size=2 face="sans-serif">Jul 11 22:01:17 barreleye kernel: CPU:
ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=0005317f</font><br><font size=2 face="sans-serif">...</font><br><font size=2 face="sans-serif">....</font><br><br><font size=2 face="sans-serif">
</font><br><br><font size=2 face="sans-serif">FFDC Wish List :</font><br><br><font size=2 face="sans-serif">For now we are delivering only for open
BMC. The base infrastructure is kept generic and simple to extend for Open
Power test framework once the porting is complete to ROBOT framework.</font><br><br><font size=2 face="sans-serif"> *
Wish list from Open BMC</font><br><font size=2 face="sans-serif">
- journalctl --no-pager</font><br><font size=2 face="sans-serif">
- bmc fw level (version)</font><br><br><font size=2 face="sans-serif"> *
Wish list from Open power</font><br><font size=2 face="sans-serif">
- sel clear/list</font><br><font size=2 face="sans-serif">
- sensor list</font><br><font size=2 face="sans-serif">
- Inventory</font><br><font size=2 face="sans-serif">
- SOL output</font><br><font size=2 face="sans-serif">
- Driver Name</font><br><font size=2 face="sans-serif">
- System ID (MTMS)</font><br><font size=2 face="sans-serif">
- System Type</font><br><font size=2 face="sans-serif">
- esel support (parsing) </font><br><br><br><br><font size=2 color=blue face="sans-serif"><b>Thanks and Regards,</b></font><br><font size=2 color=blue face="sans-serif">George Keishing</font><br><font size=2 color=blue face="sans-serif">IBM Systems &Technology
Lab, Firmware Development,</font><br><font size=2 face="sans-serif"><i>“</i></font><font size=3>There isn't
enough time in a day to be lazy!!! </font><font size=1 face="sans-serif">.”</font><br><BR>