[Test Automation] [Update Service] Failed: Test Firmware Tftp Upload Image.Install Same Image Two Times

[George Keishing]

Hi Tim,

    ${image_id}=  Wait Until Keyword Succeeds  180 sec  10 sec  Get Latest
Image ID
    Rprint Vars  image_id


           It looks like the image didn't get downloaded / transferred from
the TFTP server to BMC  /tmp/images/  within 3 minutes 10 seconds.  If the
TFTP server is slower or take longer time, it could lead to this failure.

           It worked when I tried, but hey its been sometime since, we will
try again and see if it working or not on the latest upstream master image.

Thanks and Regards,
   George Keishing




From:	CS20 CHLi30 <CHLI30 at nuvoton.com>
To:	Adriana Kobylak <anoo at linux.ibm.com>
Cc:	"openbmc at lists.ozlabs.org" <openbmc at lists.ozlabs.org>,
            "anoo at us.ibm.com" <anoo at us.ibm.com>, CS20 CHLi30
            <CHLI30 at nuvoton.com>
Date:	19-11-2020 12:38 PM
Subject:	[EXTERNAL] RE: [Test Automation] [Update Sevice] Failed: Test
            Firmware Tftp Upload Image.Install Same Image Two Times
Sent by:	"openbmc" <openbmc-bounces
            +gkeishin=in.ibm.com at lists.ozlabs.org>



Hi Adriana,

Thanks your prompt response. You can check more detail from the robot
script:
https://github.com/openbmc/openbmc-test-automation/blob/master/redfish/update_service/test_firmware_tftp_upload_image.robot


Test failed at Get Latest Image ID return false, due to /tmp/images/$
{image_id} didn't exist.
Below are failed log and test script for you refer it. If there is any
question, just let me know. Thank you.

[Failed log]
Keyword 'Get Latest Image ID' failed after retrying...
The last error was: **ERROR** Invalid variable value: image_id: <blank>
<str>

[Test script]
Same Firmware Install Two Times
    [Documentation]  Download same image twice to BMC via ImageURI. Second
attempt would fail.
    [Arguments]  ${apply_time}  ${tftp_server}=${TFTP_SERVER}  $
{image_file_name}=${IMAGE_FILE_NAME}

    # Description of argument(s):
    # apply_time       ApplyTime allowed values (e.g. "OnReset",
"Immediate").
    # tftp_server      Server IP.
    # image_file_name  Image file name.

    ImageURI Download Install  ${apply_time}

    # Download image from TFTP server via ImageURI to BMC.

Redfish.Post  /redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate
    ...  body={"ImageURI": "tftp://${tftp_server}/${image_file_name}"}

    ${image_version}=  Get Image Version From TFTP Server  ${tftp_server}
${image_file_name}
    ${software_inventory_record}=  Get Software Inventory State By Version
    ...  ${image_version}
    Rprint Vars  software_inventory_record

    ${image_id}=  Wait Until Keyword Succeeds  180 sec  10 sec  Get Latest
Image ID
    Rprint Vars  image_id

    Check Image Update Progress State  match_state='Enabled'  image_id=$
{image_id}
    # Check if the existing firmware is functional.
    Pass Execution If  ${software_inventory_record['functional']}
    ...  The existing ${image_version} firmware is already functional.

Get Latest Image ID
    [Documentation]  Return the ID of the most recently extracted image.
    # Note: This keyword will fail if there is no such file.

    # Example: # ls /tmp/images/
    #            1b714fb7
    ${image_id}=  Get Latest File  /tmp/images/
    Valid Value  image_id

    # Though an image sub-directory was found, it really isn't valid unless
    # the MANIFEST file is present.
    BMC Execute Command  ls -l /tmp/images/${image_id}/MANIFEST

    [Return]  ${image_id}

Best regards,
Tim
-----Original Message-----
From: Adriana Kobylak [mailto:anoo at linux.ibm.com]
Sent: Thursday, November 19, 2020 11:40 AM
To: CS20 CHLi30 <CHLI30 at nuvoton.com>
Cc: anoo at us.ibm.com; openbmc at lists.ozlabs.org
Subject: Re: [Test Automation] [Update Sevice] Failed: Test Firmware Tftp
Upload Image.Install Same Image Two Times

On 2020-11-18 19:51, CS20 CHLi30 wrote:
>
> Recently, we met 1 test item fail "Test Firmware Tftp Upload
> Image.Install Same Image Two Times" in Update Service of OpenBMC Test
> Automation.

> According your commit, I understand this is a good fix to clear no use
> image files in /tmp folder.
>
> But, it seems against test item of update service in openbmc test
> automation.
>
> The test script will check if the existing firmware is functional
> according image_id in /tmp folder, if existing then it's not allow to
> update same image two times.
>
> However, after your commit then test script will fail due to image_id
> dir be deleted. Did you have any suggestion for this symptom?

Hi Tim,

Could you provide a bit more details about this test case, you hint that it
may exist in the openbmc-test-automation repo?

The openbmc-test-automation handles this scenario by extracting the
MANIFEST file out of the code update tarball before the upload and reading
the VERSION string, then comparing that with the VERSION in
the /etc/os-release file in the BMC to determine if the version is already
running in the BMC.

I wouldn't recommend trying to generate the id from within the test case
because the algorithm may change. So hopefully checking the version string
is an alternative that'd work.
________________________________
________________________________
 The privileged confidential information contained in this email is
intended for use only by the addressees as indicated by the original sender
of this email. If you are not the addressee indicated in this email or are
not responsible for delivery of the email to such a person, please kindly
reply to the sender indicating this fact and delete all copies of it from
your computer and network server immediately. Your cooperation is highly
appreciated. It is advised that any unauthorized use of confidential
information of Nuvoton is strictly prohibited; and any information in this
email irrelevant to the official business of Nuvoton shall be deemed as
neither given nor endorsed by Nuvoton.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20201119/978e0bfb/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20201119/978e0bfb/attachment-0001.gif>