[OpenBMC] [Linux-5.10] useradd: cannot open /etc/passwd

Tue Mar 30 16:47:08 AEDT 2021

Hi Lei Yu,

Thanks your help to provide this information.
This issue seems relate to "cannot do write action for /etc/passwd file".
I'm trying to check whether is relate to jffs2 or overlayfs filesystem driver.
If you have any suggestion or finding, please let me know, thank you very much. BTW, I cannot see any discussion on [discord][1], is it open?

open action for /etc/passwd file without problem, but when doing "touch" or "useradd" this kind of writing action then got issue symptom.
Thus, this issue's root cause should not relate to "utimensat_time64()" function.
Due to "useradd" got same fail symptom, but it's not call this function according the result about strace.
Below are strace result on kernel 5.4 and 5.10 for you refer it. Both are using the same version useradd daemon to test it.

Kernel 5.10:
root at olympus-nuvoton:/# strace useradd timlee
openat(AT_FDCWD, "/etc/passwd", O_RDWR|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW) = -1 EINVAL (Invalid argument)
write(2, "useradd: cannot open /etc/passwd"..., 33useradd: cannot open /etc/passwd
) = 33

Kernel 5.4:
root at olympus-nuvoton:/# strace useradd timlee
openat(AT_FDCWD, "/etc/passwd", O_RDWR|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW) = 4

Best regards,
Tim

-----Original Message-----
From: Lei Yu [mailto:yulei.sh at bytedance.com]
Sent: Tuesday, March 30, 2021 10:21 AM
To: CS20 CHLi30 <CHLI30 at nuvoton.com>
Cc: openbmc at lists.ozlabs.org
Subject: Re: [OpenBMC] [Linux-5.10] useradd: cannot open /etc/passwd

As discussed in [discord][1], a similar issue could be simply reproduced by:
`touch /etc/passwd`, that gives `Invalid argument`.

With strace, the error is from `utimensat_time64()`

    # ./strace touch /etc/passwd
    execve("/bin/touch", ["touch", "/etc/passwd"], 0x7eef7dd4 /* 16 vars */) = 0
    ......
    getuid32()                              = 0
    utimensat_time64(AT_FDCWD, "/etc/passwd", NULL, 0) = -1 EINVAL (Invalid argument)
    write(2, "touch: /etc/passwd: Invalid argu"..., 37touch:
/etc/passwd: Invalid argument
    ) = 37
    exit_group(1)

[1]: https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscord.com%2Fchannels%2F775381525260664832%2F775694683589574659%2F826065967582281768&data=04%7C01%7CCHLI30%40nuvoton.com%7C09540457f4be483dc75108d8f3227ffd%7Ca3f24931d4034b4a94f17d83ac638e07%7C0%7C0%7C637526676785951652%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Ju0bjtNK%2BAVlMVtFILficzZhK6%2BC74AiVujO%2FvbjCyE%3D&reserved=0

On Mon, Mar 29, 2021 at 2:18 PM CS20 CHLi30 <CHLI30 at nuvoton.com> wrote:
>
> Hi All,
>
>
>
> We met this issue symptom after upgrading Linux kernel to 5.10.xx.
>
> However, we didn't meet this issue at kernel 5.4 and 5.8.
>
>
>
> And, we had run QEMU with latest palmetto 5.10.23 image, issue symptom also can be repo it.
>
> Is there anyone meet this issue at Linux kernel 5.10.xx? From journal log, we didn't find any obvious log about this symptom.
>
> This issue is easy to repo and 100%, you can follow below repo step on your OpenBMC project to repo it. Thanks.
>
>
>
> Symptom:
>
> ======================================================================
> ========
>
> Ipmi.Test Ipmi User :: Test suite for OpenBMC IPMI user management.
>
> ======================================================================
> ========
>
> Verify IPMI User Summary :: Verify IPMI maximum supported IPMI use...
>
> | FAIL |
>
> Set User Name command failed (user 9, name HgekZnCI): Unspecified
> error: 1 != 0
>
>
>
> Journal Log:
>
> olympus-nuvoton kernel: Linux version 5.10.14-711b69d-dirty-08f25ae
> (oe-user at oe-host)
>
> Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[1409]: useradd:
> cannot open /etc/passwd
>
> Mar 18 14:42:26 olympus-nuvoton useradd[1409]: failed adding user
> 'HgekZnCI', data deleted
>
> Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[288]: The operation failed internally.
>
> Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[288]: Unable to
> create new user
>
> Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[288]: The operation failed internally.
>
>
>
> How to repo:
>
> 1. Run robot -t Verify_IPMI_User_Summary ipmi/test_ipmi_user.robot or
>
> 2. Execute useradd command to add user in our Olympus-nuvoton runbmc device.
>
>   root at olympus-nuvoton:/# useradd timlee
>
>   useradd: cannot open /etc/passwd
>
>
>
> QEMU for plametto test result:
>
> root at palmetto:~# cat /proc/version
>
> Linux version 5.10.23-6687842 (oe-user at oe-host)
> (arm-openbmc-linux-gnueabi-gcc (GCC) 10.2.0, GNU ld (GNU Binutils)
> 2.36.1.20210209) #1 Sun Mar 28 20:54:20 UTC 2021
>
> root at palmetto:~# useradd timlee
>
> useradd: cannot open /etc/passwd
>
>
>
> Debug:
>
> We had added more debug log in overlayfs driver, but seems all overlayfs function call normally without error code return.
>
> Here is the part of log after executing useradd command.
>
>
>
> root at olympus-nuvoton:/# useradd timlee66
>
> Mar 24 05:21:41 olympus-nuvoton kernel: overlayfs:
> open(a6b0076c[sbin/useradd/l], 0400040) -> (581ac127, 0401400040)
>
> Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs:
> open(c2f826cb[etc/nsswitch.conf/l], 00) -> (f971c00e, 0401000000)
>
> Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs:
> open(6fc219b2[etc/ld.so.cache/l], 0400000) -> (0d21569c, 0401400000)
>
> Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs:
> open(48a47202[lib/libnss_files-2.32.so/l], 0400000) -> (ee6b0bfe,
> 0401400000)
>
> Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs:
> open(3c1f263f[etc/passwd/l], 00) -> (b7939c0b, 0401000000)
>
> Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs:
> open(176fd21b[etc/ld.so.cache/l], 0400000) -> (d2e6be78, 0401400000)
>
> Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs:
> open(607e8d70[lib/libnss_ldap.so.2/l], 0400000) -> (57f4f2f1,
> 0401400000)
>
> Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs:
> create(etc/passwd.985, 0100600) = 0
>
> Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs:
> open(c46e5b8b[etc/passwd.985/u], 0400001) -> (ef6449e9, 0401400001)
>
> Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs:
> link(etc/passwd.985, etc/passwd.lock) = 0
>
> Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: create(work/#1b,
> 0100000) = 0
>
> Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: unlink(work/#1b) =
> 0
>
> Mar 24 05:21:52 olympus-nuvoton useradd[985]: failed adding user
> 'timlee', data deleted
>
> useradd: cannot open /etc/passwd
>
>
>
> Best regards,
>
> Tim
>
> ________________________________
> The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Nuvoton is strictly prohibited; and any information in this email irrelevant to the official business of Nuvoton shall be deemed as neither given nor endorsed by Nuvoton.

--
BRs,
Lei YU
________________________________
________________________________
 The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Nuvoton is strictly prohibited; and any information in this email irrelevant to the official business of Nuvoton shall be deemed as neither given nor endorsed by Nuvoton.