[OpenBMC] [Linux-5.10] useradd: cannot open /etc/passwd

[CS20 CHLi30]

Hi All,

We met this issue symptom after upgrading Linux kernel to 5.10.xx.
However, we didn't meet this issue at kernel 5.4 and 5.8.

And, we had run QEMU with latest palmetto 5.10.23 image, issue symptom also can be repo it.
Is there anyone meet this issue at Linux kernel 5.10.xx? From journal log, we didn't find any obvious log about this symptom.
This issue is easy to repo and 100%, you can follow below repo step on your OpenBMC project to repo it. Thanks.

Symptom:
==============================================================================
Ipmi.Test Ipmi User :: Test suite for OpenBMC IPMI user management.
==============================================================================
Verify IPMI User Summary :: Verify IPMI maximum supported IPMI use...
| FAIL |
Set User Name command failed (user 9, name HgekZnCI): Unspecified error: 1 != 0

Journal Log:
olympus-nuvoton kernel: Linux version 5.10.14-711b69d-dirty-08f25ae (oe-user at oe-host)
Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[1409]: useradd: cannot open /etc/passwd
Mar 18 14:42:26 olympus-nuvoton useradd[1409]: failed adding user 'HgekZnCI', data deleted
Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[288]: The operation failed internally.
Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[288]: Unable to create new user
Mar 18 14:42:26 olympus-nuvoton phosphor-user-manager[288]: The operation failed internally.

How to repo:
1. Run robot -t Verify_IPMI_User_Summary ipmi/test_ipmi_user.robot or
2. Execute useradd command to add user in our Olympus-nuvoton runbmc device.
  root at olympus-nuvoton:/# useradd timlee
  useradd: cannot open /etc/passwd

QEMU for plametto test result:
root at palmetto:~# cat /proc/version
Linux version 5.10.23-6687842 (oe-user at oe-host) (arm-openbmc-linux-gnueabi-gcc (GCC) 10.2.0, GNU ld (GNU Binutils) 2.36.1.20210209) #1 Sun Mar 28 20:54:20 UTC 2021
root at palmetto:~# useradd timlee
useradd: cannot open /etc/passwd

Debug:
We had added more debug log in overlayfs driver, but seems all overlayfs function call normally without error code return.
Here is the part of log after executing useradd command.

root at olympus-nuvoton:/# useradd timlee66
Mar 24 05:21:41 olympus-nuvoton kernel: overlayfs: open(a6b0076c[sbin/useradd/l], 0400040) -> (581ac127, 0401400040)
Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs: open(c2f826cb[etc/nsswitch.conf/l], 00) -> (f971c00e, 0401000000)
Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs: open(6fc219b2[etc/ld.so.cache/l], 0400000) -> (0d21569c, 0401400000)
Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs: open(48a47202[lib/libnss_files-2.32.so/l], 0400000) -> (ee6b0bfe, 0401400000)
Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs: open(3c1f263f[etc/passwd/l], 00) -> (b7939c0b, 0401000000)
Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs: open(176fd21b[etc/ld.so.cache/l], 0400000) -> (d2e6be78, 0401400000)
Mar 24 05:21:46 olympus-nuvoton kernel: overlayfs: open(607e8d70[lib/libnss_ldap.so.2/l], 0400000) -> (57f4f2f1, 0401400000)
Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: create(etc/passwd.985, 0100600) = 0
Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: open(c46e5b8b[etc/passwd.985/u], 0400001) -> (ef6449e9, 0401400001)
Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: link(etc/passwd.985, etc/passwd.lock) = 0
Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: create(work/#1b, 0100000) = 0
Mar 24 05:21:52 olympus-nuvoton kernel: overlayfs: unlink(work/#1b) = 0
Mar 24 05:21:52 olympus-nuvoton useradd[985]: failed adding user 'timlee', data deleted
useradd: cannot open /etc/passwd

Best regards,
Tim
________________________________
The privileged confidential information contained in this email is intended for use only by the addressees as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for delivery of the email to such a person, please kindly reply to the sender indicating this fact and delete all copies of it from your computer and network server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confidential information of Nuvoton is strictly prohibited; and any information in this email irrelevant to the official business of Nuvoton shall be deemed as neither given nor endorsed by Nuvoton.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20210329/850bcbb1/attachment-0001.htm>