fuse returns ENOENT to openat() for symlink probabilistically

[Gao Xiang]

Hi Weiwen,

(add Jianan in case that he has some interest as well)

On Thu, Jan 21, 2021 at 06:12:33PM +0800, 胡玮文 wrote:
> Hi all,
> 
> I'm working on setting up CI service to run tests automatically. Now I have got
> tests with kernel mount succeeded. But some tests with fuse fails
> probabilistically. Here are my discoveries:
>

If you have some interest, it would be better to add '-d' to
erofsfuse to print the debug message when reproduing that,
if that is not enough we might need to add more debug messages
(just using fprintf is enough since it's MT-safe.)

I will checked this case this weekend as well.

> * if I run fssum in tests/src from experimental-tests branch multiple times, it
> returns different checksums for the same image and same erofsfuse process.
> 
> * if I run "diff -r" on the source and the mounted directories, all file
> content matches. but sometimes, diff reports "diff:
> test-mount/lib/.libs/liberofs.la: No such file or directory". This file is a
> symlink to "../liberofs.la". Then I use strace to confirm that openat() system
> call to this path returned ENOENT incorrectly. strace outputs:
> 
> openat(AT_FDCWD, "test-mount/lib/.libs/liberofs.la", O_RDONLY) = -1 ENOENT (No such file or directory)
> 
> * However, If I just do "cat test-mount/lib/.libs/liberofs.la" several hundreds
> of times, I cannot trigger this issue.
> 
> * I can reproduce this on both compressed and uncompressed images.
> 
> There seems a race condition, but I cannot figure it out. I'm not familiar with
> fuse. But I would like to debug further if someone can provide me any advice or
> guidance.

Not sure if this is a race condition (or not), since currently erofsfuse
itself is implemented statelessly (so no need to add more MT-safe protect
in principle). I'm afraid that could be something wrong somewhere (which
seems somewhat as stale VFS negative dentries, yet not sure why it behaves
as this...)

Thanks,
Gao Xiang

> 
> Thanks,
> Hu Weiwen
>