A little patch fix dev_read to make it works with large file

Gao Xiang hsiangkao at linux.alibaba.com
Fri Oct 21 13:22:43 AEDT 2022


Hi Linxuan,

I still cannot apply this patch since the patch itself is corrupted.

Could you check out the submitting process
https://www.kernel.org/doc/html/latest/process/submitting-patches.html

And as a reference, could you also check out another
https://lore.kernel.org/linux-erofs/20221013040011.31944-1-zbestahu@gmail.com/
erofs-utils patch compared to your patch here?
https://lore.kernel.org/linux-erofs/35DADC97C92E6537+142bad0a-97d1-5327-7188-d26c330ef061@uniontech.com/

And some minor comments as below:

On Fri, Oct 21, 2022 at 09:37:57AM +0800, chenlinxuan wrote:
> From 83e965dc6ec45e5c9811a27023c9cc213d50816b Mon Sep 17 00:00:00 2001
> From: Chen Linxuan <chenlinxuan at uniontech.com>
> Date: Thu, 20 Oct 2022 17:53:03 +0800
> Subject: [PATCH] erofs-utils: erofs-utils: lib: fix dev_read
> 
> When using `fsck.erofs` to extract some image have a very large file
> inside, for example 2G, my fsck.erofs report some thing like this:
> 
> <E> erofs_io: Failed to read data from device - erofs.image:[4096,
> 2147483648].
> <E> erofs: failed to read data of m_pa 4096, m_plen 2147483648 @ nid 40: -17
> <E> erofs: Failed to extract filesystem
> 
> You can use this script to reproduce this issue.
> 
> mkdir tmp extract
> dd if=/dev/urandom of=tmp/big_file bs=1M count=2048
> 
> mkfs.erofs erofs.image tmp
> fsck.erofs erofs.image --extract=extract
> 
> I found that dev_open will failed if we can not get all data we want
> with one pread call.
> 
> I write this little patch try to fix this issue.
> 
> Signed-off-by: Chen Linxuan <chenlinxuan at uniontech.com>
> ---
>  lib/io.c | 28 +++++++++++++++++++++-------
>  1 file changed, 21 insertions(+), 7 deletions(-)
> 
> diff --git a/lib/io.c b/lib/io.c
> index 524cfb4..5cb2b3a 100644
> --- a/lib/io.c
> +++ b/lib/io.c
> @@ -256,7 +256,7 @@ int dev_resize(unsigned int blocks)
> 
>  int dev_read(int device_id, void *buf, u64 offset, size_t len)
>  {
> -    int ret, fd;
> +    int read_count, fd;
> 
>      if (cfg.c_dry_run)
>          return 0;
> @@ -278,15 +278,29 @@ int dev_read(int device_id, void *buf, u64 offset,
> size_t len)

Here indicates that the patch itself is corrupted.

>          fd = erofs_blobfd[device_id - 1];
>      }
> 
> +    while (len > 0) {
>  #ifdef HAVE_PREAD64
> -    ret = pread64(fd, buf, len, (off64_t)offset);
> +        read_count = pread64(fd, buf, len, (off64_t)offset);
>  #else
> -    ret = pread(fd, buf, len, (off_t)offset);
> +        read_count = pread(fd, buf, len, (off_t)offset);
>  #endif
> -    if (ret != (int)len) {
> -        erofs_err("Failed to read data from device - %s:[%" PRIu64 ",
> %zd].",
> -              erofs_devname, offset, len);
> -        return -errno;
> +        if (read_count == -1 || read_count == 0) {
> +            if (errno) {
> +                erofs_err("Failed to read data from device - "
> +                      "%s:[%" PRIu64 ", %zd].",

It'd be better not to separate the print line for easy grep.

> +                      erofs_devname, offset, len);
> +                return -errno;
> +            } else {
> +                erofs_err("Reach EOF of device - "
> +                      "%s:[%" PRIu64 ", %zd].",

same here.


Thanks,
Gao Xiang


More information about the Linux-erofs mailing list