A little patch fix dev_read to make it works with large file

chenlinxuan chenlinxuan at uniontech.com
Thu Oct 20 19:12:57 AEDT 2022


When using `fsck.erofs` to extract some image have a very large file 
(3G) inside, my fsck.erofs report some thing like:

<E> erofs_io: Failed to read data from device - erofs.image:[4096, 
2147483648].
<E> erofs: failed to read data of m_pa 4096, m_plen 2147483648 @ nid 40: -17
<E> erofs: Failed to extract filesystem

You can use this script to reproduce this issue.

#!/bin/env sh

mkdir tmp extract
dd if=/dev/urandom of=tmp/big_file bs=1M count=2048

mkfs.erofs erofs.image tmp
fsck.erofs erofs.image --extract=extract

I found that dev_open will failed if we can not get all data we want 
with one pread call.

I write this little patch try to fix this issue.

This is my first patch send via email, sorry if anything goes wrong.

 From 156af5b173c1f9e2a91e4d2126214b96966babd1 Mon Sep 17 00:00:00 2001
From: chenlinxuan <chenlinxuan at uniontech.com>
Date: Thu, 20 Oct 2022 14:39:17 +0800
Subject: [PATCH] erofs-utils: lib: fix dev_read

We need to keep calling pread until we get all data we want
---
  lib/io.c | 28 +++++++++++++++++++++-------
  1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/lib/io.c b/lib/io.c
index 524cfb4..bd3d790 100644
--- a/lib/io.c
+++ b/lib/io.c
@@ -256,7 +256,7 @@ int dev_resize(unsigned int blocks)

  int dev_read(int device_id, void *buf, u64 offset, size_t len)
  {
-   int ret, fd;
+   int read_count, fd;

     if (cfg.c_dry_run)
         return 0;
@@ -278,15 +278,29 @@ int dev_read(int device_id, void *buf, u64 offset, 
size_t len)
         fd = erofs_blobfd[device_id - 1];
     }

+   while (len > 0) {
  #ifdef HAVE_PREAD64
-   ret = pread64(fd, buf, len, (off64_t)offset);
+       read_count = pread64(fd, buf, len, (off64_t)offset);
  #else
-   ret = pread(fd, buf, len, (off_t)offset);
+       read_count = pread(fd, buf, len, (off_t)offset);
  #endif
-   if (ret != (int)len) {
-       erofs_err("Failed to read data from device - %s:[%" PRIu64 ", 
%zd].",
-             erofs_devname, offset, len);
-       return -errno;
+       if (read_count == -1 || read_count ==  0) {
+           if (errno) {
+               erofs_err("Failed to read data from device - "
+                     "%s:[%" PRIu64 ", %zd].",
+                     erofs_devname, offset, len);
+               return -errno;
+           } else {
+               erofs_err("Reach EOF of device - "
+                     "%s:[%" PRIu64 ", %zd].",
+                     erofs_devname, offset, len);
+               return -EINVAL;
+           }
+       }
+
+       offset += read_count;
+       len -= read_count;
+       buf += read_count;
     }
     return 0;
  }
-- 
2.37.3



More information about the Linux-erofs mailing list