[PATCH v2 REBASED] erofs-utils: introduce tarerofs

[Gao Xiang]

On 2023/7/14 14:58, Jingbo Xu wrote:
> From: Gao Xiang <hsiangkao at linux.alibaba.com>
> 
> Let's try to add a new mode "tarerofs" for mkfs.erofs.
> 
> It mainly aims at two use cases:
>   - Convert a tarball (or later tarballs with a merged view) into
>     a full EROFS image [--tar=f];
> 
>   - Generate an EROFS manifest image to reuse tar data [--tar=i],
>     which also enables EROFS 512-byte blocks.
> 
> The second use case is mainly prepared for OCI direct mount without
> OCI blob unpacking.  This also adds another `--aufs` option to
> transform aufs special files into overlayfs metadata.
> 
> [ Note that `--tar=f` generates lots of temporary files for now which
>    can impact performance since the original tar stream(s) may be
>    non-seekable. ]
> 
> Signed-off-by: Gao Xiang <hsiangkao at linux.alibaba.com>
> Signed-off-by: Jingbo Xu <jefflexu at linux.alibaba.com>
> ---
> changes:
> - rebase to origin/dev branch
> - remove commented code lines in tarerofs_parse_tar()
> ---
>   configure.ac              |   1 +
>   include/erofs/blobchunk.h |   4 +-
>   include/erofs/inode.h     |  12 +
>   include/erofs/internal.h  |   7 +-
>   include/erofs/tar.h       |  29 ++
>   include/erofs/xattr.h     |   4 +
>   lib/Makefile.am           |   3 +-
>   lib/blobchunk.c           |  47 ++-
>   lib/inode.c               | 194 ++++++---
>   lib/tar.c                 | 807 ++++++++++++++++++++++++++++++++++++++
>   lib/xattr.c               |  46 ++-
>   mkfs/main.c               | 134 +++++--
>   12 files changed, 1182 insertions(+), 106 deletions(-)
>   create mode 100644 include/erofs/tar.h
>   create mode 100644 lib/tar.c


I will apply this patch with the following diff applied too, mainly
since some random compiler (gcc 9.2.1) reports a weird warning like
below:

tar.c: In function ‘tarerofs_parse_tar’:
tar.c:697:22: warning: ‘st.st_rdev’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   697 |    inode->u.i_rdev = erofs_new_encode_dev(st.st_rdev);

but st.st_rdev is actually initialized.

Thanks,
Gao Xiang

-----------

diff --git a/lib/tar.c b/lib/tar.c
index ef45183..8edfe75 100644
--- a/lib/tar.c
+++ b/lib/tar.c
@@ -581,10 +581,12 @@ restart:
  	} else {
  		erofs_info("unrecognized typeflag %xh @ %llu - ignoring",
  			   th.typeflag, tar_offset);
+		(void)erofs_lskip(tar->fd, st.st_size);
  		ret = 0;
  		goto out;
  	}

+	st.st_rdev = 0;
  	if (S_ISBLK(st.st_mode) || S_ISCHR(st.st_mode)) {
  		int major, minor;

@@ -599,8 +601,8 @@ restart:
  			erofs_err("invalid device minor @ %llu", tar_offset);
  			goto out;
  		}
-		st.st_rdev = (major << 8) | (minor & 0xff) | ((minor & ~0xff) << 12);

+		st.st_rdev = (major << 8) | (minor & 0xff) | ((minor & ~0xff) << 12);
  	} else if (th.typeflag == '1' || th.typeflag == '2') {
  		if (!eh.link)
  			eh.link = strndup(th.linkname, sizeof(th.linkname));