Help with EROFS for specific test scenario

[Gao Xiang]

On 2024/9/22 15:24, Fred Lotter wrote:
> Hi Gao,
> 
> On Sat, Sep 21, 2024 at 1:02 AM Gao Xiang <hsiangkao at linux.alibaba.com <mailto:hsiangkao at linux.alibaba.com>> wrote:
> 
> 
>     Hi Fred,
> 
>     On 2024/9/21 00:11, Fred Lotter wrote:
>      > Hi,
>      >
>      > In the context of this page:
>      > https://erofs.docs.kernel.org/en/latest/merging.html <https://erofs.docs.kernel.org/en/latest/merging.html> <https://erofs.docs.kernel.org/en/latest/merging.html <https://erofs.docs.kernel.org/en/latest/merging.html>>
>      >
>      > I am trying to experiment with EROFS where I want to try something crazy like the following setup:
>      >
>      > /dev/mmcblk0p3:
>      > |
>      > EROFS root image
>      > |
>      > --------
>      > |
>      > EROFS second image
>      > |
>      > --------
>      >
>      > I wanted to have a primate root EROFS filesystem written at the start of a partition. Then I would like to "append" files to the immutable root EROFS filesystem, by adding a concatenated EROFS filesystem after the root filesystem, with an external device reference pointing to the root EROFS filesystem.
> 
>     Thanks for your question.
> 
> 
> Thank you for your quick assistance. My final round of questions hopefully!
> 
> 
>      >
>      > My idea in my head was then to boot the Linux kernel with something like:
>      >
>      > root=/dev/mmcblk0p3 rootfstype=erofs rootoptions=device=/dev/mmcblk0p3,offset=<root size>
>      >
>      > 1. Is it possible to have the "blobdevice" point to a complete EROFS filesystem?
> 
>     If there are two partitions, currently EROFS kernel runtime supports
>     your requirement in practice, for example:
> 
>        /dev/mmcblk0p3:
>        |
>        EROFS root image
>        |
>        -------- /dev/mmcblk0p4:
>        |
>        EROFS second image
>        |
>        --------
> 
>     You could boot with:
>     root=/dev/mmcblk0p4 rootfstype=erofs rootoptions=device=/dev/mmcblk0p3
> 
>     to get the incremental filesystem and use:
> 
>     root=/dev/mmcblk0p3 rootfstype=erofs
> 
>     to boot the original filesystem.
> 
>     The two images are both usable for booting.
> 
> 
> This is still an interesting case for me, even if it has to be two explicit partitions.
> 
> In the case of the external device pointing to a complete EROFS filesystem.
> 
> Quoting from the documentation:
> "There are no centralized inode and directory tables because they are not quite friendly for image incremental
> updates, metadata flexibility, and extensibility. It’s up to users whether inodes or directories are arranged one
> by one or not."
> 
> Does this mean with the current kernel implementation that the super erofs filesystem (containing a link to the
> external erofs) can point to the original erofs filesystem, and only discover its contents at runtime? This would
> allow me to mkfs.erofs a super erofs filesystem, without knowing anything about the external device content
> at creation time?

The incremental filesystem uses the new on-disk superblock
which is specified by users (e.g. /dev/mmcblk0p4).  But it
will reuse the old inodes or directories metadata in
/dev/mmcblk0p3 if no changes.

I'm not sure if autometicly discovery is needed, since the
original design you could just reuse /dev/mmcblk0p3 to
mount the original filesystem.  In order to
implement autometicly discovery, either you need a modified
original superblock or a full scan of all external devices..

Or if your use cases is just like squashfs incremental use
cases (you don't need to mount the original image anymore)?
but just would like to leave incremental data in an external
device?  That is also supported by EROFS on-disk format, so
I wonder your use cases though.

> 
> 
>     The `offset` option is only supported by the loop device, and parsed by
>     util-linux `mount` command, as in:
>     https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/loop.c?h=v6.11#n52 <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/loop.c?h=v6.11#n52>
> 
>     So if you'd like to use the same partition, I guess it's still possible
>     as long as you have a minimal initramfs to generate a special loop
>     device with a specific offset, like:
> 
>     losetup -ooffset=<root size> /dev/loop0 /dev/mmcblk0p3
>     mount -odevice=/dev/mmcblk0p3 /dev/loop0 /
> 
>     for example.  I'm not sure if it meets your requirement.
> 
>     In short, the on-disk format supports your requirement, and
>     kernel runtime supports your requirement too, but just with
>     two independent devices (or loop devices with `offset`).
> 
>     I'm not sure EROFS modules needs to support `offset`
>     semantics since other filesystems don't support this too.
> 
>      >
>      > 2. If yes, is there userspace support for creating this setup?
> 
>     Unfortunately, currently mkfs.erofs incremental build doesn't
>     support this mode, but it can be implemented in the next
>     erofs-utils 1.9 version.
> 
> 
> Is this a feature you plan to add in v1.9 in any case?

Yes.

> 
> 
>     Thanks,
>     Gao Xiang
> 
>      >
>      > Kind Regards,
>      > Fred
>      >
> 
> 
> I read some discussions on erofs verification:
> https://lore.kernel.org/lkml/Y69UjZP4dNYdbXW0@sol.localdomain/T/ <https://lore.kernel.org/lkml/Y69UjZP4dNYdbXW0@sol.localdomain/T/>
> 
> My impression is that dm-verity should work for EROFS filesystems today, in the same way it works with Squashfs.
> 
> However, I guess this would not work if the erofs super block device has a device table pointing to external block devices (erofs)?

I think you could just enable dm-verity seperately for
each device in advance (and check if the root hash of
each device is expected), which is transparentto EROFS.

Thanks,
Gao Xiang